Control system, control device, control method, and program

Abstract

To ensure safety when a tactile force is transmitted to execute a predetermined action, a control system (1) includes: a master device (10) to which manipulation by a manipulator is input; and a slave device (20) that executes a predetermined action by operating in accordance with the manipulation input to the master device (10). Further, the control system (1) includes a tactile force transmission unit (352), a control unit (356), and a threshold value setting unit (355). The tactile force transmission unit (352) controls transmission of a tactile force in the master device (10) and the slave device (20). The control unit (356) controls execution of the predetermined action on the basis of a control parameter used for controlling the transmission of a tactile force by the tactile force transmission unit (352) and a threshold value corresponding to the control parameter. The threshold value setting unit (355) changes the threshold value during the execution of the predetermined action.

Description

Control system, control device, control method and program

[0001] The present invention relates to a control system, a control device, a control method, and a program.

[0002] A bilateral control technique has been known in the past, in which a master device receives input of an operator's operation and a slave device operates in response to the operation input to the master device, and a reaction force corresponding to the operation of the slave device is transmitted to the master device as a haptic sensation. Such bilateral control technique is disclosed, for example, in Patent Document 1.

[0003] Japanese Patent Application Publication No. 64-34686

[0004] In general techniques such as the technique disclosed in the above-mentioned Patent Document 1, a predetermined action can be performed by operating a slave device, and haptic sensations can be transmitted to an operator through bilateral control.

[0005] However, it is believed that there is room for further improvement in the general technology in terms of ensuring safety when a predetermined action is performed. For example, ensuring such safety is particularly important when the predetermined action is processing a workpiece that must not be damaged or a medical procedure on a living body such as a human body. In contrast, with the general technology, safety is ensured only by the operator visually inspecting an image captured near the slave device.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to ensure greater safety when a haptic sensation is transmitted to perform a predetermined action.

[0007] In order to solve the above-mentioned problems, a control system according to one aspect of the present invention includes a master device to which an operator's operation is input, and a slave device that performs a predetermined action by operating in accordance with the operation input to the master device, and is characterized by comprising: a control means for controlling the transmission of haptic sensations in the master device and the slave device; a limiting means for limiting the execution of the predetermined action based on control parameters used by the control means to control the transmission of haptic sensations and threshold values ​​corresponding to the control parameters; and a threshold setting means for changing the threshold value while the predetermined action is being performed.

[0008] According to the present invention, it is possible to ensure greater safety when transmitting haptic sensations to perform a predetermined action.

[0009] 1 is a schematic diagram showing the overall configuration of a control system 1 according to one embodiment of the present invention. FIG. 1 is a schematic diagram showing the basic principle of force haptic transmission control executed by a control device 30. FIG. 2 is a block diagram showing the hardware configuration of a control system in the control system 1. FIG. 3 is a schematic diagram showing the hardware configuration of an information processing device that constitutes the control device 30. FIG. 4 is a block diagram showing the functional configuration of the control system 1. FIG. 5 is a flowchart explaining the flow of threshold control processing executed by the control device 30. FIG. 6 is a schematic diagram showing time changes in the magnitude of the external force input to the slave device 20, the positions of the master device 10 and the slave device 20, and the position of the tip of the catheter, when puncture is performed as a predetermined action and penetration of a blood vessel occurs. FIG. 7 is a schematic diagram showing the configuration of a control system 1 that performs threshold control processing after an operator manually inserts the catheter of the slave device 20.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [Configuration] Fig. 1 is a schematic diagram showing the overall configuration of a control system 1 according to one embodiment of the present invention. As shown in Fig. 1, the control system 1 according to this embodiment is configured as a master-slave system including a master unit 10 and a slave unit 20 that are mechanically separated. As an example, in the control system 1 according to this embodiment, the master unit 10 constitutes a manipulator operated by an operator, and the slave unit 20 constitutes a catheter system equipped with an end effector to be inserted into a subject.

[0012] 1, the control system 1 includes a master device 10, a slave device 20, and a control device 30. The master device 10, the slave device 20, and the control device 30 are configured to be able to communicate with each other via a network 40, either wired or wirelessly. The control system 1 may also include a display L and multiple cameras C as appropriate. The camera C may be any of a variety of imaging devices, such as a video camera that captures the external appearance of a subject into which the slave device 20 is inserted, or an X-ray camera that captures the interior of the subject (e.g., the subject's blood vessels and organs) using X-rays. The control system 1 may also include multiple displays L that display various images captured by the multiple cameras C and various information output from the control device 30.

[0013] The master unit 10 receives operations similar to those performed on a conventional mechanically configured catheter and detects the position of a movable part (such as a movable member of a manipulator) that moves in response to the input operation. The master unit 10 transmits information indicating the detected position of the movable part to the control device 30. In response to the input operation, the master unit 10 outputs a reaction force from an actuator in accordance with instructions from the control device 30.

[0014] Specifically, the master device 10 accepts operations to advance and retract the catheter (for example, an operation to insert it into a blood vessel or an operation to slightly move it to detect force tactile sensation near a lesion), an operation to rotate the catheter around its axis (for example, an operation to change the orientation of the end effector), and an operation to operate the end effector (for example, an operation to expand or contract the end effector if the end effector is a balloon, or an operation to open or close the end effector if the end effector is a forceps or the like), and applies a reaction force to these operations while transmitting information indicating the position of the movable part moved by each operation to the control device 30.

[0015] The slave unit 20 performs an operation corresponding to the operation input to the master unit 10 by driving the actuator in accordance with instructions from the control unit 30, and detects the position of a movable part (such as a movable element of the actuator or a catheter moved by the actuator) that moves as a result of the operation. As the slave unit 20 operates, various external forces are input to the slave unit 20 from the environment. As a result, the position of the movable part in the slave unit 20 indicates the result of the various external forces acting on the output of the actuator. The slave unit 20 then transmits information indicating the detected position of the movable part to the control unit 30. Here, the various external forces input to the slave unit 20 from the environment include, for example, a thrust resistance force that a catheter inserted into a subject receives from a blood vessel, and a contact force when an end effector or the like disposed at the tip of the catheter comes into contact with a lesion, organ, or blood vessel.

[0016] The control device 30 is configured by an information processing device such as a PC (Personal Computer) or a server computer, and controls the master device 10, the slave device 20, the display L, and the camera C. For example, the control device 30 acquires the positions of the moving parts of the master device 10 and the slave device 20 (such as the rotation angle of the actuator detected by a rotary encoder or the advance / retract position of the moving part detected by a linear encoder), and executes control to transmit haptic sensations between the master device 10 and the slave device 20.

[0017] When the master unit 10 and the slave unit 20 operate as a master-slave system, the control device 30 in this embodiment performs coordinate transformation (transformation using a transformation matrix) of real-space parameters (input vectors) calculated based on information representing the position of a movable part (information representing the position of a movable element of an actuator or the position of a member moved by an actuator) into a virtual space in which position and force can be handled independently. That is, the input vector is coordinate-transformed from the real space of an oblique coordinate system in which position and force are related to each other to the virtual space of an orthogonal coordinate system in which position and force are independent of each other. The parameters calculated by the coordinate transformation represent state values ​​of the position and force corresponding to the input vector in the virtual space. Then, in the virtual space after the coordinate transformation, the control device 30 performs calculations to cause the state values ​​of the position and force calculated from the input vector to follow target values ​​for the position and force for controlling the position and force (here, transmitting haptic sensations), and then performs inverse transformation (transformation using the inverse matrix of the transformation matrix) to return the calculation results to the real space. Furthermore, the control device 30 realizes a master-slave system that transmits haptic sensations between the master device 10 and the slave device 20 by driving each actuator based on real-space parameters (such as current command values) acquired by inverse transformation.

[0018] In addition, since position and velocity (or acceleration) or angle and angular velocity (or angular acceleration) are parameters that can be replaced by differential and integral calculations, when performing processing related to position or angle, they can be replaced with velocity or angular velocity, etc. as appropriate.

[0019] In this configuration, the control system 1 of this embodiment realizes a master-slave system that transmits haptics between the master device 10 and the slave device 20 as described above, and also performs threshold control processing. Here, the threshold control processing is a series of processes that, when transmitting haptics to perform a predetermined action, restricts the execution of the predetermined action at an appropriate timing by making a judgment based on a threshold, thereby ensuring safety. Specifically, in the threshold control processing, the control system 1 controls the transmission of haptics between the master device 10 and the slave device 20. Furthermore, the control system 1 restricts the execution of the predetermined action based on control parameters used to control the transmission of haptics and thresholds corresponding to the control parameters. Furthermore, the control system 1 changes the threshold while the predetermined action is being performed.

[0020] In this way, the control system 1 determines whether to restrict the execution of a predetermined action based on the control parameters used to transmit haptics. That is, by utilizing the system's underlying configuration of transmitting haptics, the control system 1 can restrict the execution of a predetermined action at the appropriate timing and ensure safety. This allows for more accurate detection of the appropriate timing than common methods, such as having an operator perform an action while visually viewing an image of the subject or transmitting an approximate force calculated based on the torque current of a motor to the operator. In addition, the control system 1 does not uniformly determine a threshold value serving as a judgment criterion, but can dynamically change the threshold value to a more appropriate one during the execution of the predetermined action, for example, depending on the execution status of the predetermined action. Therefore, the control system 1 can solve the problem of ensuring safety when transmitting haptics to perform a predetermined action.

[0021] Figure 2 is a schematic diagram showing the basic principle of force haptic transmission control executed by the control device 30. The basic principle shown in Figure 2 determines the operation of the actuator by performing calculations in at least one of the velocity and force domains using information representing the position of the movable part (the current position of the movable part) as an input. In other words, the basic principle of the present invention is expressed as a control law including a controlled system S, a functional force-velocity allocation conversion block FT, at least one of an ideal force source block FC or an ideal velocity source block PC, and an inverse conversion block IFT.

[0022] The controlled system S is a master device 10 or a slave device 20 equipped with an actuator, and controls the actuator based on acceleration, etc. As described above, acceleration, velocity, and position are physical quantities that can be converted into one another using differential and integral calculus, and therefore, control may be performed using any of acceleration, velocity, and position. Here, the control law is primarily expressed using velocity calculated from position.

[0023] The functional force-speed allocation transformation block FT is a block that defines the transformation of control energy into speed and force ranges that are set according to the function of the controlled system S. Specifically, the functional force-speed allocation transformation block FT defines a coordinate transformation that takes as input a reference value (reference value) for the function of the controlled system S and the current position of the movable part. This coordinate transformation generally transforms an input vector having the reference value and current speed as elements into an output vector consisting of speed for calculating a target value for speed control, and also transforms an input vector having the reference value and current force as elements into an output vector consisting of force for calculating a target value for force control. Specifically, the coordinate transformation in the functional force-speed allocation transformation block FT is generalized and expressed as the following equations (1) and (2):

[0024]

[0025] However, in formula (1), x'1 to x' n (n is an integer equal to or greater than 1) is a velocity vector for deriving the velocity state value, and x' a ~x' m(m is an integer of 1 or more) is a vector whose elements are a reference value and a velocity based on the action of the actuator (the velocity of the movable element of the actuator or the velocity of the member moved by the actuator), 1a ~h nm are elements of the transformation matrix that represents the function. n (n is an integer equal to or greater than 1) is a force vector for deriving a force state value, and f'' a ~f'' m (m is an integer of 1 or more) is a vector whose elements are a reference value and a force based on the action of the actuator (the force of the mover of the actuator or the force of the member moved by the actuator).

[0026] By setting the coordinate transformation in the functional force / speed allocation transformation block FT according to the function to be realized, various operations can be realized and scaling can be performed. That is, according to the basic principle of the present invention, the functional force / speed allocation transformation block FT "converts" the variables of a single actuator (variables in real space) into a group of variables of the entire system (variables in virtual space) that represent the function to be realized, and allocates control energy to the velocity control energy and the force control energy. In other words, according to the basic principle of the present invention, calculations related to the control of velocity and force are performed after converting from a coordinate space in which velocity and force are related to each other to a coordinate space in which velocity and force are independent of each other. Therefore, compared to when control is performed using the variables of a single actuator (variables in real space), it is possible to independently assign the velocity control energy and the force control energy.

[0027] The ideal force source block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the functional force-speed allocation transformation block FT. In the ideal force source block FC, a target value for force is set when performing calculations based on the coordinate transformation defined by the functional force-speed allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be realized can be set.

[0028] The ideal speed source block PC is a block that performs calculations in the speed domain according to the coordinate transformation defined by the functional force-speed allocation transformation block FT. In the ideal speed source block PC, a target value for the speed when performing calculations based on the coordinate transformation defined by the functional force-speed allocation transformation block FT is set. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be realized can be set.

[0029] The inverse transformation block IFT is a block that converts values ​​in the velocity and force domains into values ​​in the domain of input to the controlled system S (for example, voltage values ​​or current values). According to this basic principle, when position information of the actuators of the controlled system S is input to the function-specific force / velocity allocation transformation block FT, the function-specific force / velocity allocation transformation block FT applies control laws for the position and force domains according to the function using velocity and force information obtained based on the position information. Then, the ideal force source block FC calculates the force according to the function, and the ideal velocity source block PC calculates the velocity according to the function, and control energy is distributed to each of the force and velocity.

[0030] The calculation results in the ideal force source block FC and the ideal velocity source block PC become information indicating the control target of the controlled system S, and these calculation results are used as input values ​​for the actuators in the inverse transformation block IFT and input to the controlled system S. As a result, the actuators of the controlled system S perform operations in accordance with the functions defined by the functional force / velocity allocation transformation block FT, and the desired device operation is realized.

[0031] Furthermore, when a force-tactile transmission function involving scaling (amplification or reduction of force or position) is realized, the coordinate transformation in the functional force-velocity allocation transformation block FT in FIG. 2 is expressed as the following equations (3) and (4).

[0032]

[0033] However, in formula (3), x' p is the velocity for deriving the velocity state value, x' f is the velocity related to the state value of the force. m is the speed (differential value of the current position of the master unit 10) of the reference value (input from the master unit 10), and x' s is the current velocity of the slave unit 20 (the differential value of the current position). p is the force related to the velocity state value, f f is the force for deriving the state value of the force. m is the reference force (input from the master device 10), f s is the current force of the slave unit 20.

[0034] When the coordinate transformations shown in Equations (3) and (4) are performed, the position of the slave device 20 is multiplied by α (α is a positive number), and the force of the slave device 20 is multiplied by β (β is a positive number) before being transmitted to the master device 10. For example, by setting α = 1 and β = 1, the haptic sensation is transmitted without being amplified (i.e., enlarged) or attenuated (i.e., reduced). On the other hand, by setting the values ​​of α and β according to the purpose, scaling can be achieved, in which the transmitted haptic sensation is amplified (i.e., enlarged) or attenuated (i.e., reduced). The force-haptic transmission function with such scaling can restrict the execution of a predetermined action by suppressing the operation of the slave device 20 by, for example, significantly reducing the haptic sensation transmitted from the master device 10 when performing the threshold control process.

[0035] Furthermore, when a force-tactile transmission function accompanied by positional restrictions is realized, the coordinate transformation in the functional force-velocity allocation transformation block FT in Figure 2 is expressed as the following equations (5) to (8). When realizing such a function, it is appropriate to consider the following conditions: - Continuity up to the velocity dimension (condition for the existence of the Jacobian matrix) - The position after restriction is a monotonically increasing function of the original position (condition for stability) - x s <x when a s = x shat Or x s ≒ x shat (x shat is a parameter included in the functional force / speed allocation conversion block FT in equations (7) and (8) (condition for guaranteeing control performance in the safety region) - It is a saturation function (condition for realizing the position limit) It is also possible to adopt an atan function as another function that satisfies these conditions.

[0036]

[0037] When the coordinate transformations shown in equations (5) to (8) are performed, if the position of the slave device 20 is less than a, the slave device 20 and the master device 19 are controlled to the same position by applying the coordinate transformations of equations (5) and (6). On the other hand, if the position of the slave device 20 is equal to or greater than a, a scaling function is activated by applying the coordinate transformations of equations (7) and (8), and the slave device 20 is controlled so as not to exceed the position of (1 / b + a), regardless of the operator's operation input to the master device 10. Such a force / tactile transmission function with positional limitations makes it possible to restrict the execution of a predetermined action, for example, by restricting the movement of the slave device 20 by limiting its position when executing a threshold control process.

[0038] [Hardware Configuration] Next, a hardware configuration of a control system in the control system 1 will be described. Fig. 3 is a block diagram showing the hardware configuration of the control system in the control system 1. As shown in Fig. 3, the control system 1 includes, as the hardware configuration of the control system, a control device 30 configured by an information processing device such as a PC or a server computer, a control unit 101 of the master device 10, a communication unit 102, an insertion actuator 103, a detection actuator 104, a rotation actuator 105, an operation actuator 106, linear encoders 107 and 108, rotary encoders 109 and 110, and drivers 111 to 114, a control unit 201 of the slave device 20, a communication unit 202, an insertion actuator 203, a detection actuator 204, a rotation actuator 205, an operation actuator 206, linear encoders 207 and 208, rotary encoders 209 and 210, and drivers 211 to 214, a display L, and a camera C.

[0039] The control unit 101 of the master device 10 is configured by a microcomputer equipped with a processor, memory, etc., and controls the operation of the master device 10. For example, the control unit 101 controls the driving of the insertion actuator 103, detection actuator 104, rotation actuator 105, and operation actuator 106 of the master device 10 in accordance with control parameters transmitted from the control device 30. The communication unit 102 controls communication between the master device 10 and other devices via the network 40.

[0040] The insertion actuator 103 is configured, for example, by a linear motor and applies a reaction force to an operation of advancing and retracting the catheter to insert it into a blood vessel, which the operator inputs into the master device 10 according to instructions from the control unit 101. The detection actuator 104 is configured, for example, by a voice coil motor and applies a reaction force to an operation of advancing and retracting the catheter to treat a lesion, which the operator inputs into the master device 10 according to instructions from the control unit 101. In this embodiment, the insertion actuator 103 has a longer stroke than the detection actuator 104, but the detection actuator 104 can control position and force with higher precision than the insertion actuator 103. The rotation actuator 105 is configured, for example, by a rotary motor and applies a reaction force to an operation of rotating the master device 10 around a rotation axis along the advancing and retracting direction, which the operator inputs into the master device 10 according to instructions from the control unit 101. The operation actuator 106 is composed of, for example, a rotary motor, and applies a reaction force to the operation input by the operator to a lever (grip) or the like to operate the end effector in accordance with instructions from the control unit 101.

[0041] The linear encoder 107 detects the position of the mover of the insertion actuator 103 (advance / retract position on the linear axis). The linear encoder 108 detects the position of the mover of the detection actuator 104 (advance / retract position on the linear axis). The rotary encoder 109 detects the position (rotation angle) of the mover of the rotation actuator 105. The rotary encoder 110 detects the position (rotation angle) of the mover of the operation actuator 106.

[0042] Driver 111 outputs a drive current to insertion actuator 103 in accordance with instructions from control unit 101. Driver 112 outputs a drive current to detection actuator 104 in accordance with instructions from control unit 101. Driver 113 outputs a drive current to rotation actuator 105 in accordance with instructions from control unit 101. Driver 114 outputs a drive current to operation actuator 106 in accordance with instructions from control unit 101.

[0043] The control unit 201 of the slave device 20 is configured by a microcomputer equipped with a processor, memory, etc., and controls the operation of the slave device 20. For example, the control unit 201 controls the driving of the insertion actuator 203, detection actuator 204, rotation actuator 205, and operation actuator 206 of the slave device 20 in accordance with control parameters transmitted from the control device 30. The communication unit 202 controls communication between the slave device 20 and other devices via the network 40.

[0044] The insertion actuator 203 is configured, for example, by a linear motor and advances and retracts the catheter of the slave device 20 in response to an operation input by the operator to the master device 10 to advance and retract the catheter for insertion into a blood vessel, according to instructions from the control unit 201. The detection actuator 204 is configured, for example, by a voice coil motor and advances and retracts the catheter of the slave device 20 in response to an operation input by the operator to the master device 10 to advance and retract the catheter near a lesion for treatment, according to instructions from the control unit 201. In this embodiment, the insertion actuator 203 has a longer stroke than the detection actuator 204, but the detection actuator 204 can control position and force with higher precision than the insertion actuator 203. The rotation actuator 205 is configured, for example, by a rotary motor and rotates the catheter of the slave device 20 around a rotation axis along the advance and retract direction, according to an operation input by the operator to the master device 10, according to instructions from the control unit 201. The operation actuator 206 is composed of, for example, a rotary motor, and operates the end effector (expanding, contracting, opening and closing, etc.) in accordance with the operation input by the operator to the master device 10 in accordance with instructions from the control unit 201.

[0045] The linear encoder 207 detects the position (advance / retract position on the linear axis) of the mover of the insertion actuator 203. The linear encoder 208 detects the position (advance / retract position on the linear axis) of the mover of the detection actuator 204. The rotary encoder 209 detects the position (rotation angle) of the mover of the rotation actuator 205. The rotary encoder 210 detects the position (rotation angle) of the mover of the operation actuator 206.

[0046] Driver 211 outputs a drive current to insertion actuator 203 in accordance with instructions from control unit 201. Driver 212 outputs a drive current to detection actuator 204 in accordance with instructions from control unit 201. Driver 213 outputs a drive current to rotation actuator 205 in accordance with instructions from control unit 201. Driver 214 outputs a drive current to operation actuator 206 in accordance with instructions from control unit 201.

[0047] The display L is installed in a location where the operator of the master device 10 can view the screen, and displays images (such as visible light images or X-ray images of the subject taken by the camera C) that are instructed to be displayed by the control device 30, and information that is instructed to be displayed by the control device 30. The camera C is installed in a location where the slave device 20 can take an image of the subject into which the catheter is inserted, and takes an image of the subject (such as a visible light image or X-ray image) and transmits the taken image to the control device 30.

[0048] Fig. 4 is a schematic diagram showing the hardware configuration of an information processing device that constitutes the control device 30. As shown in Fig. 4, the control device 30 includes a processor 311, a ROM (Read Only Memory) 312, a RAM (Random Access Memory) 313, a bus 314, an input unit 315, an output unit 316, a storage unit 317, a communication unit 318, and a drive 319.

[0049] The processor 311 executes various processes according to a program recorded in the ROM 312 or a program loaded from the storage unit 317 to the RAM 313. The RAM 313 also stores data and the like necessary for the processor 311 to execute various processes, as appropriate.

[0050] The processor 311, the ROM 312, and the RAM 313 are connected to one another via a bus 314. To the bus 314, an input unit 315, an output unit 316, a storage unit 317, a communication unit 318, and a drive 319 are connected.

[0051] The input unit 315 is composed of various buttons and the like, and inputs various information in response to instruction operations. The output unit 316 is composed of a display, speaker, and the like, and outputs images and sounds. If the control device 30 is configured as a smartphone or tablet terminal, the input unit 315 and the display of the output unit 316 may be arranged on top of each other to form a touch panel. The storage unit 317 is composed of a hard disk, DRAM (Dynamic Random Access Memory), or the like, and stores various data managed by each server. The communication unit 318 controls communication between the control device 30 and other devices via a network.

[0052] Removable media 331, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately loaded into the drive 319. A program read from the removable media 331 by the drive 319 is installed in the storage unit 317 as needed.

[0053] [Functional Configuration] Next, the functional configuration of the control system 1 will be described. Fig. 5 is a block diagram showing the functional configuration of the control system 1. As shown in Fig. 5, in the control system 1, the control device 30 executes various processes, causing a sensor information acquisition unit 351, a force haptic transmission unit 352, a determination data acquisition unit 353, a mode setting unit 354, a threshold setting unit 355, and a limiting unit 356 to function in the processor 311. In addition, a control parameter storage unit 371 and a threshold storage unit 372 are formed in the storage unit 317.

[0054] The control parameter storage unit 371 chronologically stores control parameters acquired by the control device 30 during control of haptic transmission between the master device 10 and the slave device 20. In this embodiment, the information stored as control parameters can be various parameters acquired during haptic transmission control, and can include various information that allows haptic transmission control to be reproduced. For example, sensor information acquired by the master device 10 and the slave device 20, state values ​​obtained by coordinate transformation of this sensor information, current command values ​​for each actuator, or various setting values ​​set in the control device 30 for haptic transmission control can be stored as control parameters.

[0055] The threshold storage unit 372 stores thresholds for determining whether or not to restrict a predetermined action. In this embodiment, the slave unit 20 operates to determine whether or not to restrict a predetermined action based on values ​​indicating various external forces input to the slave unit 20 from the environment (hereinafter referred to as "force control parameter values") and the thresholds. Therefore, the threshold storage unit 372 stores absolute values ​​set as thresholds for these force control parameter values. Note that if multiple thresholds are set, the threshold storage unit 372 stores all of these thresholds.

[0056] The sensor information acquisition unit 351 acquires sensor information detected by various sensors installed in the master unit 10 and the slave unit 20. For example, the sensor information acquisition unit 351 acquires information indicating the position (advance / retreat position or rotation angle) of the mover of each actuator detected by the linear encoders 107, 108, 207, and 208 and the rotary encoders 109, 110, 209, and 210. The sensor information acquisition unit 351 also stores the acquired sensor information in the control parameter storage unit 371 as time-series data.

[0057] The haptic transmission unit 352 controls the transmission of haptics between the master unit 10 and the slave unit 20 in accordance with the control algorithm shown in Fig. 2. For example, in the threshold control process, the haptic transmission unit 352 executes control to transmit haptics between actuators for corresponding operations of the master unit 10 and the slave unit 20.

[0058] The determination data acquisition unit 353 performs calculations, analysis, etc. on various data to acquire determination data that is used for determination by the mode setting unit 354, the threshold setting unit 355, and the limiting unit 356. The determination data acquisition unit 353 also outputs the acquired determination data to each of these functional blocks.

[0059] Specifically, the determination data acquisition unit 353 acquires a force control parameter value as the determination data. This force control parameter value can be calculated as the product of mass and acceleration. Therefore, the determination data acquisition unit 353 acquires the force control parameter value by performing calculations such as integration in real time based on the sensor information acquired by the sensor information acquisition unit 351 and stored as control parameters in the control parameter storage unit 371, and information corresponding to the results of the coordinate transformation performed by the functional force-velocity allocation transformation block FT in the control algorithm shown in FIG. 2 . The determination data acquisition unit 353 can acquire, as the force control parameter value, an instantaneous value or a value obtained by an arithmetic calculation such as a moving average. The determination data acquisition unit 353 may also acquire the force control parameter value after filtering the waveform of the instantaneous value using a band-limiting filter.

[0060] In addition, the determination data acquisition unit 353 acquires, as determination data, the analysis results of the image captured by camera C. For example, the determination data acquisition unit 353 calculates the distance between the tip of the catheter and the lesion based on the image captured by camera C. By calculating the distance between the tip of the catheter and the lesion from the image captured by camera C, it is possible to determine the approach of the catheter to the lesion based on the same criteria as when a human visually judges. Note that various sensors can be used to acquire the distance between the tip of the catheter and the lesion. For example, a magnetic detection marker may be provided at the tip of the catheter, and the position of the catheter may be detected from outside the subject using a magnetic sensor to acquire the distance from the lesion. Alternatively, a sensor for detecting the position of the tip of the catheter may be installed inside the subject in advance, and the position of the catheter may be detected using this sensor to acquire the distance from the lesion.

[0061] Additionally, for example, the determination data acquisition unit 353 calculates the thickness of the blood vessel into which the tip of the catheter is inserted based on the analysis results of the image captured by camera C as determination data. When inserting a catheter, the thinner the blood vessel, the more delicate the insertion is required. Therefore, by calculating the thickness of the blood vessel into which the tip of the catheter is inserted from the image captured by camera C, it is possible to determine how delicately the catheter should be inserted.

[0062] Additionally, the determination data acquisition unit 353 acquires various information related to the predetermined action as determination data through input operations from the operator or the like via the input unit 315 or input operations via communication from an external device (e.g., the master device 10) via the communication unit 318. Examples of the various information related to the predetermined action include attributes of the device used to perform the predetermined action, attributes of the target of the predetermined action, and the action content of the predetermined action. In the example of this embodiment, for example, the acquired device attributes include the catheter material, catheter cross-sectional area, and end effector type. Furthermore, the acquired attributes of the target include physical characteristics that affect blood vessels, such as the age of the subject and the presence or absence of a chronic illness. Furthermore, the acquired action content of the predetermined action includes information on the site where the catheter is inserted, the site where a lesion is present, and the length to insert the catheter.

[0063] The determination data acquisition unit 353 then outputs the acquired determination data to the mode setting unit 354, the threshold setting unit 355, and the limiting unit 356, depending on the type of the determination data.

[0064] The mode setting unit 354 switches between the insertion mode and the detection mode by making a determination based on the determination data acquired by the determination data acquisition unit 353. The "insertion mode" is a mode in which the insertion actuator 203 is used to advance and retract the catheter in the slave unit 20, and a haptic sensation is transmitted between the insertion actuator 103 of the master unit 10. The "insertion mode" is a mode that is set, for example, when an operator inserts a catheter into a subject and before the tip of the catheter reaches the vicinity of a lesion. The "detection mode" is a mode in which the detection actuator 204 is used to advance and retract the catheter in the slave unit 20, and a haptic sensation is transmitted between the detection actuator 104 of the master unit 10. The "detection mode" is a mode that is set, for example, when an operator inserts a catheter into a subject and after the tip of the catheter reaches the vicinity of a lesion.

[0065] As described above, in this embodiment, the insertion actuators 103, 203 have a longer stroke than the detection actuators 104, 204, while the detection actuators 104, 204 are capable of controlling the position and force with higher precision than the insertion actuators 103, 203. Therefore, in contrast to the "insertion mode," the "detection mode" is used in situations where the operator needs to sense a minute external force input to the slave device 20.

[0066] When a catheter is inserted into a subject, for example, various external forces input from the environment change between when the catheter is advancing through an artery and when it has reached a lesion such as a constricted heart, and the force control parameter value also changes accordingly. Therefore, the mode setting unit 354 determines the insertion state of the catheter based on the force control parameter value as determination data and switches between the insertion mode and the detection mode. In this case, for example, the mode setting unit 354 switches to the insertion mode when the force control parameter value is less than a predetermined value, and switches to the detection mode when the force control parameter value is equal to or greater than the predetermined value.

[0067] Alternatively, the mode setting unit 354 may switch to the insertion mode when the tip of the catheter has not reached the vicinity of the lesion, and may switch to the detection mode when the tip of the catheter has reached the vicinity of the lesion, based on the distance between the tip of the catheter and the lesion obtained by, for example, analyzing the image of the camera C as determination data. Alternatively, the mode setting unit 354 may switch between the insertion mode and the detection mode based on a mode switching operation from an operator or the like via the input unit 315, or a mode switching operation by communication from an external device (e.g., the master device 10) via the communication unit 318.

[0068] In the detection mode, the threshold setting unit 355 sets a threshold for determining whether or not the restriction unit 356 should restrict a predetermined action. Then, the threshold setting unit 355 stores the set threshold in the threshold storage unit 372. As described above, the threshold is an absolute value set for the force control parameter value.

[0069] The threshold setting unit 355 first sets an initial value of the threshold, which can be set based on, for example, actual measurements, statistical values, or estimated values ​​obtained by simulation when a catheter was previously inserted into a subject or a biological model simulating a subject.

[0070] Furthermore, the threshold setting unit 355 does not uniformly leave the threshold at this initial value based on the determination data, but dynamically changes the threshold to a more appropriate value during the execution of a predetermined action, for example, depending on the execution status of the predetermined action. For example, the threshold is dynamically changed to a more appropriate value depending on the diameter of the blood vessel into which the tip of the catheter is inserted, which is included in the determination data. As described above, when inserting a catheter, the thinner the blood vessel, the more delicate the insertion is required. Therefore, the threshold setting unit 355 changes the threshold to a smaller value when the diameter of the blood vessel into which the tip of the catheter is inserted becomes thinner. Alternatively, the threshold is set to a larger value when the diameter of the blood vessel into which the tip of the catheter is inserted becomes thicker.

[0071] The threshold setting unit 355 may also vary the initial value and the changed value based on various information related to the predetermined action, such as the device attributes, the attributes of the target, and the content of the predetermined action, included in the determination data. For example, the initial value and the changed value may be set higher as the cross-sectional area of ​​the catheter, which is an attribute of the device, increases. Additionally, the threshold may be set to a smaller value if, for example, the subject on whom the predetermined action is to be performed is elderly or has vascular characteristics such as arteriosclerosis.

[0072] Furthermore, the threshold setting unit 355 may set multiple thresholds in stages so that the restriction unit 356 can restrict predetermined actions in stages using various methods. For example, a first threshold and a second threshold that is greater than the first threshold may be set. In this case, both the first threshold and the second threshold may be changed as appropriate.

[0073] In the detection mode, the limiting unit 356 determines whether to limit a predetermined action and limits the execution of the predetermined action based on the determination result. The limiting unit 356 makes the determination by comparing the force control parameter value, which is the determination data, with the threshold value set by the threshold setting unit 355 and stored in the threshold memory unit 372. The limiting unit 356 then limits the execution of the predetermined action when the force control parameter value exceeds the threshold value (i.e., when various external forces input from the environment become too large). In this way, when the force control parameter value exceeds the threshold value, it is assumed that the operator is not operating the master device 10 appropriately, and the end effector or the like disposed at the tip of the catheter makes strong contact with a lesion, organ, or blood vessel, increasing the contact force. Therefore, if the predetermined action (here, insertion of the catheter) is continued, an unexpected inappropriate situation, such as the catheter penetrating the blood vessel, may occur.

[0074] Therefore, the limiting unit 356 limits the execution of a predetermined action to ensure safety in such cases. For example, the limiting unit 356 limits the execution of a predetermined action by utilizing the control of haptic transmission by the haptic transmission unit 352. In this case, the limiting unit 356 limits the execution of a predetermined action by, for example, suppressing the operation of the slave unit 20 by significantly reducing the haptic sensation transmitted from the master unit 10 to the slave unit 20 using a haptic transmission function with scaling (amplification or reduction of force or position) described with reference to equations (3) and (4). Alternatively, the limiting unit 356 limits the execution of a predetermined action by suppressing the operation of the slave unit 20 using a force-haptic transmission function with position limitation described with reference to equations (5) to (8) so that the catheter does not advance beyond a position where the force control parameter value exceeds a threshold but can retreat. Alternatively, the limiting unit 356 limits the execution of a predetermined action by, for example, stopping the operation of the slave unit 20 by setting to zero the output (e.g., voltage value or current value) of the inverse transform block IFT input to each actuator of the slave unit 20.

[0075] The restriction unit 356 also outputs a warning to the operator, causing the operator to stop operating the master device 10 and restricting the execution of a predetermined action on the master device 10. The warning can be realized, for example, by displaying text or an image indicating the content of the warning on the display L. Alternatively, the warning can be realized by issuing a warning sound from a speaker included in the output unit 316.

[0076] Furthermore, when multiple thresholds are set by the threshold setting unit 355, the limiting unit 356 may select from multiple methods for limiting the execution of a predetermined action. For example, the limiting unit 356 may select a method of outputting a warning when the force control parameter value exceeds a first threshold, and a method of suppressing the operation of the slave device 20 when the force control parameter value exceeds a larger second threshold. This allows for gradual restriction in various ways depending on the execution status of the predetermined action, etc. In this case, more gradual restriction may be performed, for example, by setting three or more thresholds, displaying text indicating the content of a warning when the first threshold is exceeded, sounding an additional warning when the second threshold is exceeded, and suppressing the operation of the slave device 20 when the third threshold is exceeded.

[0077] [Operation] Next, the operation of the control system 1 will be described.

[0078] [Threshold Control Process] Figure 6 is a flowchart illustrating the flow of the threshold control process executed by the control device 30. The threshold control process is initiated in response to an instruction to execute the threshold control process from the operator via the input unit 315 or an instruction to execute the threshold control process by communication from an external device (e.g., the master device 10) via the communication unit 318. In this embodiment, the threshold control process is initiated manually by an assistant assisting the operation of the slave device 20 or by remote operation from the master device 10 when the tip of the catheter has been inserted a predetermined distance into the subject (e.g., inserted approximately 1 to 10 cm). This prevents the control of the control device 30 from becoming unstable when there is a large change in external force during the initial insertion period.

[0079] In step S11, the mode setting unit 354 sets the mode to insertion mode. In step S12, the sensor information acquisition unit 351 starts acquiring sensor information detected by various sensors installed in the master device 10 and the slave device 20. This acquisition of sensor information is performed in parallel with other steps until the end of this process. In addition, this acquired sensor information is stored in the control parameter storage unit 371 as time-series data.

[0080] In step S13, the haptic transmission unit 352 starts controlling the transmission of the haptic sensation based on the sensor information. This control of the transmission of the haptic sensation is performed in parallel with other steps until the present processing ends.

[0081] In step S14, the determination data acquisition unit 353 starts acquiring determination data by performing calculations, analysis, etc. on various data. This acquisition of determination data is performed in parallel with other steps until the end of this process. In addition, the determination data acquisition unit 353 outputs the acquired determination data to the mode setting unit 354, the threshold setting unit 355, and the limiting unit 356.

[0082] In step S15, the mode setting unit 354 determines whether to switch the mode based on the determination data. If the mode is to be switched, the determination in step S15 is Yes, and the process proceeds to step S16. On the other hand, if the mode is not to be switched, the determination in step S15 is No, and the process proceeds to step S17.

[0083] In step S16, the mode setting unit 354 switches the mode, i.e., if the insertion mode is set, the mode is changed to the detection mode, and if the detection mode is set, the mode is changed to the insertion mode.

[0084] In step S17, the threshold setting unit 355 determines whether the currently set mode is the insertion mode or the detection mode. If the currently set mode is the detection mode, the detection mode is determined in step S17, and the process proceeds to step S18. On the other hand, if the currently set mode is the insertion mode, the insertion mode is determined in step S17, and the process proceeds to step S23.

[0085] In step S18, the threshold setting unit 355 sets the threshold for the force control parameter value to an initial value. This set threshold is stored in the threshold storage unit 372.

[0086] In step S19, the threshold setting unit 355 determines whether or not to change the threshold based on the determination data. If the threshold is to be changed, the determination in step S19 is Yes, and the process proceeds to step S20. On the other hand, if the threshold is not to be changed, the determination in step S19 is No, and the process proceeds to step S21.

[0087] In step S20, the threshold setting unit 355 changes the threshold value. The changed threshold value is stored in the threshold value storage unit 372.

[0088] In step S21, the restriction unit 356 determines whether or not to restrict the execution of a predetermined action based on the force control parameter value included in the determination data and the threshold value set by the threshold setting unit 355 and stored in the threshold storage unit 372. If the execution of the predetermined action is to be restricted, the determination in step S21 is Yes, and the process proceeds to step S22. On the other hand, if the execution of the predetermined action is not to be restricted, the determination in step S21 is No, and the process proceeds to step S23.

[0089] In step S22, the limiting unit 356 limits the execution of the predetermined action. If multiple thresholds are set, the limiting unit 356 limits the execution of the predetermined action using a method corresponding to the largest threshold that is determined to have been exceeded in step S23.

[0090] In step S23, the haptic transmission unit 352 determines whether a termination condition, which is a condition for terminating this process, has been satisfied. The termination condition may be, for example, an instruction to terminate the threshold control process from the operator via the input unit 315, an instruction to terminate the threshold control process by communication from an external device (e.g., the master device 10) via the communication unit 318, or the operation of the slave device 20 being suppressed due to a restriction on the execution of a predetermined action by the restriction unit 356. If the termination condition has been satisfied, step S23 is determined to be Yes, and this process is terminated. On the other hand, if the termination condition has not been satisfied, step S23 is determined to be No, and the process returns to step S15 and is repeated.

[0091] According to the threshold control process described above, it is determined whether or not to restrict the execution of a predetermined action based on the control parameters used for transmitting haptics. That is, the threshold control process utilizes the underlying configuration of the present system, namely, transmitting haptics, to restrict the execution of a predetermined action at an appropriate timing and ensure safety. Additionally, according to the threshold control process, rather than uniformly determining the threshold value serving as the judgment criterion, it is also possible to dynamically change the threshold value to a more appropriate one during the execution of the predetermined action, for example, depending on the execution status of the predetermined action. Therefore, the threshold control process can solve the problem of ensuring greater safety when transmitting haptics to perform a predetermined action.

[0092] [Verification of Effect] Figure 7 is a schematic diagram showing the changes over time in the magnitude of the external force input to the slave unit 20, the positions of the master unit 10 and the slave unit 20, and the position of the catheter tip when a catheter penetrates and perforates a blood vessel as a predetermined procedure in the embodiment described above. In Figure 7(A), the horizontal axis represents time [S], and the vertical axis represents the force control parameter value [N] indicating the magnitude of the external force input to the slave unit 20. In Figure 7(B), the horizontal axis represents time [S], and the vertical axis represents the position [M] of the master unit 10 and the slave unit 20 detected as sensor information. In Figure 7(C), the horizontal axis represents time [S], and the vertical axis represents the position [M] of the catheter tip. In Figures 7(D-1) to 7(D-5), the changes over time in the positional relationship between the catheter tip and the inner wall of the blood vessel into which the catheter is inserted.

[0093] As shown in FIG. 7A, during period P1, as the catheter continues to be inserted into the blood vessel, it advances through a steady environment within the subject (e.g., inside an artery), and the force control parameter value, which indicates the external force from the inner wall of the blood vessel, remains approximately constant. In this case, the positional relationship between the catheter tip and the inner wall of the blood vessel is as shown in FIG. 7D-1. However, if the operator does not operate the master unit 10 appropriately and the catheter tip comes into contact with the inner wall of the blood vessel as shown in FIG. 7D-2, the contact force increases, resulting in a transition as shown in period P2. Specifically, as shown in FIG. 7A, during period P2, the force control parameter value gradually increases (indicated by "Gradually increasing force" in the figure). Also, as shown in FIG. 7B, during period P2, the positions of the master unit 10 and the slave unit 20 become more difficult to advance compared to period P1, but the advancement itself continues, so the advancement proceeds gradually (indicated by "Becomes more difficult to advance" in the figure). In contrast, as shown in Figure 7(C), during period P2, the position of the catheter tip hardly changes (indicated as "almost no progress" in the figure). This is because, as shown in Figures 7(D-3) and 7(D-4), the catheter tip collides with the inner wall of the blood vessel, bending and no longer progressing. In this case, elastic force is stored in the catheter tip, like a spring.

[0094] As the elastic force continues to accumulate, a penetration force exceeding the limit of the blood vessel's inner wall is generated from the catheter tip against the blood vessel's inner wall, and as shown in Figure 7 (D-5), the tip of the catheter perforates the blood vessel ("blood vessel perforation" in the figure), and the period progresses as shown by period P3.

[0095] Specifically, as shown in Fig. 7A, during period P3, the tip of the catheter perforates the blood vessel, reducing the contact force from the inner wall of the blood vessel and decreasing the value of the force control parameter. Also, as shown in Fig. 7C, during period P3, the force generated by the perforation causes the tip of the catheter to advance further than expected ("Advance with the force of perforation" in the figure).

[0096] In this way, if the catheter penetrates the blood vessel, there is a time (in this case, period P2) before the penetration occurs when the force control parameter value increases significantly ("Gradually increasing force" in the figure). Therefore, in this embodiment, as described above, a threshold value is set for the force control parameter value, the time when this force control parameter value increases significantly is detected, and the execution of a predetermined action (here, insertion of the catheter) is restricted at this time. This makes it possible to prevent the catheter from penetrating the blood vessel at an appropriate time. In other words, greater safety can be ensured when transmitting haptic sensations to execute a predetermined action.

[0097] As shown in FIG. 7B , the positions of the master unit 10 and the slave unit 20 also have a characteristic that, during period P2, there is a timing at which they become less likely to fluctuate (indicated by "becomes less likely to advance" in the figure) just before penetrating the blood vessel. Therefore, it is possible to restrict the execution of a predetermined action (here, catheter insertion) based on the control parameter values ​​indicating the positions of the master unit 10 and the slave unit 20. However, this timing at which the value of the position control parameter becomes less likely to fluctuate cannot be detected from the instantaneous value of the position control parameter; rather, the value of the position control parameter must be statistically observed for a certain period of time. Furthermore, the change in the value of the position control parameter at this timing is slower than the change in the value of the force control parameter, making it difficult to detect. For these reasons, even if a restriction on the predetermined action is implemented after detecting this timing, the restriction may not be implemented in time and the device may still penetrate the blood vessel. From this perspective, in this embodiment, the force control parameter value is used to enable early detection of signs of penetrating the blood vessel. In other words, in this embodiment, the use of the force control parameter value further ensures safety.

[0098] [Variation 1] In the above embodiment, an example has been described in which the catheter is inserted and then operated by an actuator until it reaches the lesion, but this is not limiting. For example, the catheter may be manually inserted to the vicinity of the lesion, and insertion may be started using the master unit 10 and the slave unit 20 in a specific section near the lesion, controlling the transmission of haptic sensations and performing threshold control processing similar to that in the detection mode.

[0099] Fig. 8 is a schematic diagram showing the configuration of control system 1 that performs threshold control processing after an operator manually inserts the catheter of slave device 20. As shown in Fig. 8, in control system 1 of this modification, an operating lever (grip portion) or the like is provided on the catheter of slave device 20, allowing manual operation by the operator. Furthermore, control system 1 of this modification only includes detection actuators 104, 204 out of the linear actuators included in control system 1 of the first embodiment shown in Fig. 1, and does not include insertion actuators 103, 203.

[0100] When the operator manually inserts the catheter, the catheter is released from the movement control by the detection actuator 204 and the rotation actuator 205 in the slave unit 20, and can be operated in the same manner as a conventional catheter. At this time, it is assumed that the operator inserts the catheter to a position just short of the vicinity of the lesion, and this state is set as the initial state, and the threshold control process is started.

[0101] When the threshold control process is initiated, the catheter is held for movement control by the detection actuator 204 and the rotation actuator 205, and in response to an operation on the master unit 10, the slave unit 20 moves the catheter, and control unit 30 starts control to transmit a haptic sensation. By performing the threshold control process in the same manner as in the detection mode in the above-described embodiment, this modification also makes it possible to restrict the execution of a predetermined action based on the force control parameter value and the threshold. According to this modification, the distance over which the catheter is moved by the actuator is relatively short, and therefore it is sufficient to provide an actuator with a short stroke, such as a voice coil motor, thereby enabling the master unit 10 and the slave unit 20 to be made smaller and lighter.

[0102] [Variation 2] In the above-described embodiment, the threshold setting unit 355 dynamically changes the threshold to a more appropriate value based on the analysis results of the image captured by the camera C. For example, the threshold is dynamically changed according to the thickness of the blood vessel into which the tip of the catheter is inserted, which is included in the determination data. This is not limiting, and the threshold may be dynamically changed in consideration of other information as well. For example, the threshold may be dynamically changed to a more appropriate value in consideration of the time-series control parameters stored in the control parameter storage unit 371 and acquired when the control device 30 controls the transmission of haptics between the master device 10 and the slave device 20.

[0103] In this case, the determination data acquisition unit 353 further acquires, as determination data, time-series control parameters stored in the control parameter storage unit 371. Furthermore, the threshold setting unit 355 dynamically changes the threshold to a more appropriate one based on both the analysis results of the image captured by the camera C acquired as determination data and the control parameters.

[0104] For example, when inserting a catheter (slave unit 20) into a subject, if the catheter is twisted more than once, it may be damaged. Therefore, it is important not to rotate it too much. Therefore, the threshold setting unit 355 calculates the number of twists in the catheter by comparing the actual rotation angle of the catheter tip, determined based on the image analysis results, with the rotation angle of the mover of the rotation actuator 205 of the slave unit 20, determined based on the control parameters. That is, it calculates the difference between the rotation angle of the catheter tip and the rotation angle of the inserted portion of the catheter by the slave unit 20, and thus the degree of twist. The threshold setting unit 355 then dynamically and appropriately changes the threshold so that this twist (i.e., the difference in rotation angle) does not exceed the number of rotations that may damage the catheter. For example, if twisting of more than N rotations may damage the catheter, the threshold is dynamically changed to a smaller value when twisting of M rotations (M is a value smaller than N) occurs, and the limiting unit 356 restricts the execution of the predetermined action (here, insertion of the catheter).

[0105] Based on a similar concept, the threshold setting unit 355 also calculates the difference in insertion length by comparing the actual insertion length of the catheter tip in the thrust direction (advance / retraction direction) determined based on the image analysis results with the insertion length determined by the master unit 10 and the slave unit 20 in the thrust direction (advance / retraction direction) determined based on the control parameters. If this difference is equal to a predetermined length, it indicates that the insertion was not performed properly, and the positions of the master unit 10 and the slave unit 20 differ from the position of the catheter tip, as shown by period P2 in Figures 7(B) and 7(C). Therefore, the threshold setting unit 355 dynamically changes the threshold to a smaller value when the difference reaches the predetermined length, and the restriction unit 356 restricts the execution of a predetermined action (here, catheter insertion). As described above, by dynamically changing the threshold based on both the image analysis results and the control parameters, as in this second modification, it is possible to detect abnormalities from various perspectives, thereby ensuring greater safety.

[0106] In the above embodiment, the force in the thrust direction (advance / retraction direction) of the catheter is transmitted between the master unit 10 and the slave unit 20 via haptic transmission, but this is not limiting. For example, a force related to rotation around a rotation axis along the advance / retraction direction or a force related to the operation of the end effector may be transmitted between the master unit 10 and the slave unit 20 via haptic transmission.

[0107] Furthermore, in the above-described embodiment, an example has been described in which a catheter is remotely operated by the control system 1, but this is not limiting. In other words, various devices can be remotely operated by the control system 1, such as various devices having linear components, such as guidewires, forceps, and medical devices such as endoscopes.

[0108] Furthermore, in the above-described embodiment, the case where haptics are transmitted by associating the actuators of the master device 10 with the actuators of the slave device 20 in one-to-one correspondence has been described as an example, but this is not limiting. That is, haptics can be transmitted by associating multiple actuators of the master device 10 with one actuator of the slave device 20, or by associating one actuator of the master device 10 with multiple actuators of the slave device 20. It is also possible to transmit haptics by associating multiple actuators of the master device 10 with multiple actuators of the slave device 20. As an example, it is possible to transmit haptics by associating the insertion actuator 203 and detection actuator 204 of the slave device 20 shown in FIG. 3 with the insertion actuator 103 of the master device 10. In this case, there is no need to provide the detection actuator 104 of the master device 10, thereby reducing costs and the weight of the device.

[0109] Furthermore, in the above embodiment, an example configuration was described in which the actuators for advancing and retracting the catheter of the slave unit 20 included the insertion actuator 203 and the detection actuator 204, but this is not limited to this. That is, a single actuator may be used to advance and retract the catheter of the slave unit 20 as long as it satisfies the required performance in terms of stroke and accuracy of operation. In this case, the process of switching between the insertion mode and the detection mode is omitted, and threshold control processing is performed in the same manner as the threshold control processing when the detection mode is set in the above embodiment, thereby making it possible to restrict the execution of a predetermined action based on the force control parameter value and the threshold value, even in this modified example.

[0110] Furthermore, in the above-described embodiment, the restriction on the execution of a predetermined action based on the force control parameter value and the threshold value is performed only when the detection mode is selected. However, this is not limited to this. That is, the restriction on the execution of a predetermined action may also be performed based on the force control parameter value and the threshold value when the insertion mode is selected. In this case, step S17 shown in FIG. 6 may be omitted, and the processing from step S18 onward may be performed regardless of whether the detection mode or the insertion mode is selected. In this case, the threshold value when the detection mode is selected may be different from the threshold value when the insertion mode is selected. For example, the threshold value when the detection mode is selected may be lower than the threshold value when the insertion mode is selected. This makes it possible to restrict the execution of a predetermined action even when the operator does not operate the master device 10 appropriately in the insertion mode, causing the tip of the catheter to come into strong contact with a blood vessel bifurcation or the like.

[0111] Furthermore, while a warning is output to the operator in the above-described embodiment, this is not limiting. That is, information other than a warning may also be output to the operator. For example, the force control parameter value calculated by the determination data acquisition unit 353 may be displayed in real time on the display L to be output to the operator. In this case, as shown in FIG. 7A , by displaying the force control parameter value calculated by the determination data acquisition unit 353 in the form of a graph in which the horizontal axis represents time [S] and the vertical axis represents the force control parameter value [N] indicating the magnitude of the external force input to the slave device 20, the operator can grasp in real time that the force control parameter value is increasing. That is, safety can be further ensured from a perspective different from the restriction of a predetermined action by the restriction unit 356.

[0112] [Configuration Example] As described above, the control system 1 according to this embodiment includes the master device 10 to which an operator's operation is input, and the slave device 20 that performs a predetermined action by operating in accordance with the operation input to the master device 10. The control system 1 also includes a haptic transmission unit 352, a limiting unit 356, and a threshold setting unit 355. The haptic transmission unit 352 controls the transmission of haptics between the master device 10 and the slave device 20. The limiting unit 356 limits the execution of the predetermined action based on control parameters used by the haptic transmission unit 352 to control the transmission of haptics and thresholds corresponding to the control parameters. The threshold setting unit 355 changes the thresholds during the execution of the predetermined action. In this way, the control system 1 determines whether to limit the execution of the predetermined action based on the control parameters used to transmit haptics. In other words, by utilizing the configuration of this system, namely, transmitting haptics, the control system 1 can limit the execution of the predetermined action at the appropriate time and ensure safety. For example, compared to common methods such as an operator performing an operation while visually viewing an image of the subject or transmitting an approximate force calculated based on the torque current of a motor to the operator, the control system 1 can detect the appropriate timing with greater accuracy. In addition, rather than uniformly determining a threshold value as a judgment criterion, the control system 1 can dynamically change the threshold value to a more appropriate one while a predetermined action is being performed, for example, depending on the execution status of the predetermined action. Therefore, the control system 1 can solve the problem of ensuring greater safety when transmitting haptic sensations to perform a predetermined action.

[0113] The control system 1 further includes a determination data acquisition unit 353. The determination data acquisition unit 353 acquires image data related to the execution of a predetermined action and analyzes the image data. The threshold setting unit 355 changes the threshold based on the analysis result of the image data by the determination data acquisition unit 353. This makes it possible to reliably identify the execution status of the predetermined action by analyzing the image data, and dynamically change the threshold to a more appropriate one depending on the identification result.

[0114] A plurality of thresholds are set in stages. The restricting means selects a method for restricting the execution of the predetermined action from a plurality of methods based on the control parameter and the plurality of thresholds. This allows for a variety of methods to be used for restricting the execution of the predetermined action in stages, depending on the execution status of the predetermined action, etc.

[0115] The restriction unit 356 restricts the execution of a predetermined action by restricting the operation of the slave device 20. This physically restricts the operation of the slave device 20 (for example, stops the operation), thereby restricting the execution of a predetermined action in the slave device 20.

[0116] The restriction unit 356 restricts the execution of a predetermined action by outputting a warning to the operator, thereby making the operator stop operating the master device 10 and restricting the execution of the predetermined action on the master device 10.

[0117] The threshold setting unit 355 determines at least one of the initial threshold value and the modified threshold value based on at least one of the attributes of the device used to perform the predetermined action, the attributes of the target of the predetermined action, and the action content of the predetermined action. This makes it possible to make the initial threshold value and the modified threshold value more appropriate based on various information related to the predetermined action.

[0118] The control parameters correspond to the force input to the slave device 20 in association with the execution of a predetermined action. This makes it possible to restrict the execution of the predetermined action at an appropriate timing based on the force input to the slave device 20 (e.g., resistance force from the subject), thereby ensuring safety.

[0119] As described above, the control device 30 according to this embodiment includes a haptic transmission unit 352, a limiting unit 356, and a threshold setting unit 355. The haptic transmission unit 352 controls the transmission of haptics between the master device 10, to which an operator's operation is input, and the slave device 20, which performs a predetermined action by operating in accordance with the operation input to the master device 10. The limiting unit 356 limits the execution of the predetermined action based on control parameters used by the haptic transmission unit 352 to control the transmission of haptics and threshold values ​​corresponding to the control parameters. The threshold setting unit 355 changes the threshold value while the predetermined action is being performed. This configuration of the control device 30, like the control system 1 described above, can also solve the problem of ensuring greater safety when performing a predetermined action by transmitting haptics.

[0120] The present invention is not limited to the above-described embodiment, and modifications, improvements, and the like within the scope of achieving the object of the present invention are included in the present invention. For example, in addition to being realized as the control system 1 in the above-described embodiment, the present invention can also be realized as a control device that controls the control system 1, a control method constituted by each step executed in the control system 1, or a program executed by a processor to realize the functions of the control system 1. Furthermore, in the above-described embodiment, an example was given of a configuration in which the control device 30 is realized as an independent device, but the functions of the control device 30 can be implemented in either the control unit 101 of the master device 10 or the control unit 201 of the slave device 20, or can be distributed and implemented in both of them.

[0121] Furthermore, the processes in the above-described embodiments can be executed by either hardware or software. That is, it is sufficient that the control system 1 is provided with a function capable of executing the above-described processes, and the functional and hardware configurations for realizing these functions are not limited to the above-described examples. When the above-described processes are executed by software, the programs that make up the software are installed on a computer from a network or a storage medium.

[0122] The storage medium for storing the program may be a removable medium distributed separately from the device itself, or may be a storage medium pre-installed in the device itself. Removable media may be, for example, a semiconductor memory, a magnetic disk, an optical disk, or a magneto-optical disk. Optical disks may be, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disks), Blu-ray Discs (registered trademark), etc. Magneto-optical disks may be, for example, MDs (Mini-Disks), etc. Furthermore, storage media pre-installed in the device itself may be, for example, a ROM (Read Only Memory) or hard disk in which the program is stored, or a semiconductor memory.

[0123] The above-described embodiment shows an example of application of the present invention and does not limit the technical scope of the present invention. In other words, the present invention can be modified in various ways, such as by omission or substitution, without departing from the spirit of the present invention, and various embodiments other than the above-described embodiment can be adopted. The various embodiments and modifications that the present invention can adopt are included in the scope of the invention described in the claims and their equivalents.

[0124] 1 Control system, 10 Master device, 20 Slave device, 30 Control device, 40 Network, L Display, C Camera, FT Functional force / speed allocation conversion block, FC Ideal force source block, PC Ideal speed (position) source block, IFT Inverse conversion block, S Control target system, 101, 201 Control unit, 102, 202 Communication unit, 103, 203 Insertion actuator, 104, 204 Detection actuator, 105, 205 Rotation actuator, 106, 206 Operation actuator, 107, 108, 207, 208 Linear encoder, 109, 110, 209, 210 Rotary encoder, 111 to 114, 211 to 214 Driver, 311 Processor, 312 ROM, 313 RAM, 314 Bus, 315 Input unit, 316 Output unit, 317 storage unit, 318 communication unit, 319 drive, 331 removable media, 351 sensor information acquisition unit, 352 force haptic transmission unit, 353 determination data acquisition unit, 354 mode setting unit, 355 threshold setting unit, 356 limiting unit, 371 control parameter storage unit, 372 threshold storage unit

Claims

1. A control system including a master device to which an operation by an operator is input, and a slave device that performs a predetermined action by operating in accordance with the operation input to the master device, a control means for controlling transmission of haptic sensations between the master device and the slave device; a limiting means for limiting the execution of the predetermined action based on a control parameter used by the control means to control the transmission of the haptic sensation and a threshold value corresponding to the control parameter; a threshold setting means for changing the threshold while the predetermined action is being performed; A control system comprising:

2. The device further includes a data acquisition unit that acquires image data relating to the execution of the predetermined action and analyzes the image data, the threshold setting means changes the threshold based on at least the analysis result of the image data by the data acquisition means.

2. The control system of claim 1.

3. the threshold setting means changes the threshold based on both the analysis result of the image data by the data acquisition means and the control parameters used by the control means to control the transmission of the haptic sensation.

3. The control system of claim 2.

4. The threshold value is set in a plurality of stages, the restriction means selects a method for restricting the execution of the predetermined action from a plurality of methods based on the control parameter and the plurality of thresholds; 4. A control system according to claim 1, wherein the control system comprises:

5. the restriction means restricts the execution of the predetermined action by suppressing the operation of the slave device.

4. A control system according to claim 1, wherein the control system comprises:

6. the restriction means restricts the execution of the predetermined action by outputting a warning to the operator.

4. A control system according to claim 1, wherein the control system comprises:

7. the threshold setting means determines at least one of an initial value of the threshold and a changed value of the threshold based on at least one of an attribute of a device used to perform the predetermined action, an attribute of a target on which the predetermined action is to be performed, and an action content of the predetermined action; 4. A control system according to claim 1, wherein the control system comprises:

8. The control parameter corresponds to a force input to the slave device in association with the execution of the predetermined action.

4. A control system according to claim 1, wherein the control system comprises:

9. a control means for controlling transmission of haptic sensations between a master device to which an operation by an operator is input and a slave device that performs a predetermined action by operating in accordance with the operation input to the master device; a limiting means for limiting the execution of the predetermined action based on a control parameter used by the control means to control the transmission of the haptic sensation and a threshold value corresponding to the control parameter; a threshold setting means for changing the threshold while the predetermined action is being performed; A control device comprising:

10. A control method executed in a control system including a master device to which an operation by an operator is input, and a slave device that performs a predetermined action by operating in accordance with the operation input to the master device, a control step of controlling transmission of haptic sensations between the master device and the slave device; a limiting step of limiting the execution of the predetermined action based on a control parameter used to control the transmission of the haptic sensation in the control step and a threshold value corresponding to the control parameter; a threshold setting step of changing the threshold while the predetermined action is being performed; A control method comprising:

11. a control function for controlling the transmission of haptic sensations between a master device to which an operation by an operator is input and a slave device that performs a predetermined action by operating in accordance with the operation input to the master device; a limiting function that limits the execution of the predetermined action based on a control parameter used by the control function to control the transmission of the haptic sensation and a threshold value corresponding to the control parameter; a threshold setting function for changing the threshold while the predetermined action is being performed; A program characterized by causing a computer to realize the above.