Information presenting system, information presenting device, information presenting method, and program

Abstract

The present invention addresses the problem of providing additional support on top of support based on haptic sense transmission. An information presenting system (1) according to the present invention comprises: a master device (10) to which an operation of an operator is input; and a slave device (20) operating in response to the operation input to the master device (10). The information presenting system (1) comprises a haptic sense transmission unit (352), a calculation unit (355), and a presentation unit (356). The haptic sense transmission unit (352) controls transmission of a haptic sense between the master device (10) and the slave device (20). The calculation unit (355) calculates a physical property of a substance with which the slave device (20) comes into contact, on the basis of the external force input from the environment to the slave device (20) during the period when the slave device (20) maintains a predetermined motion state. The presentation unit (356) presents the physical property of the substance calculated by the calculation unit (355).

Description

Information presentation system, information presentation device, information presentation method and program

[0001] The present invention relates to an information presentation system, an information presentation device, an information presentation method, and a program.

[0002] A bilateral control technique has been known in the past, in which a master device receives input of an operator's operation and a slave device operates in response to the operation input to the master device, and a reaction force corresponding to the operation of the slave device is transmitted to the master device as a haptic sensation. Such bilateral control technique is disclosed, for example, in Patent Document 1.

[0003] Japanese Patent Application Publication No. 64-34686

[0004] According to the general technology described above, it is possible to provide assistance to an operator in an operation by transmitting haptic sensations between a master device and a slave device. However, it is desirable to be able to provide further assistance in addition to the assistance provided by such haptic sensation transmission.

[0005] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide further assistance in addition to assistance provided by the transmission of haptic sensations.

[0006] In order to solve the above-mentioned problems, an information presentation system according to one aspect of the present invention is an information presentation system including a master device to which an operator's operation is input, and a slave device that operates in accordance with the operation input to the master device, and is characterized by comprising: a control means for controlling the transmission of haptic sensations between the master device and the slave device; a calculation means for calculating physical properties of a substance that the slave device comes into contact with, based on an external force input from the environment to the slave device while the slave device is maintaining a predetermined motion state; and a presentation means for presenting the physical properties of the substance calculated by the calculation means.

[0007] According to the present invention, in addition to the support provided by the transmission of haptic sensations, it is possible to provide further support.

[0008] FIG. 1 is a schematic diagram showing the overall configuration of an information presentation system 1 according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the basic principle of force haptic transmission control executed by an information presentation device 30. FIG. 3 is a block diagram showing the hardware configuration of a control system in the information presentation system 1. FIG. 4 is a schematic diagram showing the hardware configuration of an information processing device that constitutes the information presentation device 30. FIG. 5 is a block diagram showing the functional configuration of the information presentation system 1. FIG. 6 is a flowchart explaining the flow of information presentation processing executed by the information presentation device 30. FIG. 7 is a schematic diagram showing time-series changes in external force input to the slave device 20 from the environment when the slave device 20 comes into contact with a substance. FIG. 8 is a schematic diagram showing the configuration of an information presentation system 1 that performs information presentation processing after an operator manually inserts a catheter of the slave device 20.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [Configuration] Fig. 1 is a schematic diagram showing the overall configuration of an information presentation system 1 according to an embodiment of the present invention. As shown in Fig. 1, the information presentation system 1 according to this embodiment is configured as a master-slave system including a master device 10 and a slave device 20 that are mechanically separated. As an example, in the information presentation system 1 according to this embodiment, the master device 10 constitutes a manipulator operated by an operator, and the slave device 20 constitutes a catheter system equipped with an end effector to be inserted into a subject.

[0011] 1 , the information presentation system 1 includes a master device 10, a slave device 20, and an information presentation device 30. The master device 10, the slave device 20, and the information presentation device 30 are configured to be able to communicate with each other via a network 40, either wired or wirelessly. The information presentation system 1 may also include a display L and multiple cameras C as appropriate. The camera C may be any of a variety of imaging devices, such as a video camera that captures the external appearance of a subject into which the slave device 20 is inserted, or an X-ray camera that captures the interior of the subject (e.g., the subject's blood vessels and organs) using X-rays. The information presentation system 1 may also include multiple displays L that display various images captured by the multiple cameras C and various information output from the information presentation device 30.

[0012] The master unit 10 receives operations similar to those performed on a conventional mechanically configured catheter and detects the position of a movable part (such as a movable member of a manipulator) that moves in response to the input operation. The master unit 10 transmits information indicating the detected position of the movable part to the information presentation device 30. In response to the input operation, the master unit 10 outputs a reaction force from an actuator in accordance with instructions from the information presentation device 30.

[0013] Specifically, the master device 10 accepts operations to advance and retract the catheter (for example, an operation to insert it into a blood vessel or an operation to slightly move it to detect force tactile sensation near a lesion), an operation to rotate the catheter around its axis (for example, an operation to change the orientation of the end effector), and an operation to operate the end effector (for example, an operation to expand or contract the end effector if the end effector is a balloon, or an operation to open or close the end effector if the end effector is a forceps or the like), and applies a reaction force to these operations while transmitting information indicating the position of the movable part moved by each operation to the information presentation device 30.

[0014] The slave unit 20 drives the actuator in accordance with instructions from the information presentation device 30, thereby performing an operation corresponding to the operation input to the master unit 10, and detects the position of a movable part (such as a movable element of the actuator or a catheter moved by the actuator) that moves as a result of the operation. As the slave unit 20 operates, various external forces are input to the slave unit 20 from the environment. As a result, the position of the movable part in the slave unit 20 indicates the result of the various external forces acting on the output of the actuator. The slave unit 20 then transmits information indicating the detected position of the movable part to the information presentation device 30. Here, the various external forces input to the slave unit 20 from the environment include, for example, a thrust resistance force that a catheter inserted into a subject receives from a blood vessel, and a contact force when a guidewire, an end effector, or the like disposed at the tip of the catheter comes into contact with a lesion, an organ, or a blood vessel.

[0015] The information presentation device 30 is configured by an information processing device such as a PC (Personal Computer) or a server computer, and controls the master device 10, the slave device 20, the display L, and the camera C. For example, the information presentation device 30 acquires the positions of the movable parts of the master device 10 and the slave device 20 (such as the rotation angle of the actuator detected by a rotary encoder or the advance / retract position of the movable part detected by a linear encoder), and executes control to transmit haptic sensations between the master device 10 and the slave device 20.

[0016] When the information presentation device 30 of this embodiment operates the master device 10 and the slave device 20 as a master-slave system, it performs coordinate transformation (transformation using a transformation matrix) of real-space parameters (input vectors) calculated based on information representing the position of a movable part (information representing the position of a movable element of an actuator or the position of a member moved by the actuator) into a virtual space in which position and force can be handled independently. That is, the input vector is coordinate-transformed from the real space of an oblique coordinate system in which position and force are related to each other to the virtual space of a Cartesian coordinate system in which position and force are independent of each other. The parameters calculated by the coordinate transformation represent state values ​​of the position and force corresponding to the input vector in the virtual space. Then, in the virtual space after the coordinate transformation, the information presentation device 30 performs calculations to cause the state values ​​of the position and force calculated from the input vector to follow target values ​​for the position and force for controlling the position and force (here, transmitting haptic sensations), and then performs inverse transformation (transformation using the inverse matrix of the transformation matrix) to return the calculation results to the real space. Furthermore, the information presentation device 30 realizes a master-slave system that transmits haptic sensations between the master device 10 and the slave device 20 by driving each actuator based on real-space parameters (such as current command values) obtained by inverse transformation.

[0017] In addition, since position and velocity (or acceleration) or angle and angular velocity (or angular acceleration) are parameters that can be replaced by differential and integral calculations, when performing processing related to position or angle, they can be replaced with velocity or angular velocity, etc. as appropriate.

[0018] In this configuration, the information presentation system 1 of this embodiment realizes a master-slave system that transmits haptic sensations between the master device 10 and the slave device 20 as described above, and also performs information presentation processing. Here, the information presentation processing is a series of processes that, when transmitting haptic sensations to perform a predetermined action, presents the physical properties of a substance that the slave device 20 comes into contact with, based on an external force input from the environment to the slave device 20.

[0019] Specifically, in the information presentation process, the information presentation system 1 controls the transmission of haptic sensations between the master device 10 and the slave device 20. The information presentation system 1 also calculates the physical properties of a material that the slave device 20 comes into contact with, based on an external force input from the environment to the slave device 20 while the slave device 20 is maintaining a predetermined motion state. Furthermore, the information presentation system 1 presents the calculated physical properties of the material.

[0020] In this way, the information presentation system 1 can present the physical properties of a substance that the slave unit 20 contacts based on an external force input from the environment to the slave unit 20 while the slave unit 20 is quantitatively maintaining a predetermined motion state. Therefore, for example, the physical properties of a substance, such as a blood vessel, that the operator cannot directly touch can be quantitatively presented as information obtained by the slave unit 20 contacting the substance (i.e., tactile information), thereby providing further support for the operator's operation. Furthermore, in addition to the operator's operation, the presented physical properties can also be used to provide further support to those performing various analyses, examinations, and the like. Therefore, the information presentation system 1 can solve the problem of providing further support in addition to the support provided by the transmission of haptic sensations.

[0021] 2 is a schematic diagram showing the basic principle of force haptic transmission control executed by the information presentation device 30. The basic principle shown in Fig. 2 determines the operation of the actuator by performing calculations in at least one of the velocity and force domains using information representing the position of the movable part (the current position of the movable part) as an input. That is, the basic principle of the present invention is expressed as a control law including a controlled system S, a functional force-velocity allocation conversion block FT, at least one of an ideal force source block FC or an ideal velocity source block PC, and an inverse conversion block IFT.

[0022] The controlled system S is a master device 10 or a slave device 20 equipped with an actuator, and controls the actuator based on acceleration, etc. As described above, acceleration, velocity, and position are physical quantities that can be converted into one another using differential and integral calculus, and therefore, control may be performed using any of acceleration, velocity, and position. Here, the control law is primarily expressed using velocity calculated from position.

[0023] The functional force-speed allocation transformation block FT is a block that defines the transformation of control energy into speed and force ranges that are set according to the function of the controlled system S. Specifically, the functional force-speed allocation transformation block FT defines a coordinate transformation that takes as input a reference value (reference value) for the function of the controlled system S and the current position of the movable part. This coordinate transformation generally transforms an input vector having the reference value and current speed as elements into an output vector consisting of speed for calculating a target value for speed control, and also transforms an input vector having the reference value and current force as elements into an output vector consisting of force for calculating a target value for force control. Specifically, the coordinate transformation in the functional force-speed allocation transformation block FT is generalized and expressed as the following equations (1) and (2):

[0024]

[0025] However, in formula (1), x'1 to x' n (n is an integer equal to or greater than 1) is a velocity vector for deriving the velocity state value, and x' a ~x' m (m is an integer of 1 or more) is a vector whose elements are a reference value and a velocity based on the action of the actuator (the velocity of the movable element of the actuator or the velocity of the member moved by the actuator), 1a ~h nm are elements of the transformation matrix that represents the function. n (n is an integer equal to or greater than 1) is a force vector for deriving a force state value, and f'' a ~f'' m(m is an integer of 1 or more) is a vector whose elements are a reference value and a force based on the action of the actuator (the force of the mover of the actuator or the force of the member moved by the actuator).

[0026] By setting the coordinate transformation in the functional force / speed allocation transformation block FT according to the function to be realized, various operations can be realized and scaling can be performed. That is, according to the basic principle of the present invention, the functional force / speed allocation transformation block FT "converts" the variables of a single actuator (variables in real space) into a group of variables of the entire system (variables in virtual space) that represent the function to be realized, and allocates control energy to the velocity control energy and the force control energy. In other words, according to the basic principle of the present invention, calculations related to the control of velocity and force are performed after converting from a coordinate space in which velocity and force are related to each other to a coordinate space in which velocity and force are independent of each other. Therefore, compared to when control is performed using the variables of a single actuator (variables in real space), it is possible to independently assign the velocity control energy and the force control energy.

[0027] The ideal force source block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the functional force-speed allocation transformation block FT. In the ideal force source block FC, a target value for force is set when performing calculations based on the coordinate transformation defined by the functional force-speed allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be realized can be set.

[0028] The ideal speed source block PC is a block that performs calculations in the speed domain according to the coordinate transformation defined by the functional force-speed allocation transformation block FT. In the ideal speed source block PC, a target value for the speed when performing calculations based on the coordinate transformation defined by the functional force-speed allocation transformation block FT is set. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be realized can be set.

[0029] The inverse transformation block IFT is a block that converts values ​​in the velocity and force domains into values ​​in the domain of input to the controlled system S (for example, voltage values ​​or current values). According to this basic principle, when position information of the actuators of the controlled system S is input to the function-specific force / velocity allocation transformation block FT, the function-specific force / velocity allocation transformation block FT applies control laws for the position and force domains according to the function using velocity and force information obtained based on the position information. Then, the ideal force source block FC calculates the force according to the function, and the ideal velocity source block PC calculates the velocity according to the function, and control energy is distributed to each of the force and velocity.

[0030] The calculation results in the ideal force source block FC and the ideal velocity source block PC become information indicating the control target of the controlled system S, and these calculation results are used as input values ​​for the actuators in the inverse transformation block IFT and input to the controlled system S. As a result, the actuators of the controlled system S perform operations in accordance with the functions defined by the functional force / velocity allocation transformation block FT, and the desired device operation is realized.

[0031] Furthermore, when a force-tactile transmission function involving scaling (amplification or reduction of force or position) is realized, the coordinate transformation in the functional force-velocity allocation transformation block FT in FIG. 2 is expressed as the following equations (3) and (4).

[0032]

[0033] However, in formula (3), x' p is the velocity for deriving the velocity state value, x' f is the velocity related to the state value of the force. m is the speed (differential value of the current position of the master unit 10) of the reference value (input from the master unit 10), and x' s is the current velocity of the slave unit 20 (the differential value of the current position). p is the force related to the velocity state value, f f is the force for deriving the state value of the force. m is the reference force (input from the master device 10), f s is the current force of the slave unit 20.

[0034] When the coordinate transformations shown in equations (3) and (4) are performed, the position of the slave device 20 is multiplied by α (α is a positive number) and the force of the slave device 20 is multiplied by β (β is a positive number) before being transmitted to the master device 10. For example, by setting α = 1 and β = 1, the haptic sensation is transmitted without being amplified (i.e., enlarged) or attenuated (i.e., reduced). On the other hand, by setting the values ​​of α and β according to the purpose, it is possible to achieve scaling, in which the transmitted haptic sensation is amplified (i.e., enlarged) or attenuated (i.e., reduced).

[0035] [Hardware Configuration] Next, a hardware configuration of a control system in the information presentation system 1 will be described. Fig. 3 is a block diagram showing the hardware configuration of the control system in the information presentation system 1. As shown in Fig. 3, the information presentation system 1 includes, as the hardware configuration of the control system, an information presentation device 30 configured by an information processing device such as a PC or a server computer, a control unit 101 of the master device 10, a communication unit 102, an insertion actuator 103, a detection actuator 104, a rotation actuator 105, an operation actuator 106, linear encoders 107 and 108, rotary encoders 109 and 110, and drivers 111 to 114, a control unit 201 of the slave device 20, a communication unit 202, an insertion actuator 203, a detection actuator 204, a rotation actuator 205, an operation actuator 206, linear encoders 207 and 208, rotary encoders 209 and 210, and drivers 211 to 214, a display L, and a camera C.

[0036] The control unit 101 of the master device 10 is configured by a microcomputer equipped with a processor, memory, etc., and controls the operation of the master device 10. For example, the control unit 101 controls the driving of the insertion actuator 103, detection actuator 104, rotation actuator 105, and operation actuator 106 of the master device 10 in accordance with control parameters transmitted from the information presentation device 30. The communication unit 102 controls communication between the master device 10 and other devices via the network 40.

[0037] The insertion actuator 103 is configured, for example, by a linear motor and applies a reaction force to an operation of advancing and retracting the catheter to insert it into a blood vessel, which the operator inputs into the master device 10 according to instructions from the control unit 101. The detection actuator 104 is configured, for example, by a voice coil motor and applies a reaction force to an operation of advancing and retracting the catheter to treat a lesion, which the operator inputs into the master device 10 according to instructions from the control unit 101. In this embodiment, the insertion actuator 103 has a longer stroke than the detection actuator 104, but the detection actuator 104 can control position and force with higher precision than the insertion actuator 103. The rotation actuator 105 is configured, for example, by a rotary motor and applies a reaction force to an operation of rotating the master device 10 around a rotation axis along the advancing and retracting direction, which the operator inputs into the master device 10 according to instructions from the control unit 101. The operation actuator 106 is composed of, for example, a rotary motor, and applies a reaction force to the operation input by the operator to a lever (grip) or the like to operate the end effector in accordance with instructions from the control unit 101.

[0038] The linear encoder 107 detects the position of the mover of the insertion actuator 103 (advance / retract position on the linear axis). The linear encoder 108 detects the position of the mover of the detection actuator 104 (advance / retract position on the linear axis). The rotary encoder 109 detects the position (rotation angle) of the mover of the rotation actuator 105. The rotary encoder 110 detects the position (rotation angle) of the mover of the operation actuator 106.

[0039] Driver 111 outputs a drive current to insertion actuator 103 in accordance with instructions from control unit 101. Driver 112 outputs a drive current to detection actuator 104 in accordance with instructions from control unit 101. Driver 113 outputs a drive current to rotation actuator 105 in accordance with instructions from control unit 101. Driver 114 outputs a drive current to operation actuator 106 in accordance with instructions from control unit 101.

[0040] The control unit 201 of the slave device 20 is configured by a microcomputer equipped with a processor, memory, etc., and controls the operation of the slave device 20. For example, the control unit 201 controls the driving of the insertion actuator 203, detection actuator 204, rotation actuator 205, and operation actuator 206 of the slave device 20 in accordance with control parameters transmitted from the information presentation device 30. The communication unit 202 controls communication between the slave device 20 and other devices via the network 40.

[0041] The insertion actuator 203 is configured, for example, by a linear motor and advances and retracts the catheter of the slave device 20 in response to an operation input by the operator to the master device 10 to advance and retract the catheter for insertion into a blood vessel, according to instructions from the control unit 201. The detection actuator 204 is configured, for example, by a voice coil motor and advances and retracts the catheter of the slave device 20 in response to an operation input by the operator to the master device 10 to advance and retract the catheter near a lesion for treatment, according to instructions from the control unit 201. In this embodiment, the insertion actuator 203 has a longer stroke than the detection actuator 204, but the detection actuator 204 can control position and force with higher precision than the insertion actuator 203. The rotation actuator 205 is configured, for example, by a rotary motor and rotates the catheter of the slave device 20 around a rotation axis along the advance and retract direction, according to an operation input by the operator to the master device 10, according to instructions from the control unit 201. The operation actuator 206 is composed of, for example, a rotary motor, and operates the end effector (expanding, contracting, opening and closing, etc.) in accordance with the operation input by the operator to the master device 10 in accordance with instructions from the control unit 201.

[0042] The linear encoder 207 detects the position (advance / retract position on the linear axis) of the mover of the insertion actuator 203. The linear encoder 208 detects the position (advance / retract position on the linear axis) of the mover of the detection actuator 204. The rotary encoder 209 detects the position (rotation angle) of the mover of the rotation actuator 205. The rotary encoder 210 detects the position (rotation angle) of the mover of the operation actuator 206.

[0043] Driver 211 outputs a drive current to insertion actuator 203 in accordance with instructions from control unit 201. Driver 212 outputs a drive current to detection actuator 204 in accordance with instructions from control unit 201. Driver 213 outputs a drive current to rotation actuator 205 in accordance with instructions from control unit 201. Driver 214 outputs a drive current to operation actuator 206 in accordance with instructions from control unit 201.

[0044] The display L is installed in a location where the operator of the master device 10 can view the screen, and displays images (such as visible light images or X-ray images of the subject captured by the camera C) that are instructed to be displayed by the information presentation device 30, and information that is instructed to be displayed by the information presentation device 30. The camera C is installed in a location where the slave device 20 can capture an image of the subject into which the catheter is inserted, and captures an image of the subject (such as a visible light image or X-ray image) and transmits the captured image to the information presentation device 30.

[0045] Fig. 4 is a schematic diagram showing the hardware configuration of an information processing device that constitutes the information presentation device 30. As shown in Fig. 4, the information presentation device 30 includes a processor (Central Processing Unit) 311, a ROM (Read Only Memory) 312, a RAM (Random Access Memory) 313, a bus 314, an input unit 315, an output unit 316, a storage unit 317, a communication unit 318, and a drive 319.

[0046] The processor 311 executes various processes according to a program recorded in the ROM 312 or a program loaded from the storage unit 317 to the RAM 313. The RAM 313 also stores data and the like necessary for the processor 311 to execute various processes, as appropriate.

[0047] The processor 311, the ROM 312, and the RAM 313 are connected to one another via a bus 314. To the bus 314, an input unit 315, an output unit 316, a storage unit 317, a communication unit 318, and a drive 319 are connected.

[0048] The input unit 315 is composed of various buttons and the like, and inputs various information in response to instruction operations. The output unit 316 is composed of a display, speaker, and the like, and outputs images and sounds. Note that when the information presentation device 30 is configured as a smartphone or tablet terminal, the input unit 315 and the display of the output unit 316 may be arranged on top of each other to form a touch panel. The storage unit 317 is composed of a hard disk or DRAM (Dynamic Random Access Memory), and stores various data managed by each server. The communication unit 318 controls communication between the information presentation device 30 and other devices via a network.

[0049] Removable media 331, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately loaded into the drive 319. A program read from the removable media 331 by the drive 319 is installed in the storage unit 317 as needed.

[0050] [Functional Configuration] Next, the functional configuration of the information presentation system 1 will be described. Fig. 5 is a block diagram showing the functional configuration of the information presentation system 1. As shown in Fig. 5, in the information presentation system 1, the information presentation device 30 executes various processes, causing a sensor information acquisition unit 351, a force-tactile transmission unit 352, a distance information acquisition unit 353, a mode setting unit 354, a calculation unit 355, and a presentation unit 356 to function in the CPU 311. In addition, the storage unit 317 includes a control parameter storage unit 371 and a physical property storage unit 372.

[0051] The control parameter storage unit 371 chronologically stores control parameters acquired by the information presentation device 30 during control of haptic transmission between the master device 10 and the slave device 20. In this embodiment, the information stored as control parameters can be various parameters acquired during haptic transmission control and can include various information that allows haptic transmission control to be reproduced. For example, sensor information acquired by the master device 10 and the slave device 20, state values ​​obtained by coordinate transformation of this sensor information, current command values ​​for each actuator, various setting values ​​set in the information presentation device 30 for haptic transmission control, etc. can be stored as control parameters.

[0052] The physical property storage unit 372 stores the physical properties of the substance with which the slave unit 20 has come into contact, as calculated by the calculation unit 355. In this embodiment, the calculation unit 355 calculates "elasticity," "viscosity," and "inertia" as the physical properties of the substance. Therefore, the physical property storage unit 372 stores each of these "elasticity," "viscosity," and "inertia." Furthermore, in this embodiment, the presentation unit 356 presents the physical properties of the substance calculated by the calculation unit 355 and the physical properties of a reference substance so that they can be compared. Therefore, the physical property storage unit 372 also stores the physical properties of this reference substance. The physical properties of this reference substance may be physical properties previously calculated by the calculation unit 355, or may be physical properties measured by an inspection method or tactile method different from the calculation by the calculation unit 355.

[0053] The sensor information acquisition unit 351 acquires sensor information detected by various sensors installed in the master unit 10 and the slave unit 20. For example, the sensor information acquisition unit 351 acquires information indicating the position (advance / retreat position or rotation angle) of the mover of each actuator detected by the linear encoders 107, 108, 207, and 208 and the rotary encoders 109, 110, 209, and 210. The sensor information acquisition unit 351 also stores the acquired sensor information in the control parameter storage unit 371 as time-series data.

[0054] The haptic transmission unit 352 controls the transmission of haptics between the master device 10 and the slave device 20 in accordance with the control algorithm shown in Fig. 2. For example, in the information presentation process, the haptic transmission unit 352 executes control to transmit haptics between actuators for corresponding operations of the master device 10 and the slave device 20.

[0055] The distance information acquisition unit 353 acquires distance information indicating the distance between the tip of the catheter and the lesion by performing calculations and analyses on various data. The distance information acquisition unit 353 also outputs the acquired distance information to the mode setting unit 354. This distance information is used by the mode setting unit 354 to determine whether or not to switch modes.

[0056] The distance information acquisition unit 353 acquires distance information using an image captured by camera C. In this case, for example, the distance information acquisition unit 353 acquires distance information by analyzing this image and calculating the distance between the tip of the catheter and the lesion. In this way, by calculating the distance between the tip of the catheter and the lesion from the image captured by camera C, it is possible to determine the approach of the catheter to the lesion using the same criteria as when a human visually judges.

[0057] Alternatively, the distance information acquiring unit 353 may acquire distance information using various sensors. In this case, for example, a magnetic detection marker may be provided at the tip of the catheter, and the position of the catheter tip may be detected from outside the subject using a magnetic sensor, and the distance from the lesion may be calculated to acquire distance information. Alternatively, the distance information acquiring unit 353 may acquire distance information by installing a sensor inside the subject in advance to detect the position of the catheter tip, detecting the position of the catheter using this sensor, and calculating the distance from the lesion.

[0058] The distance information acquisition unit 353 outputs the distance information acquired in this manner to the mode setting unit 354 .

[0059] The mode setting unit 354 switches between three modes, "insertion mode," "detection mode," and "measurement mode," as modes to be set in the information presentation device 30. Specifically, based on the distance between the catheter tip and the lesion acquired as distance information by the distance information acquisition unit 353, the mode setting unit 354 switches to the "insertion mode" when the catheter tip has not reached the vicinity of the lesion, and switches to the "detection mode" when the catheter tip has reached the vicinity of the lesion. In this case, the criterion for determining how close the distance must be to switch modes (i.e., the distance threshold for determining whether to switch) can be set based on, for example, actual measurements, statistical values, or estimated values ​​obtained by simulation when a catheter was previously inserted into a subject or a biological model simulating a subject.

[0060] Furthermore, the mode setting unit 354 switches between the "detection mode" and the "measurement mode" based on a mode switching operation from the operator via the input unit 315 or a mode switching operation by communication from an external device (for example, the master device 10) via the communication unit 318. In other words, the operator can switch between the "detection mode" and the "measurement mode" at any timing desired by the operator.

[0061] The "insertion mode" is a mode in which the insertion actuator 203 is used to advance and retract the catheter in the slave unit 20, and haptic sensation is transmitted between the insertion actuator 103 of the master unit 10. The "insertion mode" is a mode that is set, for example, when the operator inserts a catheter into a subject and before the tip of the catheter reaches the vicinity of a lesion.

[0062] The "detection mode" is a mode in which the detection actuator 204 is used to advance and retract the catheter in the slave unit 20, and a haptic sensation is transmitted between the detection actuator 104 of the master unit 10. The "detection mode" is a mode that is set, for example, after the operator inserts a catheter into a subject and the tip of the catheter reaches the vicinity of a lesion.

[0063] The "measurement mode" is a mode in which the slave unit 20 uses the detection actuator 204 to advance and retract the catheter, and calculates and presents the physical properties of a substance that the catheter comes into contact with as the catheter advances and retracts. Unlike the "insertion mode" and "detection mode," the "measurement mode" does not accept operations by the operator at the master unit 10. In the "measurement mode," the slave unit 20 does not perform an operation corresponding to an operation on the master unit 10, but instead performs an operation to maintain a predetermined state of motion by driving the detection actuator 204 in accordance with instructions for realizing a predetermined state of motion generated by the haptic force transmitter 352. The "measurement mode" is a mode set, for example, to present the physical properties of a substance in a lesion to an operator, etc.

[0064] As described above, in this embodiment, the insertion actuators 103, 203 have a longer stroke than the detection actuators 104, 204, while the detection actuators 104, 204 are capable of controlling the position and force with higher precision than the insertion actuators 103, 203. Therefore, in contrast to the "insertion mode," the "detection mode" is used in situations where the operator needs to sense a minute external force input to the slave device 20. Furthermore, the "measurement mode" is used in situations where the physical properties of a substance in a lesion need to be calculated with high precision based on a minute external force input to the slave device 20.

[0065] When the mode setting unit 354 sets the "measurement mode," the calculation unit 355 calculates the physical properties of the substance with which the slave unit 20 has come into contact (here, the substance in the lesion with which the tip of the catheter has come into contact). In order to calculate the physical properties of this substance, in the "measurement mode," the haptic transmission unit 352 generates instructions for realizing a predetermined state of motion. Furthermore, the slave unit 20 operates to maintain the predetermined state of motion by driving the detection actuator 204 in accordance with the instructions for realizing the predetermined state of motion generated by the haptic transmission unit 352.

[0066] Meanwhile, while this predetermined motion state is maintained, the calculation unit 355 acquires values ​​(hereinafter referred to as "force values") indicating various external forces input to the slave unit 20 from the environment as a result of the operation of the slave unit 20. These force values ​​can be calculated as the product of mass and acceleration. Therefore, the distance information acquisition unit 353 acquires the force values ​​by performing calculations such as integration in real time based on the sensor information acquired by the sensor information acquisition unit 351 and stored as control parameters in the control parameter storage unit 371, and information corresponding to the results of the coordinate transformation performed by the functional force-velocity allocation transformation block FT in the control algorithm shown in FIG. 2. In this case, the distance information acquisition unit 353 may acquire the force values ​​after filtering the waveform of the instantaneous values ​​using a band-limiting filter. In this way, the calculation unit 355 can calculate the force values ​​based on the control parameters. Therefore, in this embodiment, a force sensor is not required to measure the force values.

[0067] As described above, the calculation unit 355 calculates elasticity, viscosity, and inertia as the physical properties of the material. The method by which the calculation unit 355 calculates each of these physical properties and the control performed by the haptic transmission unit 352 for this purpose will be described below.

[0068] When the calculation unit 355 calculates elasticity, the predetermined motion state is a state in which the catheter is inserted at a uniform velocity. Therefore, the haptic transmission unit 352 calculates the velocity (or angular velocity) by differentiating in real time the position (or angle) of the moving part (e.g., the movable element of the actuator or the catheter moved by the actuator) that moves due to the operation of the slave unit 20, which is acquired as sensor information. The haptic transmission unit 352 then controls the operation of the slave unit 20 so that the calculated velocity (or angular velocity) becomes a constant value. The haptic transmission unit 352 continues this control until the moving part, which moves due to the operation of the slave unit 20, moves a predetermined distance (Δx). This maintains uniform motion, which is the predetermined motion state. Meanwhile, the calculation unit 355 calculates the change in force (ΔF) by subtracting the force value at the start of the uniform motion from the force value at the end of the uniform motion. The calculation unit 355 then divides this change in force (ΔF) by the distance traveled while constant velocity motion was maintained (Δx), and calculates the result as a value indicating the elasticity (corresponding to the spring constant) of the material with which the slave device 20 came into contact (here, the material in the lesion with which the tip of the catheter came into contact).

[0069] When the calculation unit 355 calculates viscosity, the predetermined motion state is a state in which the catheter is inserted with uniform acceleration. Therefore, the haptic transmission unit 352 calculates the velocity (or angular velocity) in the same manner as when calculating elasticity. The haptic transmission unit 352 then calculates the acceleration (or angular acceleration) by further differentiating the calculated velocity (or angular velocity). The haptic transmission unit 352 then controls the operation of the slave device 20 so that the calculated acceleration (or angular acceleration) becomes a constant value. The haptic transmission unit 352 continues this control until the movable part, which is moved by the operation of the slave device 20, moves a predetermined distance (Δx). This maintains uniform acceleration, which is the predetermined motion state. Meanwhile, the calculation unit 355 calculates the change in force (ΔF) by subtracting the force value at the start of the uniform acceleration motion from the force value at the end of the uniform acceleration motion. The calculation unit 355 then divides this change in force (ΔF) by the distance traveled while the uniform acceleration motion was maintained (Δx), and calculates the result as a value indicating the viscosity of the substance with which the slave unit 20 came into contact (here, the substance in the lesion with which the tip of the catheter came into contact).

[0070] When the calculation unit 355 calculates inertia, the predetermined motion state is a state in which the catheter is inserted by uniform jerk motion (also called uniform jerk motion). Therefore, the haptic transmission unit 352 calculates the acceleration (or angular acceleration) in the same manner as when calculating viscosity. The haptic transmission unit 352 then calculates the jerk (or angular jerk) by further differentiating the calculated acceleration (or angular acceleration). The haptic transmission unit 352 then controls the operation of the slave device 20 so that the calculated jerk (or angular jerk) becomes a constant value. The haptic transmission unit 352 continues this control until the movable part, which is moved by the operation of the slave device 20, moves a predetermined distance (Δx). This maintains the uniform jerk motion, which is the predetermined motion state. Meanwhile, the calculation unit 355 calculates the change in force (ΔF) by subtracting the force value at the start of the uniform jerk motion from the force value at the end of the uniform jerk motion. The calculation unit 355 then divides this change in force (ΔF) by the distance traveled while the constant jerk motion was maintained (Δx), and calculates the result as a value indicating the inertia of the material with which the slave unit 20 has come into contact (here, the material in the lesion with which the tip of the catheter has come into contact).

[0071] In this way, the elasticity, viscosity, and inertia calculated by the calculation unit 355 based on the force value under the quantitative condition that the slave unit 20 is maintaining a predetermined motion state are stored in the physical property storage unit 372. Note that the calculation unit 355 does not necessarily need to calculate all of the elasticity, viscosity, and inertia, but it is sufficient to calculate at least one of them.

[0072] The presentation unit 356 presents the physical properties such as elasticity, viscosity, and inertia calculated by the calculation unit 355 to an operator or a person performing various analyses, inspections, etc. Presentation can be realized, for example, by displaying numerical values, graphs, etc. indicating the physical properties and force values ​​on the display L. Alternatively, presentation can also be realized by outputting the numerical values, etc. as sound from a speaker included in the output unit 316.

[0073] The timing of presentation by the presentation unit 356 is, for example, in real time at the same time as the calculation unit 355 calculates the physical property in the "measurement mode." Furthermore, presentation may be continued even when the mode is subsequently switched back to the "detection mode" or "insertion mode." Alternatively, presentation may be performed after the information presentation process is completed in order to perform various analyses, tests, and the like.

[0074] Furthermore, in the presentation, the presentation unit 356 may present the physical properties of the material calculated by the calculation unit 355 in a manner that allows comparison with the physical properties of a reference material. As described above, the reference physical properties are stored in the physical property storage unit 372. The reference physical properties of the material may be physical properties previously calculated by the calculation unit 355, or may be physical properties measured by an inspection method or tactile method different from the calculation by the calculation unit 355. The reference physical properties of the material may be, for example, standard values ​​of elasticity or the like of a lesion when the lesion is calcified. By comparing the standard values ​​of a lesion in such a specific state with the values ​​calculated by the presentation unit 356, the operator or the like can grasp the state of the lesion (here, the degree of calcification).

[0075] By referring to the presented physical characteristics, the operator can grasp the state of the lesion, etc., using objective indicators such as numerical values. In other words, the operator can grasp both the subjective indicator of the transmitted haptic sensation and the objective indicator. This allows for further support in addition to the support provided by the transmitted haptic sensation.

[0076] [Operation] Next, the operation of the information presentation system 1 will be described.

[0077] [Information Presentation Process] Figure 6 is a flowchart illustrating the flow of the information presentation process executed by the information presentation device 30. The information presentation process is initiated in response to an instruction to execute the information presentation process from the operator via the input unit 315 or an instruction to execute the information presentation process via communication from an external device (e.g., the master device 10) via the communication unit 318. In this embodiment, the information presentation process is initiated when the tip of the catheter is inserted a predetermined distance into the subject (e.g., inserted approximately 1 to 10 cm) either manually by an assistant assisting the operation of the slave device 20 or by remote operation from the master device 10. This prevents the control of the information presentation device 30 from becoming unstable when there is a large change in external force during the initial insertion period.

[0078] In step S11, the mode setting unit 354 sets the mode to insertion mode. In step S12, the sensor information acquisition unit 351 starts acquiring sensor information detected by various sensors installed in the master device 10 and the slave device 20. This acquisition of sensor information is performed in parallel with other steps until the end of this process. In addition, this acquired sensor information is stored in the control parameter storage unit 371 as time-series data.

[0079] In step S13, the haptic transmission unit 352 starts controlling the transmission of the haptic sensation based on the sensor information. This control of the transmission of the haptic sensation is performed in parallel with other steps until the present processing ends.

[0080] In step S14, the distance information acquisition unit 353 acquires distance information by performing calculations, analysis, etc. on various data. The distance information acquisition unit 353 also outputs the acquired distance information to the mode setting unit 354.

[0081] In step S15, the mode setting unit 354 determines whether to switch modes based on the distance information and a mode switching operation from the operator. That is, it determines whether to switch between the "insertion mode" and the "detection mode" based on the distance between the catheter tip and the lesion indicated by the distance information, and whether to switch between the "detection mode" and the "measurement mode" based on whether the operator has performed a mode switching operation. If switching to one of the modes is to be performed, step S15 returns Yes, and the process proceeds to step S16. On the other hand, if switching modes is not to be performed, step S15 returns No, and the process proceeds to step S17.

[0082] In step S16, the mode setting unit 354 switches the mode, i.e., switches and sets the mode to one of "insertion mode," "detection mode," and "measurement mode," depending on the determination result in step S15.

[0083] In step S17, the calculation unit 355 determines whether the currently set mode is "insertion mode," "detection mode," or "measurement mode." If it is "insertion mode" or "detection mode," it is determined to be "detection mode, insertion mode" in step S17, and the process proceeds to step S23. On the other hand, if it is "measurement mode," it is determined to be "measurement mode" in step S17, and the process proceeds to step S18.

[0084] In step S18, the calculation unit 355 starts calculating the force value. In step S19, the force haptic transmission unit 352 starts controlling the operation of the slave device 20 so as to achieve a predetermined motion state.

[0085] In step S20, the haptic transmission unit 352 determines whether the movable part, which moves in response to the operation of the slave device 20, has moved a predetermined distance (Δx). If the movable part has moved the predetermined distance (Δx), the determination in step S20 is Yes, and the process proceeds to step S21. On the other hand, if the movable part has not moved the predetermined distance (Δx), the determination in step S20 is No, and the process returns to step S18 and is repeated.

[0086] In step S21, the calculation unit 355 calculates the physical properties of the substance (here, the substance in the lesion that the catheter tip has come into contact with). In step S22, the presentation unit 356 starts presenting the physical properties calculated in step S21.

[0087] Note that if the mode is switched from "measurement mode" to "detection mode" and then switched back to "measurement mode" again, or if the operator does not switch from "measurement mode" to "detection mode," the processes of steps S18 to S22 are repeated multiple times. In this case, a certain physical property (e.g., elasticity) may be repeatedly calculated and the changes in the calculated values ​​may be presented in the form of a graph, or elasticity, viscosity, and inertia may be calculated and presented in order.

[0088] In step S23, the haptic transmission unit 352 determines whether a termination condition, which is a condition for terminating this process, has been satisfied. The termination condition is, for example, an operation by the operator via the input unit 315 to instruct the end of the information presentation process, or an instruction to terminate the information presentation process by communication from an external device (e.g., master device 10) via the communication unit 318. If the termination condition is satisfied, step S23 is determined as Yes, and this process is terminated. On the other hand, if the termination condition is not satisfied, step S23 is determined as No, and the process returns to step S14 and is repeated.

[0089] According to the information presentation process described above, while the slave unit 20 is quantitatively maintaining a predetermined motion state, it is possible to present the physical properties of a substance that the slave unit 20 has come into contact with, based on an external force input from the environment to the slave unit 20. Therefore, for example, the physical properties of a substance, such as a blood vessel, that the operator cannot directly touch can be quantitatively presented as information obtained by the slave unit 20 coming into contact with the substance (i.e., tactile information), thereby providing further support for the operator's operation. Furthermore, in addition to the operator's operation, the presented physical properties can also be used to provide further support to those performing various analyses, examinations, and the like. Therefore, the information presentation process can solve the problem of providing further support in addition to the support provided by the transmission of haptic sensations.

[0090] [Comparison of External Forces] Figure 7 is a schematic diagram showing time series changes in the external force input to the slave unit 20 from the environment when the slave unit 20 comes into contact with a substance in the above-described embodiment. Figure 7(A) is a schematic diagram showing the case where the slave unit 20 comes into contact with an acrylic wall as the first contact object. Figure 7(B) is a schematic diagram showing the case where the slave unit 20 comes into contact with a blood vessel suffering from arteriosclerosis as the second contact object. In each of Figures 7(A) and 7(B), the horizontal axis represents time [S], and the vertical axis represents the force value [N] indicating the magnitude of the external force input to the slave unit 20.

[0091] 7A, the catheter begins to move as the operation begins, and the force value gradually increases. Then, when the tip of the catheter comes into contact with the first contact object, the contact force with the first contact object increases, and the force value increases sharply. Once the force value reaches a certain level, it then remains at a substantially constant magnitude. Here, the period from when the force value begins to increase sharply until the force value reaches a certain level is defined as comparison period P1.

[0092] Next, focusing on Figure 7(B), which is generally similar to Figure 7(A), the catheter begins to move as the operation begins, and the force value gradually increases. Then, when the tip of the catheter contacts the second contact object, the contact force with the second contact object increases, and the force value increases sharply. Once the force value reaches a certain level, it subsequently remains at a substantially constant magnitude. Here, the period from when the force value begins to increase sharply until a length of time equal to the comparison period P1 has elapsed is defined as comparison period P2.

[0093] Comparing the comparison periods P1 and P2, it can be seen that the change in force value (i.e., the change obtained by subtracting the force value at the start of the comparison period from the force value at the end of the comparison period) is greater during the comparison period P1 than during the comparison period P2. This can also be seen from the fact that the slope of the graph is greater during the comparison period P1 than during the comparison period P2. This comparison clearly shows that if the contacted substance is different, its physical properties will differ, and this will be reflected in the change in the force value. Therefore, as in the above-described embodiment, it is possible to calculate the physical properties of the contacted substance based on the change in the force value.

[0094] [Variation 1] In the above-described embodiment, an example has been described in which the catheter is inserted and then operated by an actuator until it reaches the lesion, but this is not limiting. For example, the catheter may be manually inserted to the vicinity of the lesion, and insertion may be started using the master unit 10 and the slave unit 20 in a specific section near the lesion, and the actuator may be operated in the same way as in the "detection mode" by controlling the transmission of haptic sensations, and information presentation processing may be performed to switch to the "measurement mode."

[0095] Fig. 8 is a schematic diagram showing the configuration of an information presentation system 1 that performs an information presentation process after an operator manually inserts the catheter of the slave device 20. As shown in Fig. 8, the information presentation system 1 of this modified example has an operation lever (grip) or the like provided on the catheter of the slave device 20, allowing manual operation by the operator. Furthermore, of the linear actuators provided in the information presentation system 1 of the first embodiment shown in Fig. 1, the information presentation system 1 of this modified example only includes detection actuators 104, 204, and does not include insertion actuators 103, 203.

[0096] When the operator manually inserts the catheter, the catheter is released from the movement control by the detection actuator 204 and the rotation actuator 205 in the slave unit 20, and can be operated in the same manner as a conventional catheter. At this time, it is assumed that the operator inserts the catheter to a position just before the vicinity of the lesion, and this state is set as the initial state, and the information presentation process is started.

[0097] When the information presentation process is started, the catheter is held for movement control by the detection actuator 204 and the rotation actuator 205, and in response to an operation on the master unit 10, the slave unit 20 moves the catheter, and control for transmitting haptic feedback by the information presentation device 30 is started. Then, by operating the actuator in the same way as when the "detection mode" was set in the above-described embodiment, and by performing the information presentation process to enable switching to the "measurement mode," this modification also makes it possible to present the physical properties of a substance calculated based on the force value. According to this modification, the distance over which the catheter is moved by the actuator is relatively short, and therefore it is sufficient to provide an actuator with a short stroke, such as a voice coil motor, thereby enabling the master unit 10 and the slave unit 20 to be made smaller and lighter.

[0098] [Other Modifications] In the above-described embodiment, haptic transmission of a force in the thrust direction (advance / retraction direction) of the catheter between the master unit 10 and the slave unit 20 has been described, but this is not limiting. For example, haptic transmission of a force related to rotation around a rotation axis along the advance / retraction direction or operation of an end effector may also be performed between the master unit 10 and the slave unit 20. Furthermore, for example, in the above-described embodiment, a case in which a catheter is remotely operated by the information presentation system 1 has been described as an example, but this is not limiting. In other words, various devices can be remotely operated by the information presentation system 1, and examples of such devices include various devices having linear components, such as medical devices such as forceps or endoscopes.

[0099] In the above embodiment, the actuators of the master device 10 and the actuators of the slave device 20 are associated one-to-one to transmit haptics. However, this is not limiting. That is, it is possible to transmit haptics by associating multiple actuators of the master device 10 with one actuator of the slave device 20, or by associating one actuator of the master device 10 with multiple actuators of the slave device 20. It is also possible to transmit haptics by associating multiple actuators of the master device 10 with multiple actuators of the slave device 20. As an example, it is possible to transmit haptics by associating the insertion actuator 203 and the detection actuator 204 of the slave device 20 shown in FIG. 3 with the insertion actuator 103 of the master device 10. In this case, it is not necessary to provide the detection actuator 104 of the master device 10, thereby reducing costs and the weight of the device.

[0100] Furthermore, in the above-described embodiment, an example configuration was described in which the actuators for advancing and retracting the catheter of the slave unit 20 included the insertion actuator 203 and the detection actuator 204, but this is not limited to this. That is, the catheter of the slave unit 20 may be advanced and retracted by a single actuator as long as the actuator satisfies the required performance in terms of stroke and accuracy of operation. In this case, the process of switching between insertion mode and detection mode is omitted, and the actuator operates in the same manner as when set to "detection mode" in the above-described embodiment, and information presentation processing is performed to enable switching to "measurement mode." This makes it possible to present the physical properties of a substance calculated based on the force value, even in this modified example.

[0101] Furthermore, in the above-described embodiment, the mode setting unit 354 switches to the "insertion mode" when the catheter tip has not reached the vicinity of the lesion based on the distance between the catheter tip and the lesion acquired as distance information by the distance information acquisition unit 353, and switches to the "detection mode" when the catheter tip has reached the vicinity of the lesion. However, this is not limited to this. When the catheter is inserted into the subject, various external forces input from the environment change between, for example, when the catheter is moving through an artery and when the catheter has reached a lesion in a constricted heart, etc., and the force value also changes accordingly. Therefore, the mode setting unit 354 calculates the force value in the same manner as the calculation unit 355, determines the insertion state of the catheter based on this force value, and switches between the "insertion mode" and the "detection mode." In this case, for example, the mode setting unit 354 switches to the "insertion mode" when the force value is less than a predetermined value, and switches to the "detection mode" when the force value is equal to or greater than the predetermined value.

[0102] Furthermore, in the above-described embodiment, the mode setting unit 354 is described as switching between the "detection mode" and the "measurement mode" based on a mode switching operation from the operator via the input unit 315 or a mode switching operation via communication from an external device (e.g., the master device 10) via the communication unit 318. However, this is not limited to this. For example, if it is possible to accurately determine whether the catheter tip has reached the vicinity of a lesion based on distance information, etc., the mode setting unit 354 may switch between the "detection mode" and the "measurement mode" based on the distance information, etc., without requiring a switching operation from the operator, etc. For example, after switching to the "detection mode," the mode setting unit 354 may switch to the "measurement mode" without requiring a switching operation from the operator, etc., if it determines that the catheter tip has further approached the lesion based on the distance information, etc. Then, when the physical properties of the substance are calculated in the "measurement mode" and presented, the mode setting unit 354 may switch to the "detection mode" without requiring a switching operation from the operator, etc.

[0103] [Configuration Example] As described above, the information presentation system 1 according to this embodiment includes the master device 10 to which an operator's operation is input, and the slave device 20 that operates in response to the operation input to the master device 10. The information presentation system 1 also includes a haptic transmission unit 352, a calculation unit 355, and a presentation unit 356. The haptic transmission unit 352 controls the transmission of haptic sensations between the master device 10 and the slave device 20. The calculation unit 355 calculates physical properties of a substance that the slave device 20 has come into contact with, based on an external force input from the environment to the slave device 20 while the slave device 20 is maintaining a predetermined motion state. The presentation unit 356 presents the physical properties of the substance calculated by the calculation unit 355. In this way, the information presentation system 1 can present the physical properties of a substance that the slave device 20 has come into contact with, based on an external force input from the environment to the slave device 20, under the quantitative condition that the slave device 20 is maintaining a predetermined motion state. Therefore, for example, the physical properties of a substance such as a blood vessel that the operator cannot directly touch can be quantitatively presented as information obtained by the slave device 20 contacting the substance (i.e., tactile information), thereby providing further support for the operator's operation. Furthermore, in addition to the operator's operation, the presented physical properties can also be used to provide further support to those who perform various analyses, examinations, etc. Therefore, the information presentation system 1 can solve the problem of providing further support in addition to the support provided by the transmission of haptic sensations.

[0104] The calculation unit 355 calculates elasticity as a physical property of a material when the slave unit 20 maintains a state of uniform motion as a predetermined motion state. This makes it possible to quantitatively present the elasticity of a material such as a blood vessel that the operator cannot directly touch.

[0105] The calculation unit 355 calculates viscosity as a physical property of a substance when the slave unit 20 is maintaining a state of uniformly accelerated motion as a predetermined operating state. This makes it possible to quantitatively present the viscosity of substances such as blood vessels that the operator cannot directly touch.

[0106] The calculation unit 355 calculates the inertia as a physical property of a material when the slave unit 20 maintains a state of constant jerk motion as a predetermined operating state. This makes it possible to quantitatively present the inertia of a material, such as a blood vessel, that the operator cannot directly touch.

[0107] The calculation unit 355 determines whether the slave unit 20 has maintained the predetermined motion state based on the distance traveled by the slave unit 20 since the predetermined motion state was initiated. This makes it possible to determine whether the predetermined motion state has been maintained by utilizing sensor information for controlling the transmission of haptics.

[0108] The presentation unit 356 presents the physical properties of the substance calculated by the calculation unit 355 in a manner that allows comparison with the physical properties of a reference substance. This makes it possible to compare quantitative physical properties based on information obtained by the slave device 20 coming into contact with the substance (i.e., tactile information) with the reference physical properties, for example, and provides further support to the operator, etc.

[0109] The presentation unit 356 presents the physical properties of the substance calculated by the calculation unit 355 to the operator operating the master device 10. This allows real-time support to be provided to the operator operating the master device 10.

[0110] As described above, the information presentation device 30 according to this embodiment includes a haptic transmission unit 352, a calculation unit 355, and a presentation unit 356. The haptic transmission unit 352 controls the transmission of haptics between the master device 10, to which an operator's operation is input, and the slave device 20, which operates in response to the operation input to the master device 10. The calculation unit 355 calculates the physical properties of a material that the slave device 20 comes into contact with, based on an external force input from the environment to the slave device 20 while the slave device 20 is maintaining a predetermined motion state. The presentation unit 356 presents the physical properties of the material calculated by the calculation unit 355. This configuration of the information presentation device 30 can also solve the problem of providing further assistance in addition to the assistance provided by haptic transmission, as with the information presentation system 1 described above.

[0111] The present invention is not limited to the above-described embodiments, and modifications, improvements, and the like are included within the scope of achieving the object of the present invention. For example, in addition to being realized as the information presentation system 1 in the above-described embodiments, the present invention can also be realized as an information presentation device that controls the information presentation system 1, an information presentation method constituted by steps executed in the information presentation system 1, or a program executed by a processor to realize the functions of the information presentation system 1. Furthermore, in the above-described embodiments, an example has been described in which the information presentation device 30 is realized as an independent device, but the functions of the information presentation device 30 can be implemented in either the control unit 101 of the master device 10 or the control unit 201 of the slave device 20, or can be distributed and implemented in both.

[0112] Furthermore, the processes in the above-described embodiments can be executed by either hardware or software. That is, it is sufficient that the information presentation system 1 is provided with a function capable of executing the above-described processes, and the functional and hardware configurations for realizing these functions are not limited to the above-described examples. When the above-described processes are executed by software, the programs constituting the software are installed on a computer from a network or a storage medium.

[0113] The storage medium for storing the program may be a removable medium distributed separately from the device itself, or may be a storage medium pre-installed in the device itself. Removable media may be, for example, a semiconductor memory, a magnetic disk, an optical disk, or a magneto-optical disk. Optical disks may be, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disks), Blu-ray Discs (registered trademark), etc. Magneto-optical disks may be, for example, MDs (Mini-Disks), etc. Furthermore, storage media pre-installed in the device itself may be, for example, a ROM (Read Only Memory) or hard disk in which the program is stored, or a semiconductor memory.

[0114] The above-described embodiment shows an example of application of the present invention and does not limit the technical scope of the present invention. In other words, the present invention can be modified in various ways, such as by omission or substitution, without departing from the spirit of the present invention, and various embodiments other than the above-described embodiment can be adopted. The various embodiments and modifications that the present invention can adopt are included in the scope of the invention described in the claims and their equivalents.

[0115] 1 Information presentation system, 10 Master device, 20 Slave device, 30 Information presentation device, 40 Network, L Display, C Camera, FT Functional force / speed allocation conversion block, FC Ideal force source block, PC Ideal speed (position) source block, IFT Inverse conversion block, S Control target system, 101, 201 Control unit, 102, 202 Communication unit, 103, 203 Insertion actuator, 104, 204 Detection actuator, 105, 205 Rotation actuator, 106, 206 Operation actuator, 107, 108, 207, 208 Linear encoder, 109, 110, 209, 210 Rotary encoder, 111 to 114, 211 to 214 Driver, 311 Processor, 312 ROM, 313 RAM, 314 Bus, 315 Input unit, 316 Output unit, 317 storage unit, 318 communication unit, 319 drive, 331 removable media, 351 sensor information acquisition unit, 352 force haptic transmission unit, 353 distance information acquisition unit, 354 mode setting unit, 355 calculation unit, 356 presentation unit, 371 control parameter storage unit, 372 physical characteristic storage unit

Claims

1. An information presentation system including a master device to which an operation by an operator is input, and a slave device that operates in response to the operation input to the master device, a control means for controlling transmission of haptic sensations between the master device and the slave device; a calculation means for calculating physical properties of a substance that the slave unit comes into contact with based on an external force input from the environment to the slave unit while the slave unit is maintaining a predetermined state of motion; a display means for displaying the physical properties of the substance calculated by the calculation means; An information presentation system comprising:

2. the calculation means calculates elasticity as the physical property of the material when the slave device maintains a state of uniform motion as the predetermined motion state.

2. The information presentation system according to claim 1, wherein:

3. the calculation means calculates viscosity as the physical property of the substance when the slave device maintains a state of uniformly accelerated motion as the predetermined operating state.

2. The information presentation system according to claim 1, wherein:

4. the calculation means calculates inertia as the physical property of the material when the slave unit maintains a state of constant jerk motion as the predetermined operating state.

2. The information presentation system according to claim 1, wherein:

5. the calculation means determines whether the slave unit has maintained the predetermined motion state based on a movement distance of the slave unit since the predetermined motion state was initiated.

5. The information presentation system according to claim 1, wherein the information presentation system comprises: a display unit configured to display a display image;

6. the presentation means presents the physical property of the substance calculated by the calculation means and the physical property of the substance as a reference in a manner that allows comparison.

5. The information presentation system according to claim 1, wherein the information presentation system comprises: a display unit configured to display a display image;

7. the presentation means presents the physical properties of the substance calculated by the calculation means to the operator of the master device.

5. The information presentation system according to claim 1, wherein the information presentation system comprises: a display unit configured to display a display image;

8. a control means for controlling transmission of haptic sensations between a master device to which an operation by an operator is input and a slave device which operates in response to the operation input to the master device; a calculation means for calculating physical properties of a substance that the slave unit comes into contact with based on an external force input from the environment to the slave unit while the slave unit is maintaining a predetermined state of motion; a display means for displaying the physical properties of the substance calculated by the calculation means; An information presentation device comprising:

9. 1. An information presentation method executed in an information presentation system including a master device to which an operation by an operator is input, and a slave device that operates in response to the operation input to the master device, a control step of controlling transmission of haptic sensations between the master device and the slave device; a calculation step of calculating physical properties of a substance that the slave unit has come into contact with based on an external force input from the environment to the slave unit while the slave unit is maintaining a predetermined state of motion; a presentation step of presenting the physical properties of the substance calculated in the calculation step; An information presentation method comprising:

10. a control function for controlling the transmission of haptic sensations between a master device to which an operator's operation is input and a slave device that operates in response to the operation input to the master device; a calculation function for calculating physical properties of a substance that the slave unit comes into contact with based on an external force input from the environment to the slave unit while the slave unit is maintaining a predetermined motion state; a presentation function that presents the physical properties of the substance calculated by the calculation function; A program characterized by causing a computer to realize the above.