Force sense transmission device

Abstract

To individually transmit the direction of force sense received from an operation object by a part of an operation part to an operator.SOLUTION: A force sense transmission device of remote operation includes a plurality of sheets which are respectively deformed in a specific direction to force sense, wherein the plurality of sheets includes the plurality of sheets arranged in a state of being combined in various directions, and a control unit for performing control so as to vibrate a corresponding sheet detected by an operation unit for operating an operation object according to a direction of force sense received from the operation object by a part of the operation unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The technology of the present disclosure relates to a force feedback device for remote operation.

Background Art

[0002] Patent Document 1 discloses a vibration control system. The vibration control system determines the content of virtual force feedback to be presented to the user according to the content of the content that changes according to the user's input such as a game, and determines the content of the virtual force feedback vibration that the vibration device should generate according to the determined content of the virtual force feedback, and generates vibrations along the content with the vibration device. The vibration device is a device that the user holds and uses with their hand, and a vibration mechanism is built in. When the vibration mechanism vibrates, the entire vibration device vibrates, and the vibration is transmitted to the hand of the user holding the vibration device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional vibration devices vibrate the entire vibration device and transmit the vibration to the hand of the user holding the vibration device. Therefore, the direction of the force feedback received by the operating part of the operation target for remote operation from the operation target cannot be individually transmitted to the user.

[0005] The technology of the present disclosure aims to provide a force feedback device for remote operation that can individually transmit the direction of the force feedback received by the operating part of the operation target from the operation target to the operator.

Means for Solving the Problems

[0006] To achieve the above object, an aspect of the technology of the present disclosure is a force feedback device for remote operation, including a plurality of sheets that deform in specific directions respectively, where the plurality of sheets are arranged in a state where the specific directions are combined in various directions, and a control unit that controls, among the plurality of sheets corresponding to a part of an operation unit that operates an operation target, the sheet corresponding to the specific direction to deform in the direction of the force feedback received by the part of the operation unit from the operation target.

Effect of the Invention

[0007] An object of the technology of the present disclosure is to provide a force feedback device for remote operation that can individually transmit to an operator the direction of the force feedback received by a part of an operation unit that operates an operation target from the operation target.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 2C

Figure 2D

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Mode for Carrying Out the Invention

[0009] Hereinafter, embodiments of the technology of the present disclosure will be described with reference to the drawings.

[0010] (Configuration) The master-slave system of the present embodiment includes a master system and a slave system.

[0011] FIG. 1 is a schematic diagram of an example of the master system 10. As shown in FIG. 1, the master system 10 includes a glove-type movable part 12 and a master control device 14. The glove-type movable part 12 includes a plurality of movable parts, specifically, a thumb part 12N1, an index finger part 12N2, a middle finger part 12N3, a ring finger part 12N4, a little finger part 12N5, and a palm part 12N6. The operator inserts a hand (in the example shown in FIG. 1, the left hand) into the glove-type movable part 12, and the operator inserts the thumb, index finger, middle finger, ring finger, and little finger of the operator's left hand into the thumb part 12N1, the index finger part 12N2, the middle finger part 12N3, the ring finger part 12N4, and the little finger part 12N5, respectively. As a result, the palm of the operator's left hand is disposed on the palm part 12N6. When the operator moves, for example, the thumb and index finger of the left hand so that the distance between them becomes narrower, the thumb part 12N1 and the index finger part 12N2 move so that the distance between them becomes narrower.

[0012] A plurality of sheets 12D that vibrate in specific directions with respect to the force sense are arranged on each of the thumb part 12N1 to the palm part 12N6. Each of the plurality of sheets 12D is arranged in a state combined in various directions. Each of the plurality of sheets 12D is connected to the master control device 14. The master system 10 is an example of the "remote operation force feedback device" of the technology of the present disclosure.

[0013] FIG. 2A is a cross-sectional view showing an example of the configuration of the sheet 12D. FIG. 2B is a cross-section showing an example of the configuration of the conversion unit 12D1. FIG. 2C is a cross-section showing an example of the configuration of the conversion unit 12D1 that bends in one direction and deforms as a whole. FIG. 2D is a cross-section showing an example of the configuration of the conversion unit 12D1 that does not deform in directions other than one direction.

[0014] As shown in FIG. 2A, the sheet 12D includes a film-shaped vibrating portion 12D2 that vibrates when a voltage is applied and operates, and a film-shaped conversion portion 12D1 that converts the vibration into deformation only in a specific direction. The sheet 12D is disposed in the glove-shaped movable portion 12 such that the conversion portion 12D1 side faces the finger.

[0015] As shown in FIG. 2B, the conversion unit 12D1 includes a plurality of nanosheets 12D11 and a gel 12D12 disposed between the nanosheets 12D11. As shown in FIG. 2C, the conversion unit 12D1 can be deformed as a whole when the nanosheets 12D11 bend in one direction, but as shown in FIG. 2D, the nanosheets 12D11 are stretched and do not deform in directions other than one direction. As described above, each of the plurality of sheets 12D is arranged in a combined state in various directions, for example, in the up, down, left, right, and other directions, and is arranged in a grid pattern.

[0016] FIG. 3 is a schematic diagram of an example of the slave system 55. As shown in FIG. 3, the slave system 55 includes a robot arm 51, an operation unit 52 disposed at the tip of the robot arm 51, and a slave control device 50.

[0017] The operation unit 52 includes a plurality of operation parts, specifically, a thumb operation part 52N1, an index finger operation part 52N2, a middle finger operation part 52N3, a ring finger operation part 52N4, a little finger operation part 52N5, and a palm operation part 52N6. The thumb operation part 52N1 to the palm operation part 52N6 respectively correspond to the thumb part 12N1 to the palm part 12N6 in the glove-type movable part 12 of the master system 10 shown in FIG. 1. When the thumb part 12N1 to the palm part 12N6 move, the thumb operation part 52N1 to the palm operation part 52N6 are each controlled to move in correspondence with the direction and speed of the movement.

[0018] FIG. 4 is a block diagram showing an example of the control system of the master control device 14. As shown in FIG. 4, the control system of the master control device 14 includes a computer 30, a driver 20 that moves each of the plurality of sheets 12D, a posture acquisition sensor that acquires the postures of the thumb part 12N1 to the palm part 12N6 of the glove-type movable part 12, for example, a camera 17 that images the thumb part 12N1 to the palm part 12N6, and a communication device 45. The posture acquisition sensor is not limited to the camera 17. For example, a method of acquiring the posture of the joint part by a signal from an encoder provided at an indirect part (bending part) between the thumb part 12N1 to the palm part 12N6 may be used. Further, the posture acquisition sensor may be a depth sensor (three-dimensional measurement sensor) that acquires distance information to each part of the thumb part 12N1 to the palm part 12N6. The depth sensor may be a stereo camera system that is composed of two cameras and calculates distance information from the parallax information of each camera, or a ToF (Time-of-Flight) camera system that emits near-infrared light, measures the time until it is reflected by an object and returns, and calculates distance information from the time.

[0019] The computer 30 includes a processor 32, a non-volatile memory (NVM) 34, a random access memory (RAM) 36, and an input / output (I / O) port 38. The processor 32, the NVM 34, the RAM 36, and the input / output (I / O) port 38 are interconnected by a bus 40. The driver 20, the camera 17, and the communication device 45 are connected to the input / output (I / O) port 38. The processor 32 is an example of the "control unit" of the technology of the present disclosure.

[0020] The processor 32 is a processing device including a digital signal processor (DSP), a central processing unit (CPU), and a graphics processing unit (GPU). The DSP and the GPU operate under the control of the CPU and are responsible for executing each of the processes described below. Here, as an example of the processor 32, a processing device including a DSP, a CPU, and a GPU is given, but this is merely an example. The processor 32 may be one or more CPUs and DSPs integrating the GPU function, or one or more CPUs and DSPs not integrating the GPU function, or a tensor processing unit (TPU) may be mounted.

[0021] The NVM 34 is a non-volatile storage device that stores programs and various parameters. Examples of the NVM 34 include flash memory (e.g., electrically erasable and programmable read-only memory (EEPROM)). A force perception expression program 34P is stored in the NVM 34.

[0022] The RAM 36 is a memory in which information is temporarily stored and is used as a work memory by the processor 32. Examples of the RAM 36 include dynamic random access memory (DRAM) or static random access memory (SRAM).

[0023] When the force perception expression program 34P is read into the RAM 36 and executed by the processor 32, the processor 32 functions as a calculation unit 32A, a communication processing unit 32B, a determination unit 32C, and a drive processing unit 32D.

[0024] FIG. 5 is a block diagram showing an example of the control system of the slave control device 50. As shown in FIG. 5, the control system of the slave control device 50 includes a computer 60, a motor 53, an encoder 54, and a torque sensor 56, and a communication device 58. The motor 53, the encoder 54, and the torque sensor 56 are provided at each joint of the robot arm 51 and at each joint of each of the thumb operation portions 52N1 to the palm operation portions 52N6 of the operation unit 52.

[0025] Since the computer 60 has the same configuration as the computer 30, its description is omitted. However, a movement processing program 64P is stored in the NVM 64. When the movement processing program 64P is read into the RAM 66 and executed by the processor 62, the processor 62 functions as a communication processing unit 62A, a movement processing unit 62B, and a calculation unit 62C.

[0026] (Operation) FIG. 6 is a timing chart showing an example of the processing of the master-slave system.

[0027] As shown in FIG. 6, in the master control device 14, the processor 32 of the computer 30 executes the force perception expression program 34P (steps 82, 84, 86, 88, 90), and in the slave control device 50, the processor 62 of the computer 60 executes the movement processing program 64P (steps 81, 83, 85, 87).

[0028] In step 82, based on the image data obtained by the camera 17 capturing the thumb part 12N1 to the palm part 12N6 of the glove-type movable part 12 at regular time intervals, the calculation unit 32A of the master control device 14 calculates the movement amount and movement direction of each of the thumb part 12N1 to the palm part 12N6 from the differences in the images of each of the thumb part 12N1 to the palm part 12N6 in each image. When any of the thumb part 12N1 to the palm part 12N6 has not moved, the movement amount is calculated as 0 and the movement direction is calculated as "none".

[0029] In step 84, the communication processing unit 32B transmits, via the communication device 45, the calculated movement amount and movement direction of the thumb part 12N1 to the palm part 12N6 to the slave control device 50 corresponding to the identification data of each of the thumb part 12N1 to the palm part 12N6.

[0030] In step 81, the communication processing unit 62A of the slave control device 50 receives, via the communication device 58, the calculated movement amount and movement direction of the thumb part 12N1 to the palm part 12N6 corresponding to the identification data of each of the thumb part 12N1 to the palm part 12N6.

[0031] In step 83, based on the calculated movement amount and movement direction of the thumb part 12N1 to the palm part 12N6 and the signals from the encoders 54 of the thumb operation part 52N1 to the palm operation part 52N6 of the operation unit 52, the movement processing 62D moves the thumb operation part 52N1 to the palm operation part 52N6 of the operation unit 52 via the motors 53 of the thumb operation part 52N1 to the palm operation part 52N6 corresponding to the magnitude and direction of the movement of the thumb part 12N1 to the palm part 12N6.

[0032] In step 85, based on the torque detected by the torque sensors 56 arranged at the joints of the thumb operation part 52N1 to the palm operation part 52N6, the calculation unit 62C calculates the magnitude and direction of the force received by the thumb operation part 52N1 to the palm operation part 52N6 from the operation target 15.

[0033] In step 87, the communication processing unit 62A transmits, via the communication device 58, the magnitudes and directions of the forces received by the thumb operation portion 52N1 to the palm operation portion 52N6 from the operation target 15 to the master control device 14.

[0034] In step 86, the communication processing unit 32B of the master control device 14 receives, via the communication device 45, the magnitudes and directions of the forces received by the thumb operation portion 52N1 to the palm operation portion 52N6 from the operation target 15.

[0035] In step 88, the determination unit 32C determines a sheet 12D that corresponds to the thumb operation portion 52N1 to the palm operation portion 52N6 and whose specific direction corresponds to the direction of the force, based on the direction of the force received by the thumb operation portion 52N1 to the palm operation portion 52N6 from the operation target 15.

[0036] In step 90, the drive processing unit 32D drives, via the driver 20, by applying a voltage of a magnitude corresponding to the magnitude of the force received by the thumb operation portion 52N1 to the palm operation portion 52N6 from the operation target 15 only to the determined sheet 12D. As a result, only the sheet 12D vibrates, and the magnitude of the vibration is proportional to the magnitude of the force received from the operation target 15. Thus, only the sheet 12D deforms by an amount proportional to the magnitude of the force received from the operation target 15.

[0037] FIG. 7 shows an example of a state in which the operation target 15 is held by the thumb operation portion 52N1 and the index finger operation portion 52N2 on the slave system 55 side where the distance between the thumb portion 12N1 and the index finger portion 12N2 on the master system 10 side has become narrow, and a downward force F is applied to the thumb operation portion 52N1 and the index finger operation portion 52N2, and only the sheet 12D whose specific direction is the downward F direction among the plurality of sheets 12D arranged on the thumb portion 12N1 corresponding to the thumb operation portion 52N1 and the index finger portion 12N2 corresponding to the index finger operation portion 52N2 vibrates.

[0038] As shown in FIG. 7, when the operator pretends to narrow the distance between the thumb and index finger of the left hand so as to pinch the object to be operated with the thumb and index finger of the left hand, based on the image data as described above, the amount of movement and the direction of movement of each of the thumb portion 12N1 and the index finger portion 12N2 are calculated. Based on the calculation result, the distance between the thumb operation portion 52N1 and the index finger operation portion 52N2 is narrowed, and the object to be operated 15 is pinched and held by the thumb operation portion 52N1 and the index finger operation portion 52N2. Since the object to be operated 15 has a certain weight, a downward force F acts on the thumb operation portion 52N1 and the index finger operation portion 52N2. The downward force F is calculated based on the detection results of torque sensors provided at the respective joints of the thumb operation portion 52N1 and the index finger operation portion 52N2. From the signals from the respective encoders, the direction (downward F) of the force applied to the thumb operation portion 52N1 and the index finger operation portion 52N2 is calculated.

[0039] The magnitude and direction (downward F) of the force applied to the thumb operation portion 52N1 and the index finger operation portion 52N2 are transmitted to the master control device 14.

[0040] Among the plurality of sheets 12D arranged on the thumb portion 12N1 corresponding to the thumb operation portion 52N1 and the index finger portion 12N2 corresponding to the index finger operation portion 52N2, only the sheet 12D in the specific direction of which the direction is downward F is activated. Also, a voltage of a magnitude proportional to the magnitude of the force applied to the thumb operation portion 52N1 and the index finger operation portion 52N2 is applied only to the sheet 12D, and the sheet 12D vibrates with a magnitude corresponding to the magnitude of the applied voltage, and the sheet 12D deforms in proportion to the magnitude of the voltage.

[0041] (Function) Therefore, in the present embodiment, the direction of the sense of force received by the operation portion of the operation unit for operating the object to be operated can be individually transmitted to the operator.

[0042] (Other examples) FIG. 8 is a diagram showing an example of a state in which, when a downward force F is applied to a thumb operation portion 52N1 and an index finger operation portion 52N2, which have a reduced interval due to a reduced interval between a thumb portion 12N1 and an index finger portion 12N2, and hold an operation target 15 larger than the operation target shown in FIG. 7, only a plurality of sheets (sheets in which a specific direction is the direction of the downward force F) arranged in the thumb portion 12N1 and the index finger portion 12N2 and more numerous than the sheet 12D shown in FIG. 7 are deformed.

[0043] In the example shown in FIG. 8, since an operation target larger than the operation target shown in FIG. 7 is held, a plurality of sheets (only the sheets in which a specific direction is the direction of the downward force F) arranged in the thumb portion 12N1 and the index finger portion 12N2 and more numerous than the sheet 12D shown in FIG. 7 are deformed.

[0044] As described above, in the example shown in FIG. 8, since a plurality of sheets (only the sheets in which a specific direction is the direction of the downward force F) arranged in the thumb portion 12N1 and the index finger portion 12N2 and more numerous than the sheet 12D shown in FIG. 7 are deformed, the operator can actually feel that an operation target larger than the operation target shown in FIG. 7 is held.

[0045] FIG. 9 is a diagram showing an example of a state in which, when an upward force is applied to the thumb operation portion 52N1 and a downward force F is applied to the index finger operation portion 52N2, which have a reduced interval due to a reduced interval between the thumb portion 12N1 and the index finger portion 12N2, and hold an operation target that rotates rightward R, only the sheet in which the specific direction is the upward R2 direction among the plurality of sheets arranged in the thumb portion 12N1 is deformed, and only the sheet in which the specific direction is the downward R1 direction among the plurality of sheets arranged in the index finger portion 12N2 is deformed.

[0046] In the example shown in FIG. 9, the operator feels as if an upward force is applied to the thumb and a downward force F is applied to the index finger. As a result, the operator can actually feel that the held operation target 15 is rotating rightward.

[0047] FIG. 10 is a diagram showing an example of a state in which the operation target is held and lifted by the thumb operation portion 52N1 and the index finger operation portion 52N2 whose interval has become narrow due to the interval between the thumb portion 12N1 and the index finger portion 12N2 becoming narrow and rising U, and a downward F force is applied to the thumb operation portion 52N1 and the index finger operation portion 52N2 and the position rises, causing each of a plurality of sheets arranged on the thumb portion 12N1 and the index finger portion 12N2 to deform.

[0048] In the example shown in FIG. 7, among the plurality of sheets 12D arranged on the thumb portion 12N1 and the index finger portion 12N2, only the sheet 12D whose specific direction is the downward F direction deforms. In contrast, in the example shown in FIG. 10, each of the plurality of sheets arranged on the thumb portion 12N1 and the index finger portion 12N2 deforms simultaneously regardless of the direction of the force sensation.

[0049] In the example shown in FIG. 7, the operator feels that the sheet deforms downward F in the thumb and index finger, whereas in the example shown in FIG. 10, the operator feels that the sheet deforms simultaneously up, down, left, and right, that is, regardless of the direction of the force sensation, in the thumb and index finger.

[0050] Incidentally, it is predetermined that the sheet deforming simultaneously up, down, left, and right, that is, regardless of the direction of the force sensation, represents that the operation target is being lifted.

[0051] Therefore, in the example shown in FIG. 10, the operator can actually feel that the operation target is being lifted. Therefore, the following aspect is proposed for the force sensation transmission device according to claim 1 or claim 2. "The moving direction of the operation target is detected, and when the detected moving direction of the operation target is the upward direction, the control unit controls a plurality of the sheets corresponding to the portion of the operation unit that receives force from the operation target to operate regardless of the direction of the force sensation, a force sensation transmission device."

[0052] FIG. 11 is a diagram showing an example of a state in which a plurality of sheets arranged on the thumb portion 12N1 and the index finger portion 12N2 are deformed more greatly than the magnitude of deformation in FIG. 7 when a stronger force than in the case of FIG. 7 causes the distance between the thumb portion 12N1 and the index finger portion 12N2 to narrow, resulting in a stronger force being applied to the operation target 15 by the narrowed thumb operation portion 52N1 and index finger operation portion 52N2 than in the case of FIG. 7.

[0053] In the example shown in FIG. 11, the magnitude of the force sensation received by the thumb operation portion 52N1 to the palm operation portion 52N6 of the operation unit 52 from the operation target 15 is detected, and according to the detected magnitude of the force sensation, the amount of change in the sheet 12D of the corresponding thumb portion 12N1 to the palm portion 12N6 changes. The amount of deformation of each of the plurality of sheets arranged on the thumb portion 12N1 and the index finger portion 12N2 is larger than that in the example shown in FIG. 10. In the example shown in FIG. 11, it is shown that a stronger force than in the case of FIG. 7 is applied to the operation target 15 by the thumb operation portion 52N1 and the index finger operation portion 52N2, and a crack 15K has occurred in the operation target 15. Also in the example shown in FIG. 11, each of the plurality of sheets arranged on the thumb portion 12N1 and the index finger portion 12N2 deforms simultaneously regardless of the direction of the force sensation.

[0054] Incidentally, it is predetermined that the fact that the amount of deformation is larger than that in the example shown in FIG. 10 means that the magnitude of the force sensation received by the thumb operation portion 52N1 to the palm operation portion 52N6 of the operation unit 52 from the operation target 15 is larger than that in the example shown in FIG. 10. For example, it is predetermined that a force sufficient to break the operation target 15 (for example, cause a crack 15K) has been applied.

[0055] Therefore, in the example shown in FIG. 11, the operator can realize that the magnitude of the force sensation received by the thumb operation portion 52N1 to the palm operation portion 52N6 of the operation unit 52 from the operation target 15 is larger than that in the example shown in FIG. 10, for example, a force sufficient to break the operation target 15 (for example, cause a crack 15K) has been applied.

Explanation of Signs

[0056] 10 Master System 12 Handbag-shaped movable part 14 Master control device 12N1 Thumb part 12N2 Index finger part 12N3 Middle finger part 12N4 Ring finger part 12N5 Little finger part 12N6 Palm part 12D Sheet 12D11 Nanoscale sheet 12D12 Gel 55 Slave system 51 Robot arm 50 Slave control device 52 Operating part 52N1 Thumb operating part 52N2 Index finger operating part 52N3 Middle finger operating part 52N4 Ring finger operating part 52N5 Little finger operating part 52N6 Palm operating part 30 Computer 20 Driver 17 Camera 45 Communication device 50 Slave control device 60 Computer 53 Motor ダ54 Encoder (It seems there is a typo here, might be "54 Encoder") 56 Torque sensor 58 Communication device

Claims

1. A force feedback device for remote operation, comprising a plurality of sheets each deforming in a specific direction, the plurality of sheets being arranged in a state where the respective specific directions are combined in various directions, and a control unit that controls, among the plurality of sheets corresponding to a portion of an operation unit that operates an operation target, the sheet corresponding to the specific direction to deform in the direction of the force feedback received by the operation target from the operation target. A force feedback device comprising the above.

2. The sheet is a film-shaped vibrating part that vibrates when operating, and a film-shaped conversion part that converts and deforms the vibration only in the specific direction. The force feedback device according to claim 1, comprising the above.

3. The sheet operates when a voltage is applied, and the amount of deformation changes according to the magnitude of the applied voltage. The magnitude of the force feedback received by the operation target from the operation target is detected. The control unit controls the magnitude of the voltage applied to the deforming sheet according to the detected magnitude of the force feedback. The force feedback device according to claim 1 or claim 2.

4. The plurality of sheets are arranged in a grid. The force feedback device according to claim 1 or claim 2.

Images