Control device, control method, haptic feedback system, and computer program

Abstract

To provide a control device, a control method, a haptic feedback system, and a computer program, for reducing processing load in a haptic feedback device.SOLUTION: A control device is for a haptic feedback device having a displacement unit that is provided to be displaced in response to an operation by an operator. The haptic feedback device is configured to generate a force sense for operation on the displacement unit according to a displacement amount of the displacement unit and present a haptic sensation of a displayed object. A start displacement amount for starting presentation of the force sense in the displacement unit and control data for generating the force sense associated with a relative displacement amount from the start displacement amount are stored in advance for each object. On the basis of a start displacement amount corresponding to a selected object and control data for each relative displacement amount, the stored control data is converted into control data associated with an absolute displacement amount of the displacement unit, to output the converted control data to the haptic feedback device.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a control device, a control method, a tactile presentation system, and a computer program for improving the perception degree of the tactile sensation presented by a tactile presentation device.

Background Art

[0002] Among the five senses of humans, the transmission technologies of images and sounds related to vision and hearing have been highly refined, and recently, various technologies for presenting touch (haptics) have been proposed.

[0003] As a technology for presenting touch, reproduction by vibration when touching an image of an object displayed on a display incorporated with a touch panel, mainly three types of vibrations (ERM: Eccentric Rotating Mass, LRA: Linear resonant Actuator, piezo element) has been proposed. However, in tactile presentation on a touch panel, it is difficult to reproduce a deeper touch, that is, not only the force sensation generated in the skin of the finger but also the force sensation generated in the muscles and tendons.

[0004] Based on the findings on magnetorheological fluids, the inventors have proposed a tactile presentation device that can be operated by an operator and reproduces different tactile sensations depending on the object (Patent Document 1, etc.).

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The tactile presentation device preferably has a compact configuration that can be worn on the operator's hand, and the configuration is preferably simple. Also, since it is a tactile sensation, it is desirable that the control response speed is high.

[0007] This invention has been made in view of the above circumstances, and aims to provide a control device, a control method, a haptic presentation system, and a computer program that reduce the processing load in a haptic presentation device. [Means for solving the problem]

[0008] A control device according to one embodiment of the present disclosure is a control device for a haptic presentation device having a displacement part that is provided to be displaceable in response to the operator's operation, and which generates force sensation in response to the operation of the displacement part according to the amount of displacement of the displacement part, and presents the tactile sensation of a displayed object, wherein the control device stores for each object a starting displacement amount to start presenting the force sensation at the displacement part and control data for generating the force sensation corresponding to the relative displacement amount from the starting displacement amount, and based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount, the stored control data is converted into control data corresponding to the absolute displacement amount of the displacement part, and the converted control data is output to the haptic presentation device.

[0009] A control method in one embodiment of the present disclosure is a control method for a haptic presentation device having a displacement part that is provided to be displaceable in response to the operator's operation, and which generates force sensation in response to the operation of the displacement part according to the amount of displacement of the displacement part, and presents the tactile sensation of a displayed object, wherein a computer connected to the haptic presentation device stores for each object a starting displacement amount at which the force sensation is to be presented at the displacement part, and control data for generating the force sensation corresponding to the relative displacement amount from the starting displacement amount, and based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount, the stored control data is converted into control data corresponding to the absolute displacement amount of the displacement part, and the converted control data is output to the haptic presentation device.

[0010] A computer program in one embodiment of the present disclosure has a displacement unit that is provided to be displaceable in response to operator operation, and in accordance with the amount of displacement of the displacement unit, generates force sensation in response to operation on the displacement unit and presents the tactile sensation of a displayed object. The computer program stores for each object a starting displacement amount for starting the presentation of the force sensation at the displacement unit and control data for generating the force sensation corresponding to the relative displacement amount from the starting displacement amount. Based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount, the stored control data is converted into control data corresponding to the absolute displacement amount of the displacement unit, and the converted control data is output to the tactile presentation device.

[0011] In the control device, control method, and computer program of this disclosure, when the displacement unit is displaced to a predetermined amount, the system starts to present force feedback, and this predetermined displacement amount is stored as the starting displacement amount. Furthermore, the control device of this disclosure stores control data for presenting the feel of an object, corresponding to the relative displacement amount from the starting displacement amount. This reduces the amount of data that needs to be stored.

[0012] In a control device according to one embodiment of the present disclosure, the control data does not include data on the amount of displacement corresponding to the area outside the tactile presentation range of the object.

[0013] In the control device of this disclosure, the control data does not include data for displacements outside the range where force sensation is not generated, i.e., the range in which the object's touch is emitted, even in terms of the relative displacement from the starting displacement, nor does it include the corresponding control data. This makes the data more compact.

[0014] A control device according to one embodiment of the present disclosure accepts adjustments to control data stored in association with the relative displacement amount and at least one of the starting displacement amount.

[0015] A tactile presentation system according to one embodiment of the present disclosure includes a tactile presentation device having a displacement part that is provided to be displaceable in response to the operator's operation, and which generates force sensation in response to the operation of the displacement part according to the amount of displacement of the displacement part, and presents the tactile sensation of a displayed object; and an information processing device that communicates with the tactile presentation device and has a display unit, wherein the information processing device stores for each object a starting displacement amount to start presenting the force sensation at the displacement part, and control data for generating the force sensation corresponding to the relative displacement amount from the starting displacement amount, and based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount, converts the stored control data into control data corresponding to the absolute displacement amount of the displacement part, outputs the converted control data to the tactile presentation device, and outputs an image to the display unit according to the amount of displacement of the displacement part based on the visual data.

[0016] In the tactile presentation system of this disclosure, when a displacement part is displaced to a predetermined amount, the system starts to generate a tactile sensation, and this predetermined displacement amount is stored as the starting displacement amount. In contrast, the image output in synchronization with the tactile data should ideally be displayed even for displacement amounts less than the starting displacement amount. Furthermore, when outputting auditory information using a speaker, it is preferable that data from the same timing as the starting displacement amount is stored and used. [Effects of the Invention]

[0017] According to this disclosure, the data used to control haptic presentation in a haptic presentation device can be made more compact, thereby reducing the processing load. [Brief explanation of the drawing]

[0018] [Figure 1] This is a schematic diagram illustrating a haptic feedback system. [Figure 2] This is a block diagram showing the configuration of an information processing device. [Figure 3] This is an explanatory diagram showing an example of the contents of a sensory database. [Figure 4] This is a block diagram showing the configuration of a tactile presentation device. [Figure 5]It is a flowchart showing an example of the basic processing procedure of tactile presentation in a tactile presentation system. [Figure 6] It is a flowchart showing an example of the basic processing procedure of tactile presentation in a tactile presentation system. [Figure 7] It is a schematic diagram showing an example of tactile presentation in a tactile presentation system. [Figure 8] It is an explanatory diagram of a method for adjusting tactile data or the starting displacement amount. [Figure 9] It is a flowchart showing an example of the basic processing procedure of tactile presentation in the tactile presentation system of the second embodiment. [Figure 10] It is a flowchart showing an example of the basic processing procedure of tactile presentation in the tactile presentation system of the second embodiment. [Figure 11] It is a schematic diagram of the tactile presentation system in the third embodiment. [Figure 12] It is a block diagram showing the configuration of the HMD of the third embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0019] The present disclosure will be specifically described with reference to the drawings showing its embodiments. In the following embodiments, the implementation of the setting method in the tactile presentation system will be described.

[0020] (First Embodiment) FIG. 1 is a schematic diagram showing a tactile presentation system 100. The tactile presentation system 100 includes an information processing device 1 and a tactile presentation device 2. The information processing device 1 and the tactile presentation device 2 are communicatively connected by short-range wireless communication and exchange data with each other.

[0021] As shown in FIG. 1, the information processing device 1 uses a smartphone. The information processing device 1 may be a tablet terminal instead of a smartphone, or may be a laptop-type PC (Personal Computer).

[0022] The tactile presentation device 2 is a device that can be operated by an operator holding the displacement part 202 with their fingers and moving their fingers. The tactile presentation device 2 reads the position of the displacement part 202 as the operator moves their fingers, and controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to that position to generate force sensation through the reaction force (rotational resistance) to the operator's operation on the displacement part 202, thereby presenting tactile sensation. The form of the displacement part 202 of the tactile presentation device 2 is not limited to that shown in Figure 1, and may be stick-shaped or cushion-shaped covered with a cover. The tactile presentation device 2 may also employ a motor or piezoelectric element instead of the MRF device 24 to generate force sensation through rotational force or vibration in response to the operator's operation, and may be combined with other elements that present vibration, warmth, or coldness in addition to the displacement part 202. It may also be a structure that is installed on the ground or wall and operated by the operator's palm or foot.

[0023] In the haptic presentation system 100, the haptic presentation device 2 works in conjunction with the information processing device 1 to output images and sounds of the object using the information processing device 1 or other display devices or speakers, while outputting the tactile sensation of the object through the haptic presentation device 2. In the example shown in Figure 1, a soft object is displayed on the display unit 13 of the information processing device 1, and when the operator presses the displacement unit 202 of the haptic presentation device 2 with their finger, the haptic presentation device 2 outputs a squishy or fluffy tactile sensation, displays an image on the display unit 13 that shows the object changing as if it has been pressed, and outputs a sound that sounds like "squishy" corresponding to the object being pressed, through the sound output unit 14.

[0024] Figure 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 comprises a processing unit 10, a storage unit 11, a communication unit 12, a display unit 13, an audio output unit 14, and an operation unit 15. The processing unit 10 is a processor using a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit). The processing unit 10 executes the processing described later based on the control program P1 for haptic presentation stored in the storage unit 11.

[0025] The storage unit 11 uses non-volatile memory such as flash memory or an SSD (Solid State Drive). The storage unit 11 stores data that the processing unit 10 references. The control program P1 (program product) is downloaded from the information processing device 1 or another program server device via the communication unit 12 and stored in an executable format. The control program P1 stored in the storage unit 11 may be a computer program P8 that was stored on a storage medium 8 readable by a computer and then read and stored by the processing unit 10.

[0026] The memory unit 11 stores a sensory database (DB:Data Base) 110 containing tactile data, visual data, and auditory data of the object to be output. The sensory DB 110 stores, in association with an object ID that identifies the object, the tactile data and auditory data (sound) to be output for each relative displacement of the displacement unit 202 in the tactile presentation device 2, as well as visual data (image) for each displacement of the displacement unit 202 (see Figure 3). The sensory DB 1100 stores, in association with the object ID, the displacement of the displacement unit 202 corresponding to the starting position of tactile or auditory presentation (hereinafter referred to as the starting displacement).

[0027] The communication unit 12 is a short-range wireless communication module, such as Bluetooth®. The processing unit 10 can send and receive data to and from the haptic presentation device 2 via the communication unit 12.

[0028] The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 13 is, for example, a touch panel-integrated display. The processing unit 10 displays an operation screen or an image of an object on the display unit 13 based on the control program P1, for outputting tactile sensations with the tactile presentation device 2.

[0029] The audio output unit 14 includes a speaker, etc. The processing unit 10 outputs the sound of the target object, music, etc., from the audio output unit 14 based on the control program P1.

[0030] The operation unit 15 is a user interface that enables input and output to and from the processing unit 10, and is a touch panel built into the display unit 13. The operation unit 15 may also consist of physical buttons. The operation unit 15 may also consist of an audio input unit.

[0031] Figure 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. As shown in Figure 3, the sensory DB 110 stores the current value of the current supplied to the MRF device of the tactile presentation device 2 for each relative displacement amount (angle) of the displacement unit 202 as tactile data, associated with the object ID. It is not limited to current values; it may also be voltage values, etc. The sensory DB 110 also stores the starting displacement amount, associated with the object ID. Here, the relative displacement amount corresponds to the difference from the starting displacement amount. There can be multiple objects, and they may include real objects that present tactile sensations, such as inanimate objects like balls or balloons, food such as gummies, vegetables or fruits, animals such as dogs, cats or fish, or characters such as slime or monsters.

[0032] The relationship between the relative displacement of the displacement unit 202 of the sensory DB110 and the current value is set according to the actual dimensions of the movable range of the displacement unit 202 and the dimensions of the object. For example, if the movable range of the displacement unit 202 is 50 mm and the object is a small, non-bursting sphere like a gummy candy with a size set to 15 mm, the relative displacement in the tactile data will correspond to a range of approximately 15 mm, as shown in Figure 3. Also, if the object is a large object such as the seat of a sofa and its size greatly exceeds the movable range of the displacement unit 202, the current value in the tactile data will be set to present a sensation up to 50 mm from the surface of the sofa seat. If the object is a virtual character such as a monster, the current value will be set based on the size according to the character's settings.

[0033] The sensory DB110 similarly stores audio data for each relative displacement (angle) of the displacement unit 202, associated with the object ID. The audio data is audio data that reproduces the sound emitted when the object is touched. The audio data may be different waveform data for each angle. It may also be a timestamp of the audio corresponding to each angle.

[0034] The sensory DB110 stores images (frame images) for each displacement amount (angle) of the displacement unit 202 as visual data, associated with the object ID. Unlike tactile and auditory data, visual data is perceived visually before contact occurs, so image data is associated with the entire range of motion of the displacement unit 202 (e.g., 0° to 90°). Here, a frame image is a single still image that is recognized as an animated image when displayed in sequence.

[0035] Figure 4 is a block diagram showing the configuration of the tactile presentation device 2. As shown in Figure 1, the tactile presentation device 2 is configured by providing a flat, bottomed cylindrical gripping body 200 with a strip-shaped plate displacement part 202 having a curved portion that partially follows the circumferential direction. The displacement part 202 is made of a material that can bend, but it may also be made of a highly rigid material and be rotatably supported on the gripping body 200 via a pivot shaft. A cloth tape-like fastener 203 is provided on the outer surface of the tip of the displacement part 202. A link mechanism 204 is provided on the inner surface of the tip of the displacement part 202, which connects to the rotation axis of the rotor of the MRF device 24 housed inside the gripping body 200.

[0036] As shown in Figure 1, the operator grips the gripping body 200 with, for example, their thumb and middle finger, and uses the device by aligning their index finger or other fingers along the displacement part 202 and inserting their index finger into the fastening device 203. The operator can move the displacement part 202 by pushing their index finger in, and can also extend their index finger to move the displacement part 202 away from the gripping body 200.

[0037] The tactile presentation device 2 comprises a gripping body 200 as shown in Figure 1, a control unit 20, a storage unit 21, a communication unit 22, a power supply unit 23, an MRF device 24, and a sensor 25. The gripping body 200 incorporates the MRF device 24. The control unit 20, storage unit 21, communication unit 22, and power supply unit 23 may be provided integrally with the gripping body 200, or they may be provided in separate units connected to the gripping body 200 wirelessly or wired.

[0038] The control unit 20 includes a processor such as a CPU or MPU (Micro-Processing Unit), and memory such as ROM (Read Only Memory) or RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the built-in ROM to realize haptic feedback.

[0039] The memory unit 21 is an auxiliary memory for the control unit 20 and stores the control data (tactile data) of the MRF device 24 in a rewritable manner.

[0040] The communication unit 22 is a communication module for short-range wireless communication, such as Bluetooth®. The control unit 20 can send and receive data with the information processing device 1 via the communication unit 22.

[0041] The control unit 20 is connected to the power supply unit 23, the MRF device 24, and the sensor 25 via I / O, and they exchange signals with each other.

[0042] The power supply unit 23 includes a rechargeable battery. When the power supply unit 23 is turned ON, it supplies power to each component and the MRF device 24.

[0043] The MRF device 24 has a yoke that is positioned to sandwich a disc-shaped rotor with a gap between them. A control current is passed through a coil provided in the yoke to generate a magnetic field, and the viscosity (shear stress) of the magnetorheological fluid sealed in the gap is controlled to provide rotational resistance to the rotor. When the control unit 20 controls the magnitude of the control current to the MRF device 24, the rotational resistance is immediately changed.

[0044] Sensor 25 measures the displacement (angle) of the displacement unit 202 and outputs it to the control unit 20. Sensor 25 measures the displacement of the displacement unit 202 as an angle and outputs it. Sensor 25 may be composed of multiple sensors such as a gyro sensor and an acceleration sensor.

[0045] In the haptic presentation device 2 configured as described above, when the displacement unit 202 is operated by the operator, the displacement of the displacement unit 202 is transmitted via the link mechanism 204 to the rotation axis of the rotor of the MRF device 24 in the rotational direction. The rotation axis rotates freely when the MRF device 24 is not operating, i.e., when the control current is zero, so the displacement unit 202 fluctuates without resistance. On the other hand, when the MRF device 24 is operating and the control current is not zero, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 is changed according to the magnitude of the current flowing to the MRF device 24. The control unit 20 can change the force of resistance to the displacement unit 202 and how it manifests by continuously changing the magnitude of the current to the MRF device 24 or by oscillating the current value at a predetermined frequency.

[0046] In this way, the tactile presentation device 2 can present a smooth tactile sensation by varying the resistance (current value) according to the amount of pressure (displacement) of the displacement part 202, present a firm tactile sensation by increasing the resistance as the amount of pressure increases, or present a rough tactile sensation by repeatedly varying the magnitude of the resistance.

[0047] Figures 5 and 6 are flowcharts illustrating an example of the basic processing procedure for haptic presentation in the haptic presentation system 100. When the operator starts the control program P1 and turns on the power of the haptic presentation device 2, the processing unit 10 of the information processing device 1 starts the following processing in cooperation with the haptic presentation device 2.

[0048] The processing unit 10 displays an operation screen on the display unit that includes a list of candidate objects for displaying tactile sensations (step S101), and accepts the selection of an object (step S102). The processing unit 10 reads tactile data, auditory data (sound), starting displacement amount, and visual data (image) corresponding to the object ID of the selected object from the sensory DB 110 of the storage unit 11 (step S103).

[0049] The processing unit 10 converts the read tactile data into tactile data where the starting displacement amount is added to the relative displacement amount to obtain an absolute displacement amount, and stores it temporarily (step S104). The processing unit 10 converts the read auditory data into auditory data where the starting displacement amount and the starting displacement amount are added to the relative displacement amount to obtain an absolute displacement amount, and stores it temporarily (step S105).

[0050] The processing unit 10 temporarily stores the read visual data (step S106).

[0051] The processing unit 10 establishes communication between the haptic presentation device 2 and the communication unit 12 (step S107), and transmits the temporarily stored converted haptic data to the haptic presentation device 2 (step S108). The processing unit 10 displays on the display unit 13 that haptic presentation has started (step S109).

[0052] When the control unit 20 of the tactile presentation device 2 receives tactile data (step S201), it stores it in the storage unit 21 (step S202).

[0053] The control unit 20 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the sensor 25 (step S203). The control unit 20 transmits the displacement amount obtained by sampling to the information processing device 1 (step S204), references the current value corresponding to the obtained displacement amount from the tactile data stored in the storage unit 21 in step S202 (step S205), outputs the referenced current to the MRF device 24 (step S206), and returns the process to step S203. In step S205, if the displacement amount corresponding to the obtained displacement amount cannot be referenced (not included) in the tactile data, the control unit 20 skips the process in step S206. The process in steps S203-S206 continues until an operation to terminate is performed on the information processing device 1 side.

[0054] The information processing device 1 receives the displacement amount from the haptic presentation device 2 (step S110), and the processing unit 10 references the image and sound corresponding to the received displacement amount from the temporarily stored visual data and converted auditory data (step S111). The processing unit 10 outputs the referenced visual data and auditory data from the display unit 13 and the sound output unit 14, respectively (step S112), and returns the process to step S110. Each time the displacement amount is transmitted from the haptic presentation device 2, the information processing device 1 outputs the corresponding image and sound.

[0055] The processing unit 10 determines whether or not a termination operation has been performed (step S113). If it is determined that a termination operation has not been performed (S113: NO), the process returns to step S110, and the process from steps S110 to S113 is repeated.

[0056] If it is determined that the termination operation has been performed (S113: YES), the processing unit 10 terminates the display and disconnects communication with the haptic presentation device 2 (step S114), and terminates the process.

[0057] Figure 7 is a schematic diagram showing an example of tactile presentation in the tactile presentation system 100. Figure 7 shows the changes in the image and sound displayed on the display unit 13 in accordance with the amount of displacement in the tactile data transmitted to the tactile presentation device 2. As shown in Figure 7, the image changes according to the amount of indentation (displacement) of the displacement unit 202, and the way indentation corresponds to the tactile sensation changes. Specifically, from a starting displacement of 10°, the tactile presentation device 2 presents tactile sensation and the image changes at the displacement at which the finger begins to make contact. As shown in Figure 7, when displaying an image of the operator's finger in the image, the image of the finger may be displayed as if touching the object from the starting displacement at which the finger makes contact, i.e., when a non-zero current value is output to the MRF device 24.

[0058] In the tactile presentation system 100 that presents tactile sensations in this manner, tactile data is stored in the sensory DB 110 as data representing the relative displacement from the starting displacement. This is expected to reduce the size of the tactile data and decrease the processing load.

[0059] (Second Embodiment) By storing the initial displacement amount and tactile data as separate data, it becomes easy to adjust only the initial displacement amount for each operator, or to adjust only the tactile data to expand or contract according to the scale.

[0060] The configuration of the tactile presentation system 100 in the second embodiment is the same as that of the tactile presentation system 100 in the first embodiment, except for the adjustment method. Therefore, among the components of the tactile presentation system 100 in the second embodiment, those that are common with the tactile presentation system 100 in the first embodiment are denoted by the same reference numerals and their detailed descriptions are omitted.

[0061] Figure 8 is an explanatory diagram of the method for adjusting tactile data or starting displacement. As shown in Figure 8, when using tactile data in which current values ​​to the tactile presentation device 2 are associated with relative displacement amounts from 0° to 48°, the processing unit 10 can adjust the scale by half to tactile data for relative displacement amounts from 0° to 24°. Furthermore, if the sensory DB 110 stores the starting displacement amount as 5° for the same object, the processing unit 10 can adjust by adding 5 to the starting displacement amount. In this case, the converted tactile data transmitted to the tactile presentation device 2 is data in which non-zero current values ​​are associated with displacement amounts in the displacement unit 202 in the range of 10° to 34°, as shown in Figure 8. Figure 8 shows an example in which both the starting displacement amount and tactile data are adjusted, but it is not limited to this, and only one of the starting displacement amount or tactile data may be adjusted. Note that the scaling and adjustment of the starting displacement amount can also be applied to auditory data and visual data.

[0062] Figures 9 and 10 are flowcharts illustrating an example of the basic processing procedure for haptic presentation in the haptic presentation system 100 of the second embodiment. For the processing procedures shown in the flowcharts of Figures 9 and 10 that are common to the processing procedures shown in the flowcharts of Figures 5 and 6 of the first embodiment, the same step numbers are used, and detailed explanations are omitted.

[0063] If a scale is set in the second embodiment, the processing unit 10 adjusts the scale of either the tactile data, auditory data, visual data, or the starting displacement amount from the tactile data, auditory data, starting displacement amount, and visual data read in step S103 (step S121). Step S121 corresponds to the adjustment in the upper right of Figure 8.

[0064] Based on the initial displacement amount after scale adjustment and the tactile data, the tactile data is converted to an absolute displacement amount obtained by adding the relative displacement amount to the initial displacement amount, and temporarily stored (step S122).

[0065] Similarly, the processing unit 10 converts the starting displacement amount after scale adjustment and auditory data into auditory data representing the starting displacement amount and the absolute displacement amount obtained by adding the relative displacement amount to the starting displacement amount, and stores it temporarily (step S123).

[0066] The processing unit 10 temporarily stores the visual data after scaling (step S124). Thereafter, the processing unit 10 executes the processes in steps S107-S114.

[0067] In this way, by storing the initial displacement amount and the control data for the relative displacement amount separately in the sensory DB110, the process of adjusting and correcting the scale becomes easier.

[0068] (Third embodiment) In the third embodiment, a Head-Mounted Display (HMD) is used to present visual information about an object as a three-dimensional image. Figure 11 is a schematic diagram of the haptic presentation system 100 in the third embodiment, and Figure 12 is a block diagram showing the configuration of the HMD3. The configuration of the haptic presentation system 100 in the third embodiment is the same as that of the haptic presentation system 100 in the first embodiment, except that it uses the HMD3 and the details of the processing performed by using the HMD3 are different. Therefore, for the configuration of the haptic presentation system 100 in the third embodiment that is common with the haptic presentation system 100 in the first embodiment, the same reference numerals are used and detailed explanations are omitted.

[0069] The HMD3 comprises a display unit 31, a motion detection unit 32, a spatial detection unit 33, and a connection unit 34. The HMD3 may have the display unit 31, motion detection unit 32, spatial detection unit 33, and connection unit 34 on the main body, or some of them may be provided in separate units so that control signals can be exchanged between them via a communication medium. The HMD3 may also be configured as an integrated unit with the functions of the information processing device 1.

[0070] The display unit 31, for example, comprises a small liquid crystal display, an optical lens, and an optical system mechanism, and is capable of displaying three-dimensional images with a viewing angle of 110° or more. The display unit 31 is a transparent or translucent glass-like display that receives and superimposes image signals (including video signals) output from the information processing device 1 onto the operator's actual field of view. The display unit 31 is not limited to a glass-like form; it may also superimpose an image based on image signals output from the information processing device 1 onto an image of real space captured by a camera positioned facing forward.

[0071] The motion detection unit 32 includes multiple three-axis acceleration sensors and gyro sensors provided in various orientations at different locations on the main body (the cover of the display unit 31 and the mounting belt), and a control circuit that aggregates and outputs signals from these sensors. The motion detection unit 32 detects the movement of the wearer's head.

[0072] The spatial detection unit 33 uses two or more infrared cameras arranged side-by-side facing outwards on the outer surface of the main body, and an infrared LED positioned between the infrared cameras to similarly emit infrared light outwards. The spatial detection unit 33 functions as a depth sensor to measure the distance to objects located outside the main body of the HMD3. The spatial detection unit 33 can measure and output the distance from the HMD3 main body to stationary objects such as walls and floors at the location of the operator wearing the HMD3. Similarly, the spatial detection unit 33 can measure and output the distance from the HMD3 main body to the operator's arms, hands, and fingers.

[0073] The connection unit 34 is an interface for connecting to the information processing device 1. The HMD3 outputs signals corresponding to the results measured by the motion detection unit 32 and the spatial detection unit 33 to the information processing device 1, and also acquires the video signal output from the information processing device 1 and displays it on the display unit 31.

[0074] Alternatively, a speaker may be provided in the HMD3, and the sound output from the audio output unit 14 of the information processing device 1 may be output through the speaker.

[0075] In the second embodiment of the haptic presentation system 100 with the configuration described above, the information processing device 1 uses the HMD 3 and the distance to a stationary object in real space measured by the spatial detection unit 33 to superimpose a three-dimensional image of a virtual object in accordance with the operator's movements detected by the motion detection unit 32, thereby realizing AR display. The selection of an object is determined when the spatial detection unit 33 detects that the fingers holding the haptic presentation device 2 have been superimposed on the coordinates (coordinates that match those in real space) within the image of the three-dimensional image of the virtual object displayed on the display unit 31 of the HMD3, and the processing unit 10 of the information processing device 1 detects that the object has been selected.

[0076] In the third embodiment, the information processing device 1 exchanges displacement and tilt data with the haptic presentation device 2 to control the AR-displayed image and the tactile feedback output by the haptic presentation device 2. The processing content may be any of the processing shown in the first and second embodiments.

[0077] By combining AR display using HMD3 with haptic presentation device 2, the operator can visually observe an image of an object virtually placed in real space, while simultaneously learning the feel of that object through the haptic presentation device 2. In this second embodiment, as well, the haptic data can be made compact, and the processing load can be reduced, similar to that described in the first embodiment. This allows for the expectation of receiving an immediate tactile response.

[0078] The embodiments disclosed above are illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are included. [Explanation of Symbols]

[0079] 1. Information Processing Device 10 Processing Unit 11 Storage section 13 Display section 130 Settings screen P1 Control Program 2. Tactile presentation device 20 Control Unit 21 Memory section 202 Displacement section 24 MRF devices

Claims

1. A control device for a tactile presentation device having a displacement part that is provided to be displaceable in response to operator operation, and which generates resistance to operation of the displacement part by an MRF (Magneto-Rheological Fluid) device according to the amount of displacement of the displacement part, thereby presenting force sensations corresponding to the displayed object, The starting displacement amount at which the resistance is initiated in the displacement unit, and the control data for generating the resistance corresponding to the relative displacement amount from the starting displacement amount, are stored separately for each object. When an object is selected, the stored control data is converted into control data corresponding to the absolute displacement of the displacement part, based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. The converted control data is output to the haptic presentation device. Control device.

2. The control data does not include data on displacement amounts corresponding to the range of force feedback of the object. The control device according to claim 1.

3. The system accepts adjustments to at least one of the control data stored in association with the relative displacement amount and the starting displacement amount. The control device according to claim 2.

4. Perform a scale adjustment on the control data for each relative displacement amount, Based on the adjusted control data and the initial displacement amount, the data is converted into control data corresponding to the absolute displacement amount of the displacement part. The control device according to any one of claims 1 to 3.

5. A control method for a tactile presentation device having a displacement part that is displaceable in response to operator operation, and which generates resistance to operation of the displacement part by an MRF device according to the amount of displacement of the displacement part, thereby presenting force sensation corresponding to the displayed object, The computer connected to the haptic presentation device, The starting displacement amount at which the resistance is initiated in the displacement unit, and the control data for generating the resistance corresponding to the relative displacement amount from the starting displacement amount, are stored separately for each object. When an object is selected, the stored control data is converted into control data corresponding to the absolute displacement of the displacement part, based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. The converted control data is output to the haptic presentation device. A control method including a processing unit.

6. A tactile presentation device having a displacement part that is displaceable in response to the operator's operation, and which generates resistance to operation on the displacement part by an MRF device according to the amount of displacement of the displacement part, and presents a force sensation corresponding to the displayed object, The aforementioned haptic presentation device is connected to an information processing device that includes a display unit. Includes, The aforementioned information processing device is The starting displacement amount at which the resistance is initiated in the displacement unit, and the control data for generating the resistance corresponding to the relative displacement amount from the starting displacement amount, are stored separately for each object. When an object is selected, the stored control data is converted into control data corresponding to the absolute displacement of the displacement part, based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. The converted control data is output to the tactile presentation device. Based on the visual data, an image is output to the display unit according to the amount of displacement of the displacement unit. A haptic feedback system.

7. A computer connected to a haptic presentation device has a displacement part that is displaceable in response to the operator's operation, and generates resistance to operation on the displacement part by an MRF device according to the amount of displacement of the displacement part, and presents force sensation corresponding to the displayed object. The starting displacement amount at which the resistance is initiated in the displacement unit, and the control data for generating the resistance corresponding to the relative displacement amount from the starting displacement amount, are stored separately for each object. When an object is selected, the stored control data is converted into control data corresponding to the absolute displacement of the displacement part, based on the starting displacement amount corresponding to the selected object and the control data for each relative displacement amount. The converted control data is output to the haptic presentation device. A computer program that executes a process.

Images