Virtual object tactile presentation device, virtual object tactile presentation method, and control program

Abstract

To provide a virtual object tactile presentation device that comprises a visual presentation device for presenting a virtual object in a visual and auditory manner and a tactile presentation device, and that can start tactile presentation without any delay behind displaying of the virtual object.SOLUTION: A visual presentation device of a virtual object tactile presentation device comprises: a storage unit which stores visual data for presenting a virtual object in a visual manner and tactile data for presenting the virtual object in a tactile manner; a display unit which displays the virtual object; a communication unit which transmits the tactile data stored in the storage unit to the tactile presentation device; an operation unit which receives a start of presentation of the virtual object; and a processing unit. The processing unit displays an image associated with presentation preparation for the virtual object when the operation unit receives the start of presentation of the virtual object, and causes the display unit to display the virtual object when the communication unit transmits the tactile data to the tactile presentation device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a virtual object tactile presentation device, a virtual object tactile presentation method, and a control program. [Background technology]

[0002] Among the five human senses, technologies for transmitting images and sounds related to vision and hearing have become increasingly accurate, and recently, various haptic technologies that present tactile sensations have been proposed.

[0003] Patent Document 1 discloses a virtual object tactile presentation device. The virtual object tactile presentation device includes an HMD (Head Mounted Display) that displays virtual objects, and a tactile presentation device that presents to the user tactile sensations corresponding to the virtual objects displayed on the HMD. The HMD includes a tactile information DB that stores tactile data for each virtual object, and reads tactile data associated with the type of virtual object from the tactile information DB and transmits it to the tactile presentation device. The tactile presentation device presents the feel of the virtual object by controlling the magnetic field applied to a magnetorheological fluid. A magnetorheological fluid is a fluid in which magnetic particles are dispersed in a dispersion medium, and has the property that its viscosity changes when a magnetic field is applied. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-17159 Summary of the Invention [Problem to be solved by the invention]

[0005] However, since the virtual object tactile presentation device of Patent Document 1 is configured to transmit tactile data from the HMD to the tactile presentation device, there is a risk that a delay will occur between the time the virtual object is displayed and the time the tactile presentation of the virtual object begins, which could cause the user to feel uncomfortable.

[0006] An object of the present disclosure is to provide a virtual object tactile presentation device, a virtual object tactile presentation method, and a control program, which are equipped with a visual presentation device that visually presents a virtual object and a tactile presentation device that tactilely presents the virtual object, and which are capable of starting tactile presentation without delay from the display of the virtual object. [Means for solving the problem]

[0007] A virtual object tactile presentation device according to one aspect of the present disclosure is a virtual object tactile presentation device including a visual presentation device that visually presents a virtual object and a tactile presentation device that tactilely presents the virtual object, wherein the visual presentation device includes a memory unit that stores visual data for visually presenting the virtual object and tactile data for tactilely presenting the virtual object, a display unit that displays the virtual object based on the visual data stored in the memory unit, a communication unit that transmits the tactile data stored in the memory unit to the tactile presentation device, an operation unit that accepts a start of presentation of the virtual object, and a processing unit that, when the operation unit accepts a start of presentation of the virtual object, causes the display unit to display an image related to preparation for presentation of the virtual object, and, when the communication unit transmits the tactile data to the tactile presentation device, causes the display unit to display the virtual object.

[0008] A virtual object tactile presentation method according to one aspect of the present disclosure includes a memory unit that stores visual data for visually presenting a virtual object and tactile data for tactilely presenting the virtual object, and is a virtual object tactile presentation method using a visual presentation device that visually presents the virtual object and a tactile presentation device that tactilely presents the virtual object, wherein the method receives a request to start presenting the virtual object, and when the method receives a request to start presenting the virtual object, displays an image related to preparation for presentation of the virtual object on the visual presentation device, transmits the tactile data from the visual presentation device to the tactile presentation device, and after transmitting the tactile data to the tactile presentation device, displays the virtual object on the visual presentation device.

[0009] A control program according to one aspect of the present disclosure includes a memory unit and a processing unit that store visual data for visually presenting a virtual object and tactile data for tactilely presenting the virtual object, and is a control program for controlling the operation of a visual presentation device that visually presents the virtual object, wherein the control program receives a request to start presenting the virtual object, and when the request to start presenting the virtual object is received, displays an image related to preparation for presentation of the virtual object, causes the processing unit to transmit the tactile data to a tactile presentation device that visually presents the virtual object, and after transmitting the tactile data to the tactile presentation device, causes the processing unit to execute a process to display the virtual object. [Effects of the Invention]

[0010] According to the present disclosure, in a virtual object tactile presentation device that includes a visual presentation device that visually presents a virtual object and a tactile presentation device that tactilely presents the virtual object, tactile presentation can begin without delay from the display of the virtual object. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a schematic diagram illustrating an example of the configuration of a virtual object tactile presentation device according to a first embodiment. [Figure 2] 1 is a block diagram showing an example of the configuration of a virtual object tactile presentation device according to a first embodiment. [Figure 3] FIG. 1 is a conceptual diagram showing the record layout of a sensory DB. [Figure 4] 1 is a flowchart showing a processing procedure for tactile presentation according to the first embodiment. [Figure 5] 1 is a conceptual diagram showing a tactile presentation method according to the first embodiment. [Figure 6] 10 is a flowchart showing a processing procedure for tactile presentation according to the second embodiment. [Figure 7] 10 is a flowchart showing the processing procedure for tactile presentation according to the third embodiment. [Figure 8] 10 is a flowchart showing the processing procedure for tactile presentation according to the fourth embodiment. [Figure 9] 13 is a flowchart showing the processing procedure for tactile presentation according to the fifth embodiment.

[0012] A virtual object tactile presentation device according to an embodiment of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

[0013] (Embodiment 1) <Overview of the virtual object tactile presentation device 100> 1 is a schematic diagram showing an example of the configuration of a virtual object tactile presentation device 100 according to embodiment 1. The virtual object tactile presentation device 100 includes an audiovisual presentation device 1 and a tactile presentation device 2. The audiovisual presentation device 1 and the tactile presentation device 2 are connected via short-range wireless communication or wired communication, and exchange data with each other.

[0014] The audiovisual presentation device 1 is a device that presents visual or auditory stimuli that a user feels in response to a virtual object. The audiovisual presentation device 1 is, for example, a smartphone, tablet terminal, laptop PC (Personal Computer), or the like, that has a display unit 13 and an audio output unit 14. These terminals are merely examples, and the configuration of the audiovisual presentation device 1 is not particularly limited as long as it is a device that has display functions, audio output functions, and communication functions. The display function is also not particularly limited, and is not particularly limited as long as it is a device that can visually present a virtual object, such as projection onto a screen, retinal projection onto the user, or virtual image projection. For example, an HMD (Head Mounted Display), a projector, or a 3D hologram display may be used.

[0015] The tactile presentation device 2 presents the tactile sensation felt by the user in relation to a virtual object. The tactile presentation device 2 includes a displacement unit 202 that is displaced by a user's operation, and a magnetorheological fluid (MRF) device 24 that presents the tactile sensation of the virtual object by applying displacement resistance to the displacement unit 202. A magnetorheological fluid (RMF) is a fluid in which magnetic particles are dispersed in a dispersion medium, and has the property of changing viscosity when a magnetic field is applied. The tactile presentation device 2 presents the feel of the virtual object by controlling the magnetic field applied to the magnetorheological fluid. Tactile presentation using a magnetorheological fluid is one example, and the configuration of the tactile presentation device 2 is not particularly limited as long as it is a device with a tactile presentation function. For example, the tactile presentation device may present tactile sensation using ultrasound, a piezoelectric element, or the like.

[0016] 1, an audiovisual presentation device 1 and a tactile presentation device 2 operate in cooperation with each other. The audiovisual presentation device 1 presents a virtual object using images and sounds, and the tactile presentation device 2 presents the tactile sensation of the virtual object.

[0017] <Details of the virtual object tactile presentation device 100> 2 is a block diagram showing an example of the configuration of a virtual object tactile presentation device 100 according to this embodiment. The audiovisual presentation device 1 is a computer, and includes a processing unit 10, a storage unit 11, a communication unit 12, a display unit 13, an audio output unit 14, and an operation unit 15. The audiovisual presentation device 1 may be configured to perform distributed processing using multiple computers, or may be partially implemented by multiple virtual machines provided in a single server, or may be implemented by a cloud server, or may be partially implemented by a quantum computer.

[0018] The processing unit 10 is a processor having an arithmetic circuit such as a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), an internal storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), an I / O terminal, a timer, etc. The processing unit 10 is connected to a storage unit 11, a communication unit 12, a display unit 13, an audio output unit 14, and an operation unit 15. The processing unit 10 executes a control program P stored in the storage unit 11 to perform processing for presenting a virtual object audiovisually and tactilely.

[0019] The storage unit 11 is a non-volatile memory such as a hard disk, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, or an SSD (Solid State Drive). The storage unit 11 stores a control program P for executing processing related to tactile presentation of a virtual object and a sensory DB (Data Base) 110. The sensory DB 110 will be described in detail later.

[0020] The control program P may be recorded in a computer-readable manner on a recording medium 3. The storage unit 11 stores the control program P read from the recording medium 3 by a reading device (not shown). The recording medium 3 may be a semiconductor memory such as a flash memory, an optical disk, a magnetic disk, a magneto-optical disk, or the like. Alternatively, various programs according to this embodiment may be downloaded from an external server (not shown) connected to the communication network N and stored in the storage unit 11.

[0021] The communication unit 12 is a communication module (near-field communication circuit) for short-distance wireless communication, for example, Bluetooth (registered trademark). The processing unit 10 transmits and receives data to and from the tactile presentation device 2 via the communication unit 12. Note that the communication unit 12 may be a circuit for wired communication.

[0022] The display unit 13 is a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. For example, the display unit 13 is a display with a built-in touch panel. The display unit 13 displays images that visually present virtual objects and various operation screens in accordance with the processing of the processing unit 10. Note that the display unit 13 may be a projection device that projects onto a screen, projects onto the user's retina, projects a virtual image, or the like.

[0023] The audio output unit 14 is a speaker, etc. Under the control of the processing unit 10, the audio output unit 14 outputs audio that audibly presents a virtual object.

[0024] The operation unit 15 is a user interface. For example, the operation unit 15 is a touch panel built into the display unit 13. The operation unit 15 may be an input device having hardware buttons and keys. The operation unit 15 may also be configured with a camera that accepts gesture input from the user or other input devices. The processing unit 10 accepts user operations via the operation unit 15 and provides data indicating the content of the accepted operation to the processing unit 10.

[0025] The tactile presentation device 2 includes a gripping body 200 that is gripped and operated by a user as shown in Fig. 1, a control unit 20, a tactile data storage unit 21, a communication unit 22, a power supply unit 23, an MRF device 24, and a displacement sensor 25 as shown in Fig. 2. Note that the circuits related to control and power supply of the control unit 20, power supply unit 23, etc. shown in Fig. 2 may be configured integrally with the gripping body 200, or may be configured separately and connected by wire or wirelessly.

[0026] The gripper 200 includes a hollow, disk-shaped casing 201, a displacement unit 202, a binder 203, and a link mechanism 204. The casing 201 houses a magnetorheological fluid and a rotor having a rotating shaft. The rotor is a disk-shaped member that rotates inside the casing 201, which is filled with magnetorheological fluid. The rotor is subjected to rotational resistance force due to the magnetorheological fluid. The gripper 200 is also provided with an electromagnet (not shown) for applying a magnetic field to the magnetorheological fluid. The electromagnet, together with the magnetorheological fluid and the rotor, constitutes the MRF device 24. Specifically, the electromagnet includes a yoke disposed to sandwich the disk-shaped rotor across a gap, and a coil attached to the yoke. When a current flows through the coil, a magnetic field is generated, which changes the viscosity (shear stress) of the magnetorheological fluid, thereby applying rotational resistance force to the rotating shaft.

[0027] The displacement part 202 is a band-shaped flat plate member curved to fit along a portion of the circumferential surface of the casing 201. The base end of the displacement part 202 is rotatably supported by a support shaft on the casing 201 so that the tip of the displacement part 202 can move in a direction toward and away from the casing 201. A fastener 203 for fastening a user's finger is provided on the outer surface of the tip of the displacement part 202. The fastener 203 is a ring-shaped band-shaped member. The tip of the displacement part 202 and the rotating shaft of the rotor are connected by a link mechanism 204. The link mechanism 204 is a mechanism that converts the movement of the displacement part 202, which moves toward or away from the casing 201, into rotational movement of the rotating shaft and transmits it.

[0028] The control unit 20 includes a processor such as a CPU or an MPU (Micro-Processing Unit), and memories such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The control unit 20 is, for example, a microcontroller.

[0029] The haptic data storage unit 21 is an auxiliary memory for the control unit 20, and stores a haptic data table, which will be described later. The haptic data storage unit 21 also stores rewritable control data for the MRF device 24. The haptic data storage unit 21 may be a non-volatile memory or a volatile memory.

[0030] The communication unit 22 is a communication module (near-field communication circuit) for short-distance wireless communication, for example, Bluetooth (registered trademark). The control unit 20 transmits and receives data to and from the audiovisual presentation device 1 via the communication unit 22. Note that the communication unit 22 may be a circuit for wired communication.

[0031] The power supply unit 23 includes a rechargeable battery. The power supply unit 23 supplies current to the electromagnets of the MRF device 24 under the control of the control unit 20. The power supply unit 23 also supplies the necessary power to each component.

[0032] The MRF device 24 is configured as described above, and a rotational resistance force is applied to the rotating shaft of the rotor by the current supplied from the power supply unit 23. When the magnitude of the current changes, the rotational resistance force changes.

[0033] The displacement sensor 25 measures the amount of displacement of the displacement unit 202 in the gripper 200 and outputs the measured amount to the control unit 20. Specifically, the displacement sensor 25 measures the rotation angle of the displacement unit 202 relative to the casing 201 and outputs data indicating the measured angle to the control unit 20. The control unit 20 acquires the data as finger position data. The finger position data is transmitted to the audiovisual presentation device 1 by the communication unit 22. The displacement sensor 25 may be composed of multiple sensors such as a gyro sensor and an acceleration sensor.

[0034] With the grip body 200 configured in this manner, a user can receive a tactile stimulus by gripping the grip body 200 and opening and closing their fingers. For example, as shown in FIG. 1 , the user can grip the grip body 200 by grasping the casing 201 with their thumb and middle finger and inserting their index finger into the binder 203 along the displacement portion 202. By closing their fingers, the user can move the displacement portion 202 in a direction approaching the casing 201. By opening their fingers, the user can move the displacement portion 202 in a direction away from the casing 201.

[0035] The control unit 20 measures the amount of displacement of the displacement unit 202 using the displacement sensor 25, and supplies a current according to the amount of displacement to the MRF device 24, thereby generating a reaction force (resistance force) against the user's operation on the displacement unit 202, and presenting a tactile sensation. Details are as follows. The displacement motion of the displacement unit 202 is transmitted as rotational motion to the rotating shaft of the rotor of the MRF device 24 via the link mechanism 204. When the MRF device 24 is not operating, i.e., when the control current is zero, the viscosity of the magnetorheological fluid in the rotating shaft and rotor is low, and the displacement unit 202 moves without resistance. When the MRF device 24 is operating and a control current is being supplied, the viscosity of the magnetorheological fluid inside the MRF device 24 increases depending on the magnitude of the current flowing to the MRF device 24, and the displacement unit 202 experiences a resistance force. The control unit 20 can change the magnitude of the resistance force on the displacement unit 202 and how it appears by changing the magnitude of the current supplied to the MRF device 24.

[0036] The tactile presentation device 2 varies the resistance force according to the amount of displacement of the displacement unit 202, i.e., the position of the user's finger, and can present a slippery tactile sensation, a firm tactile sensation, or a crunchy tactile sensation by repeatedly varying the magnitude of the resistance force.

[0037] 3 is a conceptual diagram showing the record layout of the sensory DB 110. The sensory DB 110 stores an image data table, an audio data table, and a tactile data table for presenting virtual objects. A single virtual object can be presented audiovisually and tactilely using a set of image data table, audio data table, and tactile data table. The sensory DB 110 stores data for multiple virtual objects, that is, multiple sets of image data table, audio data table, and tactile data table.

[0038] The image data table stores a plurality of image data tables, each of which associates finger position data indicating the user's finger position, i.e., the amount of displacement (angle) of the displacement unit 202, with image data (visual data) to be output at that finger position in order to visually present a virtual object, in association with an object ID for identifying the virtual object. The plurality of image data are, for example, data of frame images prepared for each displacement (angle) of the displacement unit 202 in the tactile presentation device 2. The frame image is a single still image that is displayed continuously in accordance with changes in the position of the finger or the displacement unit 202, and is recognized as an animation image of the virtual object.

[0039] The audio data table stores a plurality of image data tables that associate finger position data indicating the user's finger position, i.e., the displacement amount (angle) of the displacement unit 202, with audio data (auditory data) to be output at the finger position in order to audibly present a virtual object, in association with an object ID for identifying the virtual object. The audio data may be audio waveform data that differs depending on the displacement amount (angle) of the displacement unit 202 in the tactile presentation device 2. It may also be a timestamp of the audio data corresponding to the position of the displacement unit 202.

[0040] The tactile data table stores a plurality of tactile data tables that associate finger position data indicating the user's finger position, i.e., the displacement amount (angle) of the displacement unit 202, with tactile data to be output at that finger position in order to tactilely present a virtual object, in association with an object ID for identifying the virtual object. The tactile data is data that indicates the value of a current to be passed through the MRF device 24 of the tactile presentation device 2.

[0041] Fig. 4 is a flowchart showing the processing procedure of tactile presentation according to the present embodiment 1, and Fig. 5 is a conceptual diagram showing the tactile presentation method according to the present embodiment 1. It is assumed that the audiovisual presentation device 1 and the tactile presentation device 2 are activated and communication is established.

[0042] As shown in the left diagram of FIG. 5, the processing unit 10 of the audiovisual presentation device 1 displays a virtual object list screen 131 on the display unit 13 (step S111). Specifically, the processing unit 10 arranges representative images (thumbnail images) of multiple virtual objects stored in the sensory DB 110 and displays them on the display unit 13. The processing unit 10 can generate a representative image by reading image data associated with a predetermined displacement amount (angle) from the image data table and reducing the image data. The representative image may also be created and stored in advance.

[0043] Next, the processing unit 10 accepts the selection of a virtual object to be subjected to tactile presentation via the operation unit 15 (step S112). The user can select a desired virtual object from the virtual object list screen 131 by operating the operation unit 15.

[0044] When one virtual object is selected, the processing unit 10 displays a virtual object presentation preparation screen 132 on the display unit 13, as shown in the center diagram of Fig. 5 (step S113). The preparation screen 132 may be a monochrome screen, or may include text indicating that preparation is in progress, a still image, an animated image, or the like. The animated image may enlarge, reduce, or move the image of the selected virtual object.

[0045] Next, the processing unit 10 reads out the image data table, audio data table, and tactile data table associated with the object ID of the selected virtual object from the sensory DB 110 (step S114), and transmits the read tactile data table to the tactile presentation device 2 via the communication unit 12 (step S115).

[0046] The control unit 20 of the tactile presentation device 2 receives the tactile data table transmitted from the audiovisual presentation device 1 (step S151) and stores the received tactile data table in the tactile data storage unit 21 (step S152). When the tactile data storage unit 21 has stored the tactile data table, making it possible to provide tactile presentation of the virtual object, the control unit 20 transmits preparation completion notification data to the audiovisual presentation device 1 (step S153).

[0047] After transmitting the preparation completion notification data, the control unit 20 measures the displacement amount of the displacement unit 202 using the displacement sensor 25 (step S154) and transmits the measured displacement amount to the audiovisual presentation device 1 (step S155). The control unit 20 refers to the tactile data table and identifies tactile data corresponding to the measured displacement amount, i.e., a current value (step S156). The control unit 20 controls the magnitude of the current supplied from the power supply unit 23 to the MRF device 24 based on the identified current value, thereby presenting a tactile sensation (step S157). Through current control, a current indicated by the current value is supplied to the MRF device 24, generating a rotational resistance force according to the finger position, and presenting a tactile sensation to the user.

[0048] Meanwhile, the processing unit 10 of the audiovisual presentation device 1 that transmitted the tactile data table monitors the data transmitted from the tactile presentation device 2, and when it receives preparation completion notification data (step S116), it displays the object presentation screen 133 as shown in the right diagram of FIG. 5 and starts presenting the virtual object (step S117). That is, the processing unit 10 displays the image of the virtual object on the display unit 13 based on the image data of the virtual object selected in step S112. From step S117 onwards, the audiovisual presentation device 1 executes processing to visually and audibly present the virtual object, and from step S153 onwards, the tactile presentation device 2 starts processing to tactilely present the virtual object.

[0049] The processing unit 10 receives the displacement amount data transmitted from the tactile presentation device 2 (step S118), and refers to the image data table and audio data table read out in step S114 to identify the image data and audio data corresponding to the received displacement amount (step S119).

[0050] Next, the processing unit 10 displays an image of the virtual object on the display unit 13 and outputs sound from the sound output unit 14 based on the identified image data and sound data (step S120).

[0051] Next, the processing unit 10 determines whether or not an end operation has been performed (step S121). The user can perform an end operation of the virtual object presentation process by operating the operation unit 15. The processing unit 10 can accept the end operation via the operation unit 15.

[0052] If it is determined that the termination operation has not been performed (step S121: NO), the processing unit 10 returns the process to step S118. If it is determined that the termination operation has been performed (step S121: YES), the processing unit 10 ends the virtual object presentation process (step S122) and ends the process. The display of the virtual object image and the output of sound are stopped. The processing unit 10 may disconnect the communication as necessary, or may maintain the connection.

[0053] The processing unit 10 and the control unit 20 of the audiovisual presentation device 1 and the tactile presentation device 2 repeatedly execute the processes of steps S118 to S121 and steps S154 to S157, thereby reproducing the images and sounds of the virtual object and presenting the tactile sensation of the virtual object to the user.

[0054] According to the virtual object tactile presentation device 100 of embodiment 1 configured as described above, when an operation to start presenting a virtual object is performed, the preparation screen 132 is displayed, and the tactile data table is sent to the tactile presentation device 2, and then the object presentation screen 133 is displayed. Therefore, tactile presentation can be started without delay from the timing when the display of the virtual object starts. The audiovisual presentation device 1 is configured to start displaying virtual objects and outputting audio after confirming that preparation for tactile presentation on the tactile presentation device 2 side has been completed, thereby reliably preventing a delay between the start of displaying the virtual objects and the start of tactile presentation.

[0055] (Embodiment 2) The virtual object tactile presentation device 100 according to the second embodiment differs in the processing procedure related to the start of presentation of a virtual object. Since the other configurations of the virtual object tactile presentation device 100 are the same as those of the first embodiment, the same reference numerals and step numbers are used for the same parts, and detailed explanations are omitted. The same applies to the other embodiments.

[0056] 6 is a flowchart showing the processing procedure for tactile presentation according to embodiment 2. When the processing unit 10 of the audiovisual presentation device 1 according to embodiment 2 starts presentation (image display) of a virtual object (step S117), it transmits display start notification data indicating that display of the virtual object has started to the tactile presentation device 2 (step S218). The subsequent processing (steps S118 onwards) is the same as in embodiment 1.

[0057] After transmitting the completion notification data, the control unit 20 of the tactile presentation device 2 waits for tactile presentation, and when it receives display start notification data transmitted from the audiovisual presentation device 1 (step S217), it starts the process of tactile presentation of the virtual object (steps S154, S155, ...). Note that, if the displacement amount of the displacement unit 202 is less than a predetermined amount, the control unit 20 may be configured to wait until the displacement amount becomes equal to or greater than the predetermined amount, that is, until the user's fingers are opened, before starting the process of tactile presentation.

[0058] According to the virtual object tactile presentation device 100 of embodiment 2, it is possible to prevent the timing of starting the display of a virtual object from being delayed relative to the timing of the tactile presentation of the virtual object, and it is possible to start the audiovisual presentation and the tactile presentation of the virtual object at approximately the same timing.

[0059] (Embodiment 3) FIG. 7 is a flowchart showing the processing procedure for tactile presentation according to the third embodiment. The storage unit 11 of the audiovisual presentation device 1 according to the third embodiment stores correction data for correcting the image data table, audio data table, and tactile data table according to individual physical differences of users. The correction data is data for shifting the correspondence between the amount of displacement and the image data, audio data, and tactile data. For example, if the amount of displacement θ in the image data table is associated with the image data (θ), the correction data Δθ can be used to perform a correction to associate the amount of displacement (θ+Δθ) with the image data (θ). The same applies to the audio data and tactile data. Note that the content of the correction data is an example, and the correction data may also be data for performing correction according to the tactile presentation device 2 and the external environment.

[0060] The processing unit 10 of the audiovisual presentation device 1 according to the third embodiment corrects the haptic data table read out in step S114 based on the correction data stored in the storage unit 11 (step S315), and transmits the corrected haptic data table to the haptic presentation device 2 (step S316). Next, the processing unit 10 corrects the image data table and the audio data table based on the correction data (step S317). The subsequent processes (steps S116 onwards) are the same as those in the first embodiment.

[0061] According to the virtual object tactile presentation device 100 of embodiment 3, the tactile data table that needs to be sent to the tactile presentation device 2 is corrected first, and the image data table and audio data table are corrected after the tactile data table has been sent, so that presentation of the virtual object can begin sooner.

[0062] (Embodiment 4) 8 is a flowchart showing the processing procedure for tactile presentation according to embodiment 4. As in embodiment 3, the storage unit 11 of the audiovisual presentation device 1 according to embodiment 4 stores correction data for correcting the image data table, audio data table, and tactile data table according to individual physical differences of users.

[0063] The processing unit 10 of the audiovisual presentation device 1 according to the fourth embodiment transmits the tactile data table read out in step S114 and the correction data stored in the storage unit 11 to the tactile presentation device 2 (step S415). Next, the processing unit 10 corrects the image data table and the audio data table based on the correction data (step S416). The subsequent processes (steps S116 onwards) are the same as those in the first embodiment.

[0064] Meanwhile, the control unit 20 of the tactile presentation device 2 receives the tactile data table and correction data transmitted from the tactile presentation device 2 (step S451), corrects the received tactile data table based on the correction data (step S452), and stores the corrected tactile data table in the tactile data storage unit 21 (step S453). The subsequent processes (steps S153 onward) are the same as those in the first embodiment.

[0065] According to the virtual object tactile presentation device 100 of embodiment 4, the correction processing of the tactile data table, image data table, and audio data table is shared and executed by the audiovisual presentation device 1 and the tactile presentation device 2, so that the presentation of the virtual object can begin sooner.

[0066] (Embodiment 5) 9 is a flowchart showing the processing procedure for tactile presentation according to the fifth embodiment. The audiovisual presentation device 1 according to the fifth embodiment transmits an object ID and a tactile data table to the tactile presentation device 2, and the tactile presentation device 2 stores the tactile data table and the object ID in the tactile data storage unit 21. The tactile data table is a table containing tactile data for presenting a virtual object indicated by the object ID. The tactile data storage unit 21 stores at least one tactile data table even after the tactile presentation process. The built-in memory of the tactile control unit may be configured to have an area for storing and holding the tactile data table as a cache. Furthermore, the built-in memory of the tactile control unit and the tactile data storage unit 21 may each be configured to store one or more tactile data tables.

[0067] The processing unit 10 of the audiovisual presentation device 1 displays the virtual object list screen 131 on the display unit 13 (step S511). During a user's virtual object selection operation, the processing unit 10 inquires of the tactile presentation device 2 about the cache status of the tactile presentation device 2 and obtains a response indicating the virtual object ID of the tactile data table stored in the tactile presentation device 2 (steps S512, S551). In detail, the audiovisual presentation device 1 transmits request data inquiring about the cache status to the tactile presentation device 2. The processing unit 10 of the tactile presentation device 2 identifies the virtual object ID of the tactile data table stored in the tactile data storage unit 21 and / or the internal memory, and transmits the identified virtual object ID to the audiovisual presentation device 1. By receiving the virtual object ID transmitted from the tactile presentation device 2, the processing unit 10 of the audiovisual presentation device 1 recognizes the tactile data table stored in the tactile presentation device 2. The process of checking the cache state using the virtual object ID is an example, and the tactile data table stored in the tactile presentation device 2 may be managed using other data associated with the virtual object ID.

[0068] After completing the process of step S512, the processing unit 10 accepts the selection of a virtual object to be subjected to haptic presentation via the operation unit 15 (step S513). When one virtual object is selected, the processing unit 10 displays a virtual object presentation preparation screen 132 on the display unit 13 (step S514). Next, the processing unit 10 reads out an image data table, an audio data table, and a haptic data table associated with the object ID of the selected virtual object from the sensory DB 110 (step S515).

[0069] Then, the processing unit 10 determines whether the tactile presentation device 2 stores a tactile data table for the virtual object selected in step S513 (step S516). Based on the query result of step S512, the processing unit 10 can determine whether the tactile data table is cached. If it is determined that the tactile presentation device 2 does not store the tactile data table (step S516: NO), the processing unit 10 transmits the read tactile data table to the tactile presentation device 2 (step S115). The following processes (steps S116 to S151) are the same as those in the first embodiment.

[0070] If it is determined that the tactile presentation device 2 stores the tactile data table (step S516: YES), the processing unit 10 skips the process of transmitting the tactile data table and starts the process of presenting the virtual object (step S117). The subsequent processes (steps S118 to S154) are the same as those in the first embodiment.

[0071] According to the virtual object tactile presentation device 100 of embodiment 5, the tactile data table transmission process is skipped depending on the cache state of the tactile data table by the tactile presentation device 2, so that the presentation of the virtual object can be started earlier depending on the cache state.

[0072] In the fifth embodiment, an example was described in which the processing unit 10 reads various data tables and then checks whether or not a cache exists. However, the configuration may be such that the cache status is checked first, and if a cache exists, the image data table and audio data table of the virtual object are read without reading the haptic data table. [Explanation of symbols]

[0073] 1: Audiovisual presentation device 2: Tactile presentation device 3: Recording media 10: Processing section 11: Storage section 12: Communications Department 13: Display section 14: Audio output section 15:Operation section 20: Control section 21: Tactile data storage unit 22: Communications Department 23: Power supply section 24: MRF device 25: Displacement sensor 100: Virtual object tactile presentation device 131: Virtual object list screen 132: Preparation screen 133: Object presentation screen

Claims

1. A virtual object tactile presentation device including a visual presentation device that visually presents a virtual object and a tactile presentation device that tactilely presents the virtual object, The visual presentation device is a storage unit that stores visual data for visually presenting the virtual object and tactile data for tactilely presenting the virtual object; a display unit that displays the virtual object based on the visual data stored in the storage unit; a communication unit that transmits the tactile data stored in the storage unit to the tactile presentation device; an operation unit that accepts a start of presentation of the virtual object; a processing unit that, when the operation unit receives a command to start presenting the virtual object, causes the display unit to display an image related to preparation for presenting the virtual object, and transmits the haptic data to the haptic presentation device via the communication unit; Equipped with the tactile presentation device receives the tactile data transmitted from the visual presentation device, and when tactile presentation of the virtual object becomes possible, transmits to the visual presentation device preparation completion notification data indicating that preparation for tactile presentation is complete; The processing unit of the visual presentation device When the preparation completion notification data is received, the virtual object is displayed on the display unit. Virtual object tactile presentation device.

2. The communication unit of the visual presentation device When the display unit starts displaying the virtual object, display start notification data is transmitted to the tactile presentation device; When the display start notification data is received, the haptic presentation device starts a process of presenting a haptic sensation of the virtual object. The virtual object tactile presentation device according to claim 1 .

3. The processing unit when an image related to preparation for presentation of the virtual object is displayed on the display unit, reading and correcting the haptic data from the storage unit, and transmitting the corrected haptic data to the communication unit; The visual data read from the storage unit is corrected after the visual data is transmitted. The virtual object tactile presentation device according to claim 1 or 2.

4. A virtual object tactile presentation method using a visual presentation device that visually presents a virtual object and a tactile presentation device that tactilely presents the virtual object, the method comprising: Accepting a start of presentation of the virtual object; When a request to start presenting the virtual object is received, an image related to preparation for presenting the virtual object is displayed on the visual presentation device; transmitting the haptic data from the visual presentation device to the haptic presentation device; when the tactile presentation device receives the tactile data transmitted from the visual presentation device and becomes capable of tactile presentation of the virtual object, transmitting preparation completion notification data from the tactile presentation device to the visual presentation device, the preparation completion notification data indicating that preparation for tactile presentation is complete; When the visual presentation device receives the preparation completion notification data, the virtual object is displayed on the visual presentation device. A method for tactile presentation of virtual objects.

5. A control program for controlling an operation of a visual presentation device that visually presents a virtual object, the control program comprising: a storage unit that stores visual data for visually presenting a virtual object; and a processing unit that stores tactile data for tactilely presenting the virtual object, the control program comprising: Accepting a start of presentation of the virtual object; When a request to start presenting the virtual object is received, an image related to preparation for presenting the virtual object is displayed; transmitting the haptic data to a haptic presentation device that haptically presents the virtual object; the tactile presentation device receives the tactile data transmitted from the visual presentation device, and when tactile presentation of the virtual object becomes possible, receives preparation completion notification data transmitted from the tactile presentation device, the preparation completion notification data indicating that preparation for tactile presentation is complete; When the preparation completion notification data is received, the virtual object is displayed. A control program for causing the processing unit to execute the processing.

Images