Information processing device, tactile sense presentation method and computer program

Abstract

To provide an information processing device which can present tactile sensation of an object according to motion speed of the object.SOLUTION: An information processing device comprises a processing unit which executes processing for presenting tactile sense of an object by using a tactile sense presentation device, and has a standard mode, and a slow motion mode or a quick motion. The processing unit, when it is in the standard mode, presents the tactile sense of the object, on the basis of tactile sense data for presenting the tactile sense of the object which changes in a standard speed motion, and when it is in the slow motion mode or the quick motion, presents the tactile sense of the object, on the basis of the tactile sense data for presenting the tactile sense of the object which changes in the slow motion or quick motion.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a tactile presentation method, and a computer program. [Background technology]

[0002] Various technologies for presenting the tactile sensation of an object (haptics) have been proposed. Based on knowledge of magnetorheological fluids, the inventors have proposed a tactile sensation presentation device that can be operated by a user and presents different sensations depending on the object (Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6906275 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide an information processing device, a tactile sensation presentation method, and a computer program that can present the tactile sensation of an object according to the motion speed of the object. [Means for solving the problem]

[0005] An information processing device according to one aspect of the present disclosure is an information processing device that includes a processing unit that executes processing for presenting the tactile sensation of an object using a tactile presentation device, and has a standard mode and a slow motion mode or a fast motion mode, and when in standard mode, the processing unit presents the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in standard speed motion, and when in slow motion mode or fast motion mode, presents the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in slow motion or fast motion.

[0006] A tactile presentation method according to one aspect of the present disclosure is a tactile presentation method for presenting the tactile sensation of an object using a tactile presentation device, which has a standard mode and a slow motion mode or a fast motion mode, and when in standard mode, presents the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in standard speed motion, and when in slow motion mode or fast motion mode, presents the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in slow motion or fast motion.

[0007] A computer program according to one aspect of the present disclosure is a computer program for causing a computer to execute a process for presenting the tactile sensation of an object using a tactile presentation device, the computer program having a standard mode and a slow motion mode or a fast motion mode, and causing the computer to execute a process for presenting the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in standard speed motion when in standard mode, and for presenting the tactile sensation of the object based on tactile data for presenting the tactile sensation of the object changing in slow motion or fast motion when in slow motion mode or fast motion mode. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to present a tactile sensation of an object according to the motion speed of the object. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing a game system according to the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the internal configuration of an MR device. [Figure 3] FIG. 10 is an explanatory diagram showing an example of the contents of a sensory DB. [Figure 4] FIG. 1 is a block diagram showing a configuration of a tactile presentation device. [Figure 5] FIG. 2 is a block diagram showing the internal configuration of the server device. [Figure 6] 10 is a flowchart illustrating an example of a processing procedure of the MR device and the tactile presentation device. [Figure 7] 10A and 10B are schematic diagrams showing changes in the image of a character in standard mode. [Figure 8] 10A and 10B are schematic diagrams showing changes in the image of a character in slow motion mode. [Figure 9] FIG. 10 is a conceptual diagram showing a method for changing haptic data according to a setting mode. [Figure 10] FIG. 10 is a conceptual diagram showing a method for changing haptic data in slow motion according to the second embodiment. [Figure 11] FIG. 10 is a conceptual diagram showing a method for changing haptic data in a quick motion mode according to the second embodiment. [Figure 12] 10A and 10B are schematic diagrams showing changes in the image of a character according to the third embodiment. [Figure 13] FIG. 10 is a conceptual diagram showing a method for changing haptic data according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] A tactile presentation system, an MR device, a tactile presentation method, and a computer program according to embodiments of the present disclosure will be described below with reference to the drawings. In the following embodiments, a game system to which the tactile presentation system according to the present embodiments is applied will be described. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims. Furthermore, at least some of the embodiments described below may be combined in any manner.

[0011] (Embodiment 1) FIG. 1 is a schematic diagram showing a game system 100 in the first embodiment. The game system 100 includes an MR device 1, which is an information processing device, a tactile presentation device 2, and a server device 3. The MR device 1 and the tactile presentation device 2 are communicatively connected via short-range wireless communication and exchange data with each other. The MR device 1 and the server device 3 exchange data with each other via a network N. The network N includes the Internet N1, a carrier network N2, a base station BS, and an access point AP.

[0012] The MR device 1 is a see-through head-mounted display that performs spatial mapping of the real space that the user is viewing through a see-through display 15 and projects a hologram of a virtual object, a character C, thereby constructing a mixed reality in which a virtual object is visualized in the real space. The MR device 1 also has a function of audibly presenting the character C. The MR device 1 can be, for example, HoloLens (registered trademark) from Microsoft Corporation or MagicLeap (registered trademark) from MagicLeap Corporation. The MR device 1 is a type of augmented reality device that creates an augmented reality, and the augmented reality device also includes an AR (Augmented Reality) device (not shown). Instead of the MR device 1, a smartphone, a tablet terminal, a laptop PC (Personal Computer), a dedicated game console, etc. may be used. The MR device 1 may be a dedicated game console integrated with the haptic presentation device 2, i.e., a game console including the haptic presentation device 2 in the controller.

[0013] The tactile presentation device 2 is a device that a user can operate by moving their finger while holding it along the displacement unit 202. The tactile presentation device 2 reads the position of the displacement unit 202, which is displaced by the user's finger movement, and controls the built-in MRF (Magneto-Rheological Fluid) device 24 according to the position to generate a force sensation due to a reaction force (rotational resistance) against the user's operation on the displacement unit 202, thereby presenting a tactile sensation. The form of the displacement unit 202 of the tactile presentation device 2 is not limited to that shown in FIG. 1 , but may be a button, a stick, a cushion-like object covered with a cover, or a glove-type object. The tactile presentation device 2 may be an active tactile presentation device that uses a motor, a piezoelectric element, an ultrasonic oscillator, or the like instead of the MRF device 24 to generate a force sensation through rotational force or vibration in response to a user's operation, and may be combined with devices that present vibration, warmth, coldness, or electrical stimulation in addition to the displacement unit 202. An active type tactile presentation device may be combined with a passive type tactile presentation device such as the present invention. The device may be installed on the ground or a wall and operated by the user's palms or feet.

[0014] In the game system 100, an MR device 1 having a game application program (hereinafter referred to as a game application P1) installed therein communicates with a server device 3 to execute a game. The MR device 1 acquires game content from a content database (DB) 311 of the server device 3 via a network N. The game content includes visual data, auditory data, and tactile data for visually, auditorily, and tactilely presenting a virtual object, i.e., a game character C. The MR device 1 displays an image of the character C on the see-through display 15 based on the visual data included in the acquired game content (visually presenting the character C). Furthermore, the MR device 1 outputs the voice and sound effects of the character C from the audio output unit 17 based on the auditory data included in the acquired game content (auditory presenting the character C). Furthermore, the MR device 1 transmits tactile data included in the acquired game content to the tactile presentation device 2 to tactilely present the character C.

[0015] In the game system 100, a character C to be operated can be selected. The MR device 1 presents a tactile sensation corresponding to the selected character C by the tactile presentation device 2. When the user presses the displacement unit 202 of the tactile presentation device 2 with a finger, a tactile sensation corresponding to the type of character C is generated. In addition, the MR device 1 changes and outputs images and sounds according to the displacement of the displacement unit 202. The image of the displayed character C changes with the operation of the displacement unit 202 and the presentation of tactile sensation, and sounds or sound effects according to the changes are output from the audio output unit 17.

[0016] 2 is a block diagram showing the internal configuration of the MR device 1. The MR device 1 is a see-through head-mounted display, and application software (not shown) for constructing mixed reality is pre-installed on it. By executing the application software, the MR device 1 can construct mixed reality in which a character C, which is a virtual object, is projected onto the real space visually recognized by the user.

[0017] The MR device 1 includes a processing unit 10, a storage unit 11, a first communication unit 12, a second communication unit 13, an imaging unit 14, a see-through display 15, an eye tracking sensor 16, and an audio output unit 17. The processing unit 10 is a processor using a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit). The processing unit 10 executes processing to construct mixed reality and visually present a character C using application software for constructing MR stored in the storage unit 11 and a game application P1 including a computer program (program product) according to the first embodiment.

[0018] The memory unit 11 is a non-volatile memory such as a flash memory. The memory unit 11 stores the game application P1, a sensory DB 110, and other data referenced by the processing unit 10. The sensory DB 110 is a storage that stores tactile data, visual (images, videos, text) data, and auditory (audio, sound effects) data of the character C. Details of the data stored in the sensory DB 110 will be described later. The sensory DB 110 may sequentially acquire this data for each character C from the content DB 311 of the server device 3, or may store this data in advance.

[0019] The game application P1, i.e., the computer program according to the first embodiment, may be provided by a non-transitory recording medium M1 on which the game application P1 or the computer program is readably recorded. The storage unit 11 stores the game application P1 or the computer program read from the recording medium M1 by a reading device (not shown). The recording medium M1 may be, for example, a magnetic disk, an optical disk, or a semiconductor memory. Alternatively, the computer program may be downloaded from an external server connected to a communications network and stored in the storage unit 11. The game application P1 may be composed of a single computer program or multiple computer programs, and may be executed on a single computer or multiple computers interconnected by a communications network.

[0020] The first communication unit 12 includes a communication circuit that realizes communication with the server device 3 via a network N including the Internet N1 or a carrier network N2. Specifically, the first communication unit 12 may be a wireless communication device that connects to the carrier network N2, or may be a wireless communication device for Wi-Fi. The processing unit 10 transmits and receives data to and from the server device 3 via the first communication unit 12.

[0021] The second communication unit 13 is a communication module for short-range wireless communication, for example, Bluetooth (registered trademark). The communication module includes a communication circuit. The processing unit 10 transmits and receives data to and from the tactile presentation device 2 via the second communication unit 13.

[0022] The imaging unit 14 is a camera that captures an image of the field of view of a user wearing the MR device 1, and includes an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device). The imaging unit 14 captures a moving image based on a control instruction from the processing unit 10.

[0023] The see-through display 15 has a see-through holographic lens and a projection device, and constructs a mixed reality in which a character C, which is a virtual object, is visualized in the real space that the user sees through the see-through display 15.

[0024] The eye tracking sensor 16 includes an infrared camera that captures an image of the user, and measures the direction of the user's gaze based on the captured infrared image.

[0025] The audio output unit 17 includes a speaker, etc. The processing unit 10 causes the audio output unit 17 to output, based on the game application P1, sounds, music, etc. corresponding to the auditory data included in the game content stored in the storage unit 11 or the game content provided by the server device 3.

[0026] FIG. 3 is an explanatory diagram showing an example of the contents of the sensory DB 110. The sensory DB 110 stores a plurality of haptic data that differ depending on the type of content, in association with a content ID. The haptic data is, for example, a table that stores the displacement amount (angle) of the displacement unit 202 and the value of the current supplied to the MRF device 24 of the haptic presentation device 2, in association with each other. The haptic data is a current value for presenting the haptic sensation of character C that changes at a predetermined standard speed. As will be described later, the haptic presentation system according to this embodiment has a standard mode in which character C changes at a standard speed motion, a slow motion mode in which character C changes at a slower speed than in the standard mode, and a quick motion mode in which character C changes at a faster speed than in the standard mode. The haptic sensation of character C presented to the user is corrected depending on each mode.

[0027] The sensory DB 110 stores image data, such as images or videos (frame images) of character C, which vary depending on the type of content, as visual data in association with a content ID. The sensory DB 110 may store multiple different patterns of visual data, such as pattern α, pattern β, ..., in association with one content ID. In the example shown in FIG. 3, the visual data of pattern α is image data in which the image does not change depending on the displacement amount of the displacement unit 202. The visual data of pattern β is image data in which the image changes depending on the displacement amount of the displacement unit 202. The visual data of pattern β includes multiple image data respectively associated with multiple displacement amounts (angles) of the displacement unit 202. The visual data of pattern α, pattern β, and other patterns may be selected randomly or may be selected according to the scenario and situation of the game.

[0028] The sensory DB 110 stores audio data of the voice or sound effects of character C, which differ depending on the type of content, as auditory data, in association with a content ID. The sensory DB 110 may store multiple different patterns of auditory data, such as pattern α, pattern β, ..., in association with one content ID. In the example shown in FIG. 3, the auditory data of pattern α is audio data in which the audio does not change depending on the displacement amount of the displacement section 202. The auditory data of pattern β is audio data in which the audio changes depending on the displacement amount of the displacement section 202. The auditory data of pattern β includes multiple pieces of audio data respectively associated with multiple displacement amounts (angles) of the displacement section 202.

[0029] Fig. 4 is a block diagram showing the configuration of the tactile presentation device 2. The tactile presentation device 2 includes a gripping body 201 that is gripped and operated by a user as shown in Fig. 1, a control unit 20, a tactile data storage unit 21, a communication unit 22, a power supply unit 23, an MRF device 24, and a displacement sensor 25 shown in Fig. 4. Note that circuits related to control and power supply of the control unit 20, power supply unit 23, etc. shown in Fig. 4 may be configured integrally with the gripping body 201, or may be configured separately and connected by wire or wirelessly.

[0030] The gripper 201 includes a hollow, disk-shaped casing, a displacement unit 202, a binder 203, and a link mechanism 204. The casing houses a magnetorheological fluid and a rotor having a rotating shaft. The rotor is a disk-shaped member that rotates inside the casing filled with magnetorheological fluid. The rotor is subjected to rotational resistance force by the magnetorheological fluid. The gripper 201 is also provided with an electromagnet (not shown) for applying a magnetic field to the magnetorheological fluid. The electromagnet, together with the magnetorheological fluid and the rotor, constitutes the MRF device 24. Specifically, the electromagnet has a yoke arranged to sandwich the disk-shaped rotor with a gap therebetween, and a coil attached to the yoke. When a current flows through the coil, a magnetic field is generated, which changes the viscosity (shear stress) of the magnetorheological fluid, thereby applying rotational resistance force to the rotating shaft.

[0031] The displacement portion 202 is a band-shaped member curved along a portion of the circumferential surface of the casing. The displacement portion 202 is formed of a flexible but highly rigid material. The base end of the displacement portion 202 is rotatably supported on the casing by a support shaft so that the tip of the displacement portion 202 can move toward and away from the casing. A binder 203 for binding the user's fingers is provided on the outer surface of the tip of the displacement portion 202. The binder 203 is a ring-shaped band-shaped member, such as a cloth tape. The displacement portion 202 may have a surface covered with a material that provides a variety of textures, such as silicone rubber or a fur-like material. The tip of the displacement portion 202 and the rotor's rotation shaft are connected by a link mechanism 204. The link mechanism 204 converts the movement of the displacement portion 202 toward and away from the casing into rotational movement of the rotation shaft and transmits it.

[0032] 1, a user holds the gripping body 201 with, for example, the thumb and middle finger, and inserts the index finger or other fingers into the binder 203 along the displacement portion 202. The user can move the displacement portion 202 by pushing the index finger in, or can extend the index finger to move the displacement portion 202 away from the gripping body 201.

[0033] The control unit 20 includes a control processor 20a such as a CPU or an MPU (Micro-Processing Unit), memories such as a ROM (Read Only Memory) 20b and a RAM (Random Access Memory) 20c, and an I / O 20d. The control unit 20 is, for example, a microcontroller. The control unit 20 controls each component based on a control program P2 stored in the ROM 20b to provide tactile sensations. The control program P2 may be a control program P2 stored in a storage medium (not shown) that is read by the control unit 20 and copied to a ROM 20b such as a flash memory. The control program P2 may also be a program downloaded from the server device 3 or another program server device and stored in an executable format.

[0034] The haptic data storage unit 21 is an auxiliary memory for the control unit 20, and stores haptic data, which is control data for the MRF device 24, in a rewritable manner.

[0035] The communication unit 22 is a communication module for short-range wireless communication, such as Bluetooth (registered trademark). The communication module includes a communication circuit. The control unit 20 transmits and receives data to and from the MR device 1 via the communication unit 22.

[0036] The control unit 20 is connected to the power supply unit 23, the MRF device 24, and the displacement sensor 25 via the I / O 20d, and exchanges signals with them.

[0037] The power supply unit 23 includes a rechargeable battery. When the power supply unit 23 is turned on, it supplies power to each component and the MRF device 24.

[0038] The displacement sensor 25 measures the position (angle) of the displacement portion 202 and outputs the position (angle) to the control portion 20. The displacement sensor 25 measures and outputs the displacement of the displacement portion 202 as an angle. The displacement sensor 25 may be configured with a plurality of sensors such as a gyro sensor and an acceleration sensor.

[0039] In the tactile presentation device 2 configured as shown in FIG. 4, when the displacement unit 202 is operated by a user, the displacement of the displacement unit 202 is transmitted to the rotational axis of the rotor of the MRF device 24 via the link mechanism 204. When the MRF device 24 is not operating, i.e., when the control current is zero, the rotational axis rotates freely, and the displacement unit 202 moves without resistance. On the other hand, when the MRF device 24 is operating and a control current is flowing, the viscosity (shear stress) of the magnetorheological fluid inside the MRF device 24 changes depending on the magnitude of the current flowing to the MRF device 24. The control unit 20 can change the resistance force against the displacement unit 202 and how it appears by continuously changing the magnitude of the current to the MRF device 24 or by varying the current value at a predetermined frequency. In other words, the control unit 20 can present any tactile sensation by controlling the waveform and current value of the current flowing to the MRF device 24.

[0040] In this way, the tactile presentation device 2 can present a slippery tactile sensation by varying the resistance (current value) according to the amount of pressure applied to the displacement section 202, or a firm tactile sensation by increasing the resistance as the amount of pressure applied increases, or a crunchy tactile sensation by repeatedly varying the resistance or by repeatedly turning the resistance on and off using a square wave.

[0041] 5 is a block diagram showing the internal configuration of the server device 3. The server device 3 is a game server capable of transmitting and receiving data to and from the MR device 1. In the following description, the server device 3 will be described as a single game server, but it may also be configured such that multiple server computers are communicatively connected via a network N to perform distributed processing.

[0042] The server device 3 includes a server processing unit 30, a storage unit 31, and a server communication unit 32. The server processing unit 30 is a processor that uses a CPU and / or a GPU. The server processing unit 30 executes processing as a game server based on a server program P3 stored in the storage unit 31.

[0043] The storage unit 31 is a non-volatile memory such as a hard disk or a flash memory. The storage unit 31 stores data referenced by the server processing unit 30. The storage unit 31 stores a server program P3. The server program P3 may be a server program P3 stored in a storage medium M3 that is read by the server processing unit 30 and copied to the storage unit 31. The server program P3 may also be a program downloaded by the server processing unit 30 from another program server device via the server communication unit 32 and stored therein.

[0044] The storage unit 31 constitutes a content DB 311 and stores game content used in the game. The game content stored in the storage unit 31 includes character data and the like. The storage area of ​​the storage unit 31 that stores the game content data may be an external storage medium connected for communication via the network N.

[0045] The character data is data on a character C who appears in the game. The character data includes a name (identification data), tactile data to be included in the sensory DB 110, visual data (images, videos, text, etc.), and auditory data (voice, sound effects).

[0046] The storage unit 31 stores the entity (image data, etc.) of these game contents in association with content identification data (hereinafter referred to as content ID). The storage unit 31 stores the content ID of the game content available to each user on each MR device 1 in association with a user ID that identifies the user. For example, the storage unit 31 stores the content ID and the number of content IDs that identify characters C, etc. that the user can use in the game application P1 in association with the user ID. In response to an acquisition request that specifies a content ID, the server processing unit 30 reads a copy of the game content data from the content table and provides it to the requester. The storage area that stores the data associated with these user IDs may be an external storage medium that is communicatively connected via the network N.

[0047] The server communication unit 32 includes a communication circuit that realizes communication with the MR device 1 via the network N. Specifically, the server communication unit 32 is a network card. The server communication unit 32 may be a wireless communication module that connects to the carrier network N2, or may be a wireless communication module for WiFi. The server processing unit 30 transmits and receives data to and from the MR device 1 via the server communication unit 32.

[0048] 6 is a flowchart showing an example of the processing procedure of the MR device 1 and the tactile presentation device 2. Here, it is assumed that communication between the MR device 1 and the tactile presentation device 2 is established, and the main screen is projected and visualized as mixed reality by the MR device 1. The main screen includes images of multiple characters C and selection buttons for each character C.

[0049] The processing unit 10 of the MR device 1 accepts the selection of a character C from the main screen and acquires game content for the selected character C from the server device 3 (step S101). The storage unit 11 stores tactile data, visual data, and auditory data of the character C included in the game content.

[0050] The processing unit 10 transmits tactile data for the selected character C and initially set mode setting information to the tactile presentation device 2 (step S102). The initially set mode setting information is information indicating that the mode is the standard mode. Furthermore, the processing unit 10 displays a game screen for the selected character C (step S103). By the processing of step S103, an image of character C and a finger image F are displayed on the game screen. Furthermore, a sound corresponding to character C is output from the sound output unit 17.

[0051] The tactile presentation device 2 receives tactile data and initially set mode setting information transmitted from the MR device 1 during communication connection (step S201), and stores the received tactile data and mode setting information in the tactile data storage unit 21 (step S202). The tactile data received in step S201 is a list table of current values ​​for each displacement amount described above.

[0052] The control unit 20 of the tactile presentation device 2 samples a signal corresponding to the displacement amount (angle) of the displacement unit 202 output from the displacement sensor 25 (step S203). The control unit 20 transmits the displacement amount obtained by sampling to the MR device 1 (step S204). The displacement amount (angle) may be a relative displacement amount from the upper end or an absolute position detected by the displacement sensor 25.

[0053] The control unit 20 references the haptic data stored in the haptic data storage unit 21 to find a current value corresponding to the amount of displacement obtained by sampling (step S205). The control unit 20 then changes the current value of the haptic data in accordance with the mode setting information stored in the haptic data storage unit 21 (step S206). That is, when character C moves in slow motion, the control unit 20 changes the current value associated with the haptic data to a larger value than when moving at standard speed. Similarly, when character C moves in quick motion, the control unit 20 changes the current value associated with the haptic data to a smaller value than when moving at standard speed. The method of changing the current value in accordance with the set mode will be described in detail later.

[0054] Then, the processing unit 10 outputs a current according to the referenced and corrected current value to the MRF device 24 (step S207), and returns the process to step S203. As will be described later, if mode setting information has been transmitted from the MR device 1, the tactile presentation device 2 receives the mode setting information transmitted from the MR device 1 (step S208). The tactile presentation device 2 stores the received mode setting information in the tactile data storage unit 21 (step S209), and returns the process to step S203. That is, the processing unit 10 changes the set mode, and returns the process to step S203.

[0055] By repeating the processing of steps S203 to S207 or steps S203 to S209, it is possible to present the tactile sensation of the character C according to the operation amount (displacement amount) of the tactile presentation device 2. In addition, a natural tactile sensation suitable for the set mode related to the motion speed of the character C is presented.

[0056] Meanwhile, the processing unit 10 of the MR device 1 displaying the game screen receives the displacement amount transmitted from the tactile presentation device 2 (step S104). The processing unit 10 refers to the visual data and audio data stored in the sensory DB 110, and reads out image data and audio data corresponding to the received displacement amount (step S105). Then, the processing unit 10 outputs the read image data and audio data on the see-through display 15 and the audio output unit 17, respectively (step S106).

[0057] Next, the processing unit 10 displays a mode switching image SW on the game screen (step S107). Modes that can be switched using the mode switching image SW include slow motion mode, quick motion mode, and standard mode. In the example shown in FIG. 7B, the current mode is standard mode, and a mode switching image SW for switching to slow motion mode and a "SLOW" button are displayed. In the example shown in FIG. 8C, the current mode is slow motion mode, and a mode switching image SW for switching to standard mode and a "BACK" button are displayed. Similarly, the processing unit 10 may appropriately display a mode switching image SW for switching to quick motion mode. Of course, the processing unit 10 may display both a mode switching image SW for switching to slow motion mode and a mode switching image SW for switching to quick motion mode.

[0058] The processing unit 10 may always display the mode switching image SW while displaying the character C, or may display the mode switching image SW in a specific scene. Data indicating that the scene is one in which the mode switching image SW should be displayed may be included in the image data or audio data, for example.

[0059] Next, the processing unit 10 determines whether or not to switch modes, that is, whether or not the mode switching image SW has been operated by the user (step S108). If it is determined that the mode switching image SW has been operated (step S108: YES), the processing unit 10 switches modes and transmits mode setting information indicating the switched mode to the tactile presentation device 2 (step S109).

[0060] When the processing of step S109 is completed or when it is determined that the mode switching image SW has not been operated (step S108: NO), the processing unit 10 determines whether or not to end the game application P1 based on the game scenario (step S110). The processing unit 10 may determine to end the game application P1 when the displacement unit 202 is pressed all the way and the displacement amount reaches a predetermined end position. When it is determined not to end the game application P1 (step S110: NO), the processing unit 10 returns the processing to step S104 and continues the processing related to the tactile presentation of the character C. When it is determined to end the game application P1 (step S110: YES), the processing unit 10 ends the game application P1.

[0061] By repeating steps S104 to S109, the image and sound of the character C according to the operation amount (displacement amount) of the tactile presentation device 2 can be output.

[0062] Fig. 7 is a schematic diagram showing the change in the image of character C in standard mode, and Fig. 8 is a schematic diagram showing the change in the image of character C in slow motion mode. Note that Figs. 7 and 8 are schematic illustrations of the screen visualized in mixed reality. The rectangular frame is drawn for convenience of drawing, and does not necessarily have to be visualized in mixed reality.

[0063] For example, the processing unit 10 displays an image of a ball, which is an example of character C, and a finger image F, as shown in FIGS. 7A and 8A. When the displacement unit 202 of the tactile presentation device 2 is pressed and the displacement amount changes, the finger image F approaches and contacts the image of the ball, as shown in FIGS. 7B and 8B. Around the time when the finger image F contacts the image of the ball, a current is supplied to the MRF device 24, and tactile presentation corresponding to character C begins. Here, the processing unit 10 displays a mode switching image SW for switching to slow motion mode and a "slow" button. If the "slow" button is not operated and the displacement unit 202 of the tactile presentation device 2 is further pressed while in standard mode, the processing unit 10 displays an image of a crushed ball, as shown in FIG. 7C. The tactile presentation device 2 provides a tactile presentation corresponding to the displacement amount based on the tactile data in standard mode. When character C is a ball, the tactile presentation device 2 provides a tactile presentation in which the resistance increases as the displacement amount increases.

[0064] When the mode switching image SW is operated to switch to the slow motion mode, the processing unit 10 displays an enlarged image of the character C, as shown in Fig. 8C. The processing unit 10 also displays the mode switching image SW for switching to the standard mode and a "back" button. When the "back" button is operated, the mode returns to the standard mode, and the operation shown in Fig. 7 is performed.

[0065] When the displacement unit 202 of the tactile presentation device 2 is pressed further in slow motion mode, the finger slowly bites into the ball, and an image of the dented ball is displayed, as shown in FIG. 8C. The processing unit 10 changes the tactile data to data for slow motion mode, and the tactile presentation device 2 provides a tactile presentation according to the amount of displacement. Note that when character C is a ball, the tactile presentation device 2 provides a tactile presentation in which the resistance increases as the amount of displacement increases. When the displacement unit 202 of the tactile presentation device 2 is pressed further and reaches a predetermined end position, the processing unit 10 displays an image of a crushed ball, as shown in FIG. 8D.

[0066] Here, the change in the image accompanying the displacement of the displacement unit 202 has been mainly described, but sound is also output according to the amount of displacement of the displacement unit 202. For example, as the ball dents as shown in Fig. 8C, sound effects such as "squeeze" are output from the sound output unit 17. Also, as shown in Figs. 7C and 8D, when the ball is crushed, sound effects such as "splat" and "bang" are output from the sound output unit 17.

[0067] Although FIGS. 7 and 8 show an example of switching from standard mode to slow motion mode, switching from standard mode to quick motion mode can also be achieved in a similar manner.

[0068] Fig. 9 is a conceptual diagram showing how to change haptic data according to the set mode. The haptic data table at the center of Fig. 9 stores the displacement amount (angle) of the displacement unit 202 in standard mode and the value of the current supplied to the MRF device 24 in association with each other.

[0069] The haptic data on the left side of FIG. 9 represents haptic data in slow motion mode. The control unit 20 changes the current value in standard mode to a larger current value. For example, the control unit 20 can obtain haptic data in slow motion mode by multiplying the current value in standard mode by a first correction coefficient (>1). In the example shown in FIG. 9, the first correction coefficient is 1.5. The value of the first correction coefficient may be stored in advance in the tactile presentation device 2, may be included in the haptic data, or may be included in the mode setting information. Note that while an example in which all current values ​​in standard mode are multiplied by the first correction coefficient has been described, some current values ​​may be multiplied by the first correction coefficient. Furthermore, the current value may be multiplied by a first correction coefficient that varies depending on the displacement amount of the displacement unit 202. Furthermore, the control unit 20 may change the current value using a function. The control unit 20 may perform interpolation processing on the displacement amounts and current values ​​stored in the haptic data table. In other words, if the intervals between the displacement amounts stored in the table are coarse, additional displacement amounts may be added so that the intervals between the displacement amounts become closer. The current value corresponding to the amount of displacement to be added is determined by interpolation.

[0070] The haptic data on the right side of FIG. 9 represents haptic data in quick motion mode. The control unit 20 changes the current value in standard mode to a smaller current value. For example, the control unit 20 can obtain haptic data in slow motion mode by multiplying the current value in standard mode by a second correction coefficient (<1). In the example shown in FIG. 9, the second correction coefficient is 0.5. The value of the second correction coefficient may be stored in advance in the tactile presentation device 2, or may be included in the haptic data or mode setting information. While an example in which all current values ​​in standard mode are multiplied by the second correction coefficient has been described, some current values ​​may also be multiplied by the second correction coefficient. Furthermore, the current value may be multiplied by a second correction coefficient that varies depending on the displacement amount of the displacement unit 202. Furthermore, the control unit 20 may change the current value using a function. The control unit 20 may also execute a process of thinning out the displacement amounts and current values ​​stored in the haptic data table.

[0071] As described above, according to the game system 100, MR device 1, tactile presentation method, and computer program of game application P1 of this embodiment 1, by correcting the tactile data according to the motion speed of the virtual object character C, a natural tactile sensation according to the motion speed can be presented.

[0072] In the first embodiment, a tactile presentation device 2 using a magnetorheological fluid is exemplified, but there is no particular limitation on the type of tactile presentation device 2. Even in a tactile presentation device 2 that generates a force sensation in the user using a motor, a piezoelectric element, ultrasonic waves, or the like, a natural tactile sensation according to the motion speed of the character C can be presented by changing the control value according to the motion speed of the character C.

[0073] (Embodiment 2) The game system (tactile presentation system) 100 in embodiment 2 differs from embodiment 1 in the method of changing tactile data according to the motion speed setting mode. The other configurations of the game system 100 are the same as those of the game system 100 in embodiment 1, so the same reference numerals are used for the same parts and detailed descriptions will be omitted.

[0074] The tactile presentation device 2 according to the second embodiment presents a tactile sensation including a scratching pseudo vibration by passing a rectangular wave current through the MRF device 24. Note that the rectangular shape is an example of a periodically varying current waveform, and the current waveform is not particularly limited.

[0075] 10 is a conceptual diagram showing a method for changing haptic data in slow motion according to embodiment 2. The haptic data table in the upper diagram is haptic data in standard mode. The haptic data according to embodiment 2 is a table that stores the displacement amount (angle) of the displacement unit 202, the value of the current supplied to the MRF device 24 of the tactile presentation device 2, the frequency of the square wave current, and the duty ratio in association with each other.

[0076] The haptic data in the lower diagram shows haptic data in slow motion mode. The control unit 20 increases the current value in standard mode and decreases the frequency. In other words, the control unit 20 increases the current fluctuation period. For example, as in the first embodiment, the control unit 20 can obtain haptic data in slow motion mode by multiplying the current value in standard mode by a first correction coefficient (>1) and multiplying the frequency by a first frequency correction coefficient (<1). In the example shown in FIG. 10, the first correction coefficient is 1.5 and the first frequency correction coefficient is 0.5. While the example in which all frequencies in standard mode are multiplied by the first frequency correction coefficient has been described, some frequencies may be multiplied by the first frequency correction coefficient. Alternatively, different first frequency correction coefficients may be multiplied depending on the displacement amount of the displacement unit 202. Furthermore, the control unit 20 may change the frequency using a function. The control unit 20 may also perform interpolation processing on the displacement amount and frequency stored in the haptic data table.

[0077] FIG. 11 is a conceptual diagram illustrating a method for changing haptic data in quick motion mode according to the second embodiment. The upper diagram is the same as the upper diagram in FIG. 10. The haptic data in the lower diagram shows haptic data in quick motion mode. The control unit 20 reduces the standard mode current value and increases the frequency. In other words, the control unit 20 shortens the current fluctuation period. For example, as in the first embodiment, the control unit 20 can obtain haptic data in quick motion mode by multiplying the standard mode current value by a second correction coefficient (<1) and multiplying the frequency by a second frequency correction coefficient (>1). In the example shown in FIG. 11, the second correction coefficient is 0.5 and the second frequency correction coefficient is 1.5. Note that while an example has been described in which all frequencies in standard mode are multiplied by the second frequency correction coefficient, some frequencies may be multiplied by the second frequency correction coefficient. Alternatively, different second frequency correction coefficients may be multiplied depending on the amount of displacement of the displacement unit 202. Furthermore, the control unit 20 may change the frequency using a function. The control unit 20 may execute a process of thinning out the displacement amounts and frequencies stored in the tactile data table.

[0078] As described above, the game system 100, MR device 1, tactile presentation method, and computer program of game application P1 according to this embodiment 2 can present natural force and vibration sensations according to the motion speed of the virtual object character C.

[0079] (Embodiment 3) The game system 100 in embodiment 3 differs from embodiment 1 in the method of presenting a tactile sensation according to the motion speed setting mode. Since the other configurations of the game system 100 are the same as those of the game system 100 according to embodiment 1, the same reference numerals are used for the same parts and detailed descriptions are omitted.

[0080] The tactile presentation device 2 according to the third embodiment is a glove-shaped device that includes a vibration element and presents a vibration sensation by a rectangular wave current. The other hardware configurations related to control, storage, and communication are the same as those of the first embodiment.

[0081] In the game system 100 according to the third embodiment, the MR device 1 accepts a mode setting before the start of a game. When the slow motion mode is selected, the MR device 1 automatically switches from the standard mode to the slow motion mode when the moving speed of the character C reaches or exceeds a predetermined speed.

[0082] Fig. 12 is a schematic diagram showing changes in the image of character C according to embodiment 3, and Fig. 13 is a conceptual diagram showing a method for changing tactile data according to embodiment 3. The horizontal axis in Fig. 13 represents time, and the vertical axis represents the value of the current flowing through the vibration element of the tactile presentation device 2. Fig. 13A shows the current value in standard mode, and the lower diagram in Fig. 13 shows the current value in slow motion mode.

[0083] In the third embodiment, the MR device 1 displays, for example, a ball-shaped character C and a glove image as shown in FIG. 12A. The MR device 1 moves the character C toward the glove image over time as shown in FIG. 12B. The user can visually see the ball moving toward the glove. As shown in FIG. 12B, when the distance between the ball (character C) and the glove image becomes equal to or less than a certain value, the MR device 1 transmits haptic data including frequency and current amplitude value information to the haptic presentation device 2. The haptic presentation device 2 receives the haptic data transmitted from the MR device 1. Based on the information about the current frequency and current amplitude value included in the haptic data, the haptic presentation device 2 passes a rectangular wave current as shown in FIG. 13A through the vibration element to present a vibration sensation.

[0084] As shown in FIG. 12C, the MR device 1 further moves the character C toward the glove image over time. When the distance between the ball, which is character C, and the glove image becomes equal to or less than a predetermined threshold, the MR device 1 switches to slow motion mode and transmits mode setting information indicating the slow motion mode to the tactile presentation device 2. The tactile presentation device 2 receives the mode setting information transmitted from the MR device 1. When the mode setting information indicates switching to slow motion mode, the tactile presentation device 2 changes the frequency and current amplitude value of the current. For example, as shown in FIG. 13B, the frequency is decreased and the current value is increased by a predetermined offset value. More specifically, the control unit 20 multiplies the standard frequency by the frequency correction coefficient and adds the offset value to the current amplitude value. In the example shown in Fig. 13B, the frequency correction coefficient is 0.5. On the other hand, the MR device 1 may cause the character C to move in slow motion.

[0085] By changing the frequency and current amplitude of the current flowing through the vibration element in this way, the user can see the ball moving in slow motion from just before the ball character C reaches the glove image, and can experience the vibration sensation that gives the impression of slow motion caused by the ball moving in slow motion. Specifically, the user can get the sensation of slow motion from the force sensation that is larger by the offset amount and the low-frequency vibration.

[0086] Although the operation in the slow motion mode has been described, the same applies when switching to the quick motion mode. When switched to the quick motion mode, the MR device 1 and the tactile presentation device 2 can present the tactile sensation of the character C with a quick motion feeling by increasing the frequency and decreasing the current amplitude.

[0087] The means for solving the problems of the present disclosure are described below. (Appendix 1) An information processing device including a processing unit that executes a process for presenting a tactile sensation of an object using a tactile presentation device, It has a standard mode and a slow motion mode or a quick motion mode, The processing unit When in a standard mode, presenting a haptic sensation of the object based on haptic sensation data for presenting a haptic sensation of the object that changes at a standard speed motion; When in slow motion mode or fast motion, presenting a haptic sensation of the object based on haptic data for presenting a haptic sensation of the object changing in slow motion or fast motion. Information processing device. (Appendix 2) The tactile sensation of the object changing in slow motion (or fast motion) is greater (or less) than the tactile sensation of the object changing in standard speed motion, at least for a portion of the motion change. 2. The information processing device according to claim 1. (Appendix 3) The tactile data is data for presenting a haptic sensation whose magnitude periodically varies in at least a portion of the motion change; The period of change of the periodically changing haptic sensation of the object that changes in slow motion (or fast motion) is longer (or shorter) than the period of change of the periodically changing haptic sensation of the object that changes in standard speed motion. 10. The information processing device according to claim 1 or 2. (Appendix 4) The processing unit The haptic data relating to the standard speed motion is multiplied by a correction coefficient to generate the haptic data relating to the slow motion or the quick motion. 4. An information processing device according to any one of claims 1 to 3. (Appendix 5) The processing unit Accepts switching between standard mode and slow motion or quick motion mode 5. An information processing device according to any one of claims 1 to 4. (Appendix 6) The processing unit visually presenting the object; If in slow motion or fast motion mode, change the object in slow motion or fast motion. 6. An information processing device according to any one of Supplementary Note 1 to Supplementary Note 5. [Explanation of symbols]

[0088] 1: MR device 2: Tactile presentation device 3: Server device 10: Processing section 11: Storage section 12: First Communications Department 13: Second Communications Department 14: Imaging unit 15: See-through display 16: Eye tracking sensor 17: Audio output section 20: Control section 20a: Control processor 20b:ROM 20d: I / O 21: Tactile data storage unit 22: Communications Department 23: Power supply section 24: MRF device 25: Displacement sensor 30: Server processing section 31: Storage section 32: Server communication section 100: Game System 110: Sensory DB 311: Content DB M1: Recording medium M3:Storage medium N: Network P1: Game app

Claims

1. An information processing device including a processing unit that executes a process for presenting a tactile sensation of an object using a tactile presentation device, It has a standard mode and a slow motion mode or a quick motion mode, The processing unit When in a standard mode, presenting a haptic sensation of the object based on haptic sensation data for presenting a haptic sensation of the object that changes at a standard speed motion; When in the slow motion mode or the fast motion mode, a haptic sensation of the object is presented based on haptic sensation data for presenting a haptic sensation of the object changing in slow motion or fast motion. Information processing device.

2. The tactile sensation of the object changing in slow motion (or fast motion) is greater (or less) than the tactile sensation of the object changing in standard speed motion, at least for a portion of the motion change. The information processing device according to claim 1 .

3. The tactile data is data for presenting a haptic sensation whose magnitude periodically varies in at least a portion of the motion change; The period of change of the periodically changing haptic sensation of the object changing in slow motion (or fast motion) is longer (or shorter) than the period of change of the periodically changing haptic sensation of the object changing in standard speed motion.

3. The information processing device according to claim 1.

4. The processing unit The haptic data relating to the standard speed motion is multiplied by a correction coefficient to generate the haptic data relating to the slow motion or the quick motion.

3. The information processing device according to claim 1.

5. The processing unit Accepts switching between standard mode and slow motion or quick motion mode 3. The information processing device according to claim 1.

6. The processing unit visually presenting the object; If in slow motion or fast motion mode, change the object in slow motion or fast motion.

3. The information processing device according to claim 1.

7. A tactile presentation method for presenting a tactile sensation of an object using a tactile presentation device, comprising: It has a standard mode and a slow motion mode or a quick motion mode, When in a standard mode, presenting a haptic sensation of the object based on haptic sensation data for presenting a haptic sensation of the object that changes at a standard speed motion; When in the slow motion mode or the fast motion mode, a haptic sensation of the object is presented based on haptic sensation data for presenting a haptic sensation of the object changing in slow motion or fast motion. Tactile presentation method.

8. A computer program for causing a computer to execute a process of presenting the tactile sensation of an object using a tactile presentation device, It has a standard mode and a slow motion mode or a quick motion mode, The computer, When in a standard mode, presenting a haptic sensation of the object based on haptic sensation data for presenting a haptic sensation of the object that changes at a standard speed motion; When in the slow motion mode or the fast motion mode, a haptic sensation of the object is presented based on haptic sensation data for presenting a haptic sensation of the object changing in slow motion or fast motion. A computer program for executing a process.

Images