Tactile sensation generating device, tactile sensation generating method and program

Abstract

To provide a tactile sense generating device, a tactile sense generating method, and a program for allowing a user to recognize fluid in a virtual space as one closer to the reality.SOLUTION: A VR device 20 (tactile sense generating device) comprises: a finger information generating unit 263 that acquires attitude information representing the attitude of wearing portions of a wearer of vibration modules 153; a fluid vector information generating unit 264 that acquires fluid direction information representing the direction of flow of fluid in a virtual space; a comparison unit 265 that compares attitude information on the wearing portions and the fluid direction information; and vibration control unit 266 that controls vibration of the vibration modules 153 based on a result of comparison performed by the comparison unit 265.SELECTED DRAWING: Figure 1E

Description

[Technical Field]

[0001] The present invention relates to a tactile sensation generating device, a tactile sensation generating method, and a program. [Background technology]

[0002] With the development of computer technology, systems for humans to experience virtual spaces have become more advanced. For example, Patent Document 1 discloses a system for generating haptic effects related to virtual objects in a virtual environment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-126766 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a tactile sensation generating device, a tactile sensation generating method, and a program that allow a user to perceive fluid in a virtual space as being closer to reality. [Means for solving the problem]

[0005] A tactile sensation generating device according to one aspect of the present invention includes a posture information generating unit that generates posture information representing the posture of the part of the wearer where the vibration element is worn, a fluid direction information acquiring unit that acquires fluid direction information representing the direction of fluid flow in virtual space, a comparison unit that compares the posture information of the part of the wearer with the fluid direction information, and a vibration control unit that controls the vibration of the vibration element based on the results of the comparison made by the comparison unit.

[0006] A tactile sensation generation method according to one aspect of the present invention is one in which a tactile sensation generation device executes the following steps: a posture information generation step for generating posture information representing the posture of the part of the wearer wearing the vibration element; a fluid direction information acquisition step for acquiring fluid direction information representing the direction of fluid flow in a virtual space; a comparison step for comparing the posture information of the part of the wearer with the fluid direction information; and a vibration control step for controlling the vibration of the vibration element based on the comparison result.

[0007] A program according to one aspect of the present invention causes a computer to execute a posture information generation step of generating posture information representing the posture of the part of the body where the vibration element is worn by the wearer, a fluid direction information acquisition step of acquiring fluid direction information representing the direction of fluid flow in a virtual space, a comparison step of comparing the posture information of the part of the body where the vibration element is worn with the fluid direction information, and a vibration control step of controlling the vibration of the vibration element based on the comparison result. [Effects of the Invention]

[0008] The present invention provides a tactile sensation generating device, a tactile sensation generating method, and a program that enable a user to perceive a fluid in a virtual space as being closer to reality. [Brief explanation of the drawings]

[0009] [Figure 1A] 1 is a block diagram showing a configuration example of a VR (Virtual Reality) system according to a first embodiment; [Figure 1B] FIG. 2 is a block diagram showing an example of a fingerstall according to the first embodiment. [Figure 1C] FIG. 2 is a cross-sectional view showing an example of a fingerstall according to the first embodiment. [Figure 1D] 1A and 1B are diagrams illustrating an example of how the fingerstall according to the first embodiment is worn. [Figure 1E] FIG. 2 is a block diagram showing an example of an information processing unit according to the first embodiment. [Figure 1F] FIG. 2 is a diagram showing an example of finger coordinates of a fingerstall according to the first embodiment; [Figure 2A]FIG. 2 is a diagram illustrating the correspondence between the experience of a virtual space and the control of a real space according to the first embodiment. [Figure 2B] 10 is a table showing an example of control of the tactile sensor corresponding to the type of wind according to the first embodiment. [Figure 3A] 4 is a diagram illustrating an example of wind direction and positions of tactile sensors according to the first embodiment. FIG. [Figure 3B] 10 is a table showing an example of control of the tactile sensor in response to wind direction according to the first embodiment. [Figure 3C] FIG. 10 is a diagram illustrating another example of control of the tactile sensor according to the first embodiment. [Figure 4A] 10 is a flowchart illustrating an example of processing by the VR device according to the first embodiment. [Figure 4B] 10 is a flowchart illustrating an example of processing by the VR device according to the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Embodiment 1 Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] 1A is a block diagram showing an example of the configuration of a VR system. The VR system S1 is a system that provides a user with the experience of a virtual space, and includes an imaging unit 11, a depth detection unit 12, a motion detection unit 13, an image display unit 14, a finger cot 15, and a VR device 20. Each component of the VR system S1 will be described below.

[0012] The imaging unit 11 is a camera that captures images of real space, generates captured image data as RGB data, and outputs the captured image data to the VR device 20. The depth detection unit 12 is provided to match the angle of view of the imaging unit 11 as closely as possible, and is a sensor that generates depth data for each pixel of the captured image data and outputs the data to the VR device 20. The depth detection unit 12 may be, for example, an infrared light receiving sensor. The depth detection unit 12 may also be a 3D camera such as a stereo camera or a ToF (Time Of Flight) camera. In this case, the imaging unit 11 and the depth detection unit 12 can be configured as the same 3D camera. As will be described later, this depth data is used to recognize the positional relationship (e.g., distance) between the imaging unit 11 and a part of the human body such as a finger, and to determine the coordinates of the finger in real space.

[0013] The motion detection unit 13 detects the motion state (amount of change in orientation and position) of the imaging unit 11 by comprehensively detecting acceleration, rotational acceleration, gravitational acceleration, and the like accompanying the movement of the imaging unit 11 itself, and outputs the motion state to the VR device 20. The motion detection unit 13 is composed of, for example, an acceleration sensor or a gyro sensor, and may be attached to the imaging unit 11 or the user.

[0014] The image display unit 14 is a display that displays an image of the virtual space generated by the VR device 20. In this example, the imaging unit 11 to the image display unit 14 and the VR device 20 described above are integrally provided as an HMD (Head Mounted Display) that is worn on the user's head, but they may also be provided as other devices.

[0015] The finger cot 15 is a wearable device that wirelessly communicates with the VR device 20 and is attached to each finger of the user. The finger cot 15 controls at least one of vibration and temperature change in conjunction with an event in the virtual space displayed on the image display unit 14, thereby generating a tactile sensation on the user's finger that is related to the event in the virtual space.

[0016] 1B is a block diagram showing an example configuration of fingerstall 15. Fingerstall 15 includes a wireless module 151, a microcontroller 152, a vibration module (vibration element) 153, an electric heating module (electric heating element) 154, and a battery 155. These components are miniaturized so that they can be attached inside the fingerstall body. Each component of fingerstall 15 will be described below.

[0017] The wireless module 151 receives at least one control signal of vibration or temperature change from the VR device 20 by short-distance wireless communication such as Bluetooth (registered trademark).

[0018] Based on the control signal received by the wireless module 151, the microcontroller 152 generates a vibration control signal for the vibration module 153 to control vibration, and a temperature control signal for the electric heating module 154 to control the temperature state, and outputs these to each module.

[0019] Vibration module 153 is configured with a module that vibrates, for example, a small coil using magnetic force. Vibration module 153 vibrates itself in response to a vibration control signal from microcontroller 152, thereby transmitting vibrations to the user's fingers via the finger cot body.

[0020] The electric heating module 154 is composed of an element, such as a Peltier element, whose temperature can be changed by an electric signal. The electric heating module 154 generates heat or cools according to a temperature control signal from the microcontroller 152, thereby transmitting high or low temperature to the user's finger via the finger cot body.

[0021] A battery 155 powers the other components of the finger cot 15.

[0022] FIG. 1C is a cross-sectional view showing an example of a finger stall 15. (a) of FIG. 1C is a cross-sectional view of the finger stall 15 cut vertically along the long side, (b) is a cross-sectional view of the finger stall 15 cut horizontally along the long side, and (c) is a cross-sectional view of the finger stall 15 cut vertically along the short side. Referring to (a) to (c) of FIG. 1C, the finger stall 15 has an electric heating module 154 provided on the inner surface of the main body (i.e., the portion that comes into direct contact with the user's fingertips), and a vibration module 153 provided below that (inside the main body of the finger stall 15). Because the user's fingertips come into direct contact with the electric heating module 154, the user can easily feel temperature changes.

[0023] One vibration module 153 and one electric heating module 154 are provided to constitute one tactile sensor, and three sets of these tactile sensors are provided to surround the pad of the user's finger (i.e., in the circumferential direction when the direction of extension of the finger is the axis of rotation). In this way, the tactile sensor is divided into multiple areas. Also, an IC (Integrated Circuit) including a wireless module 151 and a microcontroller 152, and a battery 155 are provided inside the main body of the fingerstall 15 (i.e., between the inner and outer surfaces).

[0024] FIG. 1D is a diagram showing an example of how fingerstalls 15 are worn. In FIG. 1D, fingerstalls 15a to 15e are worn on the thumb through little finger of the user's right hand H, allowing the user to experience the tactile sensations of events in the virtual space through the sense of touch of the fingers. However, fingerstalls 15 do not have to be worn on all fingers, and may be worn on any one or more fingers.

[0025] Returning to FIG. 1A, the explanation will be continued. The VR device 20 includes a DB 21, a wireless module 22, and a control unit 23. The DB 21 is configured with a storage device such as a flash memory, a memory card, a hard disk drive (HDD), an optical disk drive, etc., but the type of storage device is not limited to these. The DB 21 may also be provided outside the VR device 20. In this case, the VR device 20 may connect to the DB 21 via an information transmitting / receiving unit (not shown) and acquire data stored in the DB 21.

[0026] The DB21 stores image data of the background of the virtual space, and image data and coordinate data of objects (virtual objects) in the virtual space. A virtual object is an object having a 3D shape such as a cube, and its position is specified by coordinate data. The DB21 may also store image data of an avatar (a character representing the user) of the user of the VR device 20. The DB21 also stores settings of the xyz coordinate space in the real space and the virtual space.

[0027] DB21 also stores data regarding the direction (fluid direction) and speed (fluid velocity) of wind flowing through the virtual space. The wind flowing through the virtual space is defined at least in the area of ​​the virtual space where the user can move, and the direction and speed of the wind can change over time. Note that "wind speed" includes not only a constant speed but also wind speeds with varying strength. The amount of variation may include wind speeds with a fixed pattern of strength or wind speeds with irregular fluctuations that approximately reflect wind in real space. By adding irregular fluctuations to the wind speed, the user can experience wind in the virtual space that feels more natural. DB21 may also store the temperature of the virtual space as another environmental setting.

[0028] Furthermore, the DB 21 also stores control settings for the vibration module 153 and the electric heating module 154 according to the type of wind (wind speed) and wind direction, which will be described in detail later.

[0029] The wireless module 22 enables wireless communication with the wireless module 151 of the finger cot 15 via short-range wireless communication, and transmits a control signal of at least one of vibration or temperature change generated by the control unit 23 to the wireless module 151 as tactile data.

[0030] The control unit 23 acquires the captured video data, depth data, and the motion state of the imaging unit 11 from the imaging unit 11, depth detection unit 12, and motion detection unit 13, respectively. Then, using this data and the data related to the virtual space stored in the DB 21, the control unit 23 generates video data representing the virtual space and outputs the video data to the image display unit 14. Furthermore, the control unit 23 generates tactile data representing the feel of an object in the virtual space via the wireless module 22 and outputs the tactile data to the finger cot 15. As described below, the tactile data includes a vibration control signal from the vibration module 153 and a temperature control signal from the electric heating module 154. This will be described in more detail later.

[0031] The control unit 23 includes a memory 24, an I / O (Input / Output) unit 25, and an information processing unit 26. Each unit of the control unit 23 will be described below.

[0032] The memory 24 is configured with a volatile memory, a nonvolatile memory, or a combination thereof. The memory 24 is not limited to one, and multiple memories may be provided. The volatile memory may be, for example, a RAM (Random Access Memory) such as a DRAM (Dynamic Random Access Memory) or an SRAM (Static Random Access Memory). The nonvolatile memory may be, for example, a PROM (Programmable ROM), an EPROM (Erasable Programmable Read Only Memory), or a flash memory.

[0033] The memory 24 is used to store one or more instructions. Here, the one or more instructions are stored in the memory 24 as a group of software modules. The information processing unit 26 can perform the following processes by reading and executing the one or more instructions from the memory 24.

[0034] The I / O unit 25 is a hardware interface that executes input and output of information from and to the outside of the control unit 23. In this embodiment, the control unit 23 is connected to the imaging unit 11, the depth detection unit 12, the motion detection unit 13, the image display unit 14, and the finger cot 15, and appropriately inputs and outputs information to and from these units via the I / O unit 25.

[0035] The information processing unit 26 is configured with any processor, such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), etc. The memory 24 may include a memory provided outside the information processing unit 26, as well as a memory built into the information processing unit 26.

[0036] 1E is a block diagram showing an example configuration of information processing unit 26. Information processing unit 26 reads and executes software (computer programs) from memory 24 to realize functions such as a device information generation unit 261, a virtual object information generation unit 262, a finger information generation unit 263, a fluid vector information generation unit 264, a comparison unit 265, a vibration control unit 266, and a temperature control unit 267. Each of these functions will be described below.

[0037] The device information generation unit 261 generates coordinate and orientation data of a specific location of the VR device 20 using the captured video data, depth data, and the motion state of the imaging unit 11. In this example, the device information generation unit 261 generates coordinate and orientation data of the imaging unit 11 in real space. The device information generation unit 261 determines the position of the imaging unit 11 at the time when the VR device 20 is powered on or the reset button is pressed as the initial position of the coordinates in real space. The device information generation unit 261 also determines the orientation of the imaging unit 11 at the time when the VR device 20 is powered on or the reset button is pressed as the reference orientation (initial orientation) in real space. Thereafter, the device information generation unit 261 updates in real time changes in the position and orientation (orientation) of the imaging unit 11 in the xyz coordinate space of real space tracked by the motion detection unit 13, using the initial position and reference orientation as the initial state, and continues to output the updated information to the virtual object information generation unit 262, the finger information generation unit 263, and the fluid vector information generation unit 264.

[0038] The virtual object information generation unit 262 is composed of a GPU (Graphics Processing Unit) and the like. Using the image data and coordinate data of the virtual object stored in the DB 21, as well as settings of the xyz coordinate space in the virtual space, the virtual object information generation unit 262 can place a virtual object at any position in the virtual space. Using the background image data of the virtual space stored in the DB 21, the virtual object information generation unit 262 generates a 2D image of the virtual space showing the state in which the virtual object is placed, by a rendering method using a program stored in the memory 24. At this time, the virtual object information generation unit 262 updates and generates the 2D image of the virtual space so that the visual orientation of the virtual object changes in accordance with changes in the position and orientation of the imaging unit 11 in the xyz coordinate space of the real space. The virtual object information generation unit 262 outputs the updated image to the image display unit 14 for display. Each vertex of the virtual object (i.e., a point specifying the boundary of the virtual object) is expressed as a coordinate in the xyz coordinate space in the virtual space, and is output to the image display unit 14 as coordinate data of the virtual object.

[0039] The finger information generation unit 263 (posture information generation unit) uses the captured video data and depth data described above to recognize the presence of the user's finger (and the fingerstall 15) shown in the captured video data. Then, taking into account the posture of the imaging unit 11, it compares the relative relationship between the position of the imaging unit 11 and the position of the finger to generate finger coordinate data in the xyz coordinate space of the real space. The generated finger coordinate data of the fingerstall 15 is updated in real time.

[0040] Furthermore, the finger information generation unit 263 acquires the posture (orientation) of the fingerstall 15 as posture information. The finger information generation unit 263 can detect the posture of the fingerstall 15, for example, by an infrared receiving sensor (not shown) provided in the VR device 20. However, the method of acquiring the posture of the fingerstall 15 is not limited to this. For example, a movement detection sensor may be provided in the fingerstall 15, and the posture (and position) of the fingerstall 15 may be detected based on the detection results of the sensor. As another example, the finger information generation unit 263 may analyze captured video data to detect the posture of the fingerstall 15 captured in the video. The detected posture of the fingerstall 15 is updated in real time.

[0041] FIG. 1F is a diagram showing an example of finger coordinates of the fingerstall 15. FIG. 1F shows an enlarged cross-section of the roughly circular fingerstall 15 shown in FIG. 1C(c). Three sets of tactile sensors are provided: an area L on the left side of the finger, an area F in the center of the finger, and an area R on the right side of the finger. The finger information generation unit 263 can generate, for example, the coordinates of the center O of the cross-section of the fingerstall 15 shown in FIG. 1F as finger coordinates in real space, and generate the orientation of the center O as the orientation of the fingerstall 15 in real space.

[0042] The finger information generation unit 263 generates information about finger coordinates and posture in the xyz coordinate space in the virtual space using the data of these fingers in the real space and the settings of the xyz coordinate space in the virtual space stored in the DB 21. In this way, the finger information generation unit 263 generates data about the finger coordinates and posture of the fingers wearing the fingerstalls 15 as finger information, and outputs the information for each fingerstall 15 to the comparison unit 265. By outputting updated finger information data from the finger information generation unit 263 in accordance with the movement in the real space, the fingers in the virtual space can move in synchronization with the movement in the real space. Note that the finger information generation unit 263 may generate hand information by recognizing the entire hand rather than individual fingers.

[0043] The finger information generation unit 263 can also generate finger coordinate data and orientation of the tactile sensors associated with the fingerstall 15 for each finger. The positions and orientations of the tactile sensors provided in each of the regions L to R relative to the center O are fixed, and this information is stored in the DB 21. For example, as shown in FIG. 1F, assuming a horizontal line V that is parallel to the ground and passes through the center O, the positions and orientations of the tactile sensors provided in each of the regions L to R relative to the center O can be uniquely defined by the angles θ1 to θ3 between each of the regions L to R and the horizontal line V, and the distance r between the center O and the main body of the fingerstall 15. Therefore, the finger information generation unit 263 can generate finger coordinate data and orientations of the tactile sensors shown in each of the regions L to R using the finger coordinate and orientation data relative to the center O and the position and orientation data of the tactile sensors shown in each of the regions L to R relative to the center O. The process of using the finger coordinate data and orientations of the tactile sensors thus determined will be described later.

[0044] Furthermore, when a finger is captured by the imaging unit 11, the finger information generation unit 263 can generate an image of the finger in the virtual space using the finger data of the avatar stored in the DB 21, and output and display the image to the image display unit 14. This allows the user to recognize their own finger in the virtual space.

[0045] The fluid vector information generation unit 264 (fluid direction information acquisition unit) acquires wind direction information (fluid direction information) and wind speed information (fluid velocity information) of the wind flowing through the virtual space stored in DB21 as wind speed vector information (hereinafter referred to as wind vector (fluid vector)). This wind vector is updated over time based on the information stored in DB21. The fluid vector information generation unit 264 outputs the acquired wind vector information to the comparison unit 265.

[0046] The amount of change in the strength of the wind speed in the wind vector may be set as the environmental data of the virtual space in the DB 21 as described above, but may also be set by the fluid vector information generating unit 264.

[0047] The comparison unit 265 acquires and compares the finger information and wind vector information of each finger cot 15 to detect in virtual space which location on the finger cot 15 the wind is blowing, in what direction, and at what speed. In particular, the comparison unit 265 compares the finger posture information with the wind direction information to detect the direction in which the wind is blowing on each set of tactile sensors. The comparison unit 265 outputs this information on the wind blowing on each set to the vibration control unit 266 and the temperature control unit 267.

[0048] The vibration control unit 266 generates a vibration control signal for controlling the vibration of each vibration module 153 of each finger stall 15, based on the information about the wind blowing on each set output from the comparison unit 265. In particular, the vibration control unit 266 controls the vibration of each vibration module 153 based on the result of comparing the finger posture information and wind direction information in the comparison unit 265. Then, the vibration control unit 266 outputs a vibration control signal for each vibration module 153 of each finger stall 15 to the wireless module 22.

[0049] Temperature control unit 267 generates a temperature control signal for controlling the temperature of each electric heating module 154 of each finger cot 15, based on the information about the wind blowing through each set output from comparison unit 265. Temperature control unit 267 then transmits the temperature control signal for each electric heating module 154 of each finger cot 15 to wireless module 22.

[0050] The wireless module 22 transmits tactile data including a vibration control signal and a temperature control signal to the wireless module 151 of each finger cot 15. As described above, the microcontroller 152 of each finger cot 15 controls the vibration module 153 and the electric heating module 154 based on the control signals received by the wireless module 151.

[0051] Next, an example of control of the vibration module 153 and the electric / thermal module 154 determined by the vibration control unit 266 and the temperature control unit 267 based on the state of the wind blowing in the virtual space and the position and posture of the finger will be described. Below, the aspects of wind speed and wind direction will be described respectively.

[0052] 2A is a diagram comparing the state of wind blowing in virtual space with the control of the fingerstall 15 in real space. In order to allow the user to experience a state in which wind W is blowing in the x direction from the left onto the user's right hand H in virtual space, the vibration modules 153 and electric heating modules 154 of the fingerstall 15 attached to the user's right hand H are controlled in real space. Qualitatively, at the location of the fingers where the wind is blowing in the virtual space, the vibration module 153 vibrates and the electric heating module 154 is cooled.

[0053] The wind W can also be handled as a virtual object, and may be displayed in a 2D image in the virtual space as information visualized by the virtual object information generation unit 262. However, the wind may not be visualized by the virtual object information generation unit 262, but may be displayed as a background in the virtual space or as a visual representation related to the user's avatar.

[0054] 2B is an example of control settings stored in DB21, a table showing an example of tactile sensor control corresponding to wind type (wind speed). Here, four types of wind are assumed: (1) gentle breeze (less than 3 m / s), (2) moderately strong wind (3 m / s or more but less than 7 m / s), (3) strong wind (7 m / s or more but less than 10 m / s), and (4) strong wind (10 m / s or more). However, the number of wind speed types and how they are classified are not limited to this example. Furthermore, the parameters shown below, namely, the amplitude (vibration strength) and vibration frequency of vibration module 153, and the temperature drop from room temperature of electric heating module 154, are merely examples and can be adjusted as appropriate depending on the actual characteristics of vibration module 153 and electric heating module 154, as well as the environmental settings of the virtual space.

[0055] (1) In the case of a gentle breeze (less than 3 m / s), the amplitude of the vibration module 153 is set to be minute, the vibration frequency to be medium, and the temperature drop from room temperature of the electric heating module 154 to be between 0°C and 3°C. A user wearing the finger cot 15 can experience the sensation of a gentle breeze touching their fingers in a virtual space by experiencing the vibration on their sensitive skin and a slight drop in perceived temperature.

[0056] (2) In the case of a moderately strong wind (3 m / s or more and less than 7 m / s), the amplitude of vibration module 153 is set to be small, the vibration frequency to be medium, and the temperature drop from room temperature of electric heating module 154 to be 3°C or more and less than 7°C. Compared to (1), the amplitude of vibration module 153 is set to be larger and electric heating module 154 to be at a lower temperature. Therefore, a user wearing finger cot 15 can experience stronger vibrations on the skin and a drop in perceived temperature, thereby experiencing the sensation of a moderately strong wind blowing against their fingers in virtual space.

[0057] (3) In the case of strong winds (7 m / s or more and less than 10 m / s), the amplitude of vibration module 153 is set to medium, the vibration frequency is set to high, and the temperature drop from room temperature of electric heating module 154 is set to 7°C or more and less than 10°C. Compared to (2), the amplitude of vibration module 153 is set to be even larger, the vibration frequency is set to be higher, and the temperature of electric heating module 154 is set to be even lower. Therefore, a user wearing finger cot 15 can experience even stronger and suddener vibrations on the skin and a larger drop in perceived temperature, thereby experiencing the sensation of a moderately strong wind blowing against their fingers in virtual space.

[0058] (4) In the case of strong winds (10 m / s or more), the amplitude of vibration module 153 is set to be large, the vibration frequency is set to be high, and the temperature drop from room temperature of electric heating module 154 is set to be 10°C or more. Compared to (3), the amplitude of vibration module 153 is set to be even larger, and electric heating module 154 is set to be even lower. Therefore, a user wearing finger cot 15 can experience even stronger vibrations on the skin and a further drop in perceived temperature, and can experience the sensation of strong wind blowing against their fingers in virtual space.

[0059] In (1) to (4), the temperature drop from room temperature of the electric heating module 154 can be any function as long as it monotonically increases with increasing wind speed. For example, when the wind speed is Nm / s, the temperature drop from room temperature of the electric heating module 154 can be N°C. The amplitude and vibration frequency of the vibration module 153 can also be any function as long as they monotonically increase with increasing wind speed.

[0060] Furthermore, the user can experience a more realistic wind in the virtual space by setting the amplitude of the vibration module 153 to have the irregular fluctuations described above. However, similar fluctuations can also be added to the vibration frequency of the vibration module 153 and the value of the temperature drop from room temperature of the electric heating module 154.

[0061] FIG. 3A is a block diagram showing an example of the direction of wind blowing in virtual space and the position of a tactile sensor. In FIG. 3A, a cross-sectional view of the finger cot 15 shown in FIG. 1C(c) is shown, with wind blowing on the finger in virtual space superimposed on it. The explanation of areas L, F, and R is as shown in FIG. 1F. The wind blowing on the finger is hypothetically shown as wind WF from the pad (front) of the finger, wind WR from the right side of the finger as viewed from FIG. 3A, wind WL from the left side of the finger as viewed from FIG. 3A, and wind WB from the nail (back) of the finger. However, the wind direction is not limited to this example.

[0062] FIG. 3B is an example of control settings stored in DB21, a table showing an example of tactile sensor control corresponding to wind direction. FIG. 3B shows how the tactile sensor is controlled as shown in FIG. 2B according to wind speed, and how the degree of control changes. However, the control settings according to wind speed may be any setting other than that shown in FIG. 2B. The three operation levels "strong," "weak," and "off" (non-operating state) in FIG. 3B indicate the magnitude of a coefficient multiplied by the control details of FIG. 2B (the amplitude and vibration frequency of vibration module 153, and the temperature drop from room temperature of electric heating module 154). For example, "strong" is 1, "weak" is 0.5, and "off" is 0; however, these coefficient values ​​are merely examples.

[0063] When the wind direction is WL, the operation level of the tactile sensors in area L is set to "strong," the operation level of the tactile sensors in area F is set to "weak," and the operation level of the tactile sensors in area R is set to "off." This is because, as shown in Figure 3A, the wind WL blows strongly against the tactile sensors in area L, but weakly against the tactile sensors in area F, and barely at all against the tactile sensors in area R.

[0064] When the wind direction is WR, the operation level of the tactile sensors in area L is set to "off," the operation level of the tactile sensors in area F is set to "weak," and the operation level of the tactile sensors in area R is set to "strong." This is because, as shown in Figure 3A, the wind WR blows strongly against the tactile sensors in area R, but weakly against the tactile sensors in area F, and barely at all against the tactile sensors in area L.

[0065] When the wind direction is WF, the operation level of the tactile sensors in area L is set to "weak," the operation level of the tactile sensors in area F is set to "strong," and the operation level of the tactile sensors in area R is set to "weak." This is because, as shown in Figure 3A, the wind WF blows strongly against the tactile sensors in area F, but weakly against the tactile sensors in areas L and R.

[0066] When the wind direction is WB, the operation level of the tactile sensors in area L is set to "weak," the operation level of the tactile sensors in area F is set to "off," and the operation level of the tactile sensors in area R is set to "weak." This is because, as shown in Figure 3A, the wind WB blows weakly onto the tactile sensors in areas L and R, but barely onto the tactile sensors in area F.

[0067] When the vibration control unit 266 and the temperature control unit 267 receive information output from the comparison unit 265 (information indicating the direction and speed of the wind blowing through each set of tactile sensors), they refer to the control settings shown in FIGS. 2B and 3B. As described above, FIGS. 2B and 3B set the operating tactile sensors (modules) and their operating levels. Based on these settings, the vibration control unit 266 and the temperature control unit 267 generate vibration control signals and temperature control signals, respectively, for setting the amplitude and vibration frequency of the vibration module 153 and the temperature drop from room temperature of the electric heating module 154 so as to reproduce the speed and direction of the wind blowing through each set of tactile sensors in the virtual space. In this way, the vibration control unit 266 and the temperature control unit 267 allow the user to experience the wind speed and direction set in the virtual space.

[0068] The temperature control unit 267 may further change the temperature of the electric heating module 154 according to the "air temperature of the virtual space" in the environmental settings of the virtual space stored in the DB 21. For example, if the air temperature of the virtual space is equal to or lower than a predetermined threshold, the temperature of the electric heating module 154 may be set to be further lowered by a predetermined value relative to the temperature set by the control settings shown in FIGS. 2B and 3B described above. In this case, the temperature control unit 267 may further lower the temperature of the electric heating module 154 from room temperature so that the temperature decrease with increasing wind speed becomes a monotonically increasing function. Furthermore, if the air temperature of the virtual space is equal to or higher than a predetermined threshold (e.g., close to or higher than body temperature), the temperature control unit 267 may increase the temperature of the electric heating module 154 from room temperature by a predetermined value. In this case, the temperature control unit 267 may increase the temperature of the electric heating module 154 from room temperature so that the temperature increase with increasing wind speed becomes a monotonically increasing function. In this way, the temperature control unit 267 can also express hot air being blown toward the user in the virtual space. The threshold values ​​and functions relating to the temperature increase or decrease are stored in DB 21, and temperature control unit 267 can use the stored information as appropriate.

[0069] The method for controlling and setting the operating level of the tactile sensors in each area is not limited to the method shown in FIGS. 3A and 3B.

[0070] This method will be explained below with reference to Figure 3C. Figure 3C shows the configuration of fingerstall 15 similar to that shown in Figure 1F. Figure 3C also shows wind W blowing in a direction W toward the upper left of the figure.

[0071] First, in the coordinate system of the virtual space, lines SR, SF, and SL are defined, connecting the center O and the tactile sensors in each of the areas R, F, and L. Next, a line E is defined, with respect to wind W, pointing toward the center O. The angles between line SR and line E, the angle between line SF and line E, and the angle between line SL and line E are defined as θR, θF, and θL, respectively (these angles are also collectively referred to as angle θ). FIG. 3C shows the lines SR, SF, SL, and E, and the angles θR, θF, and θL. Note that the angles θR, θF, and θL are defined as values ​​between 0° and 180°. The angles θR, θF, and θL can be derived by generating the finger coordinate data and posture of the tactile sensors shown in each of the areas L to R.

[0072] After determining the angles θR, θF, and θL in the above manner, the vibration control unit 266 compares the absolute values ​​of each angle and controls the vibration module 153 in the area corresponding to the smallest angle to vibrate at the strongest operating level. In the example shown in FIG. 3C, when the absolute values ​​of the angles θR, θF, and θL are compared, |θR|<|θF|<|θL| (1) The magnitude relationship between the above values ​​is established. Therefore, the vibration control unit 266 vibrates the vibration module 153 in the region R corresponding to θR at the strongest operation level. The magnitude relationship between the operation levels of the vibration modules 153 in each region is the opposite of the magnitude relationship between the regions shown in (1). That is, the operation levels are highest in the region R, followed by the region F and the region L. Instead of (1), the operation levels may be set based on the following magnitude relationship. cos|θR| <cos|θF|<cos|θL|···(2)

[0073] When the angle θ is 0°, wind blows directly onto the vibration modules 153 in the corresponding area, resulting in a maximum operation level. On the other hand, when the angle θ is between 90° and 180°, wind barely blows onto the vibration modules 153 in the corresponding area, resulting in a minimum operation level (for example, 0). The operation level can take any value as long as it satisfies the magnitude relationship described above. For example, if the operation level is 1 when the angle θ is 0°, the vibration control unit 266 may set an operation level proportional to cosθ when the angle θ is between 0° and 90°, and may set the operation level to 0 when the angle θ is between 90° and 180°.

[0074] The above has described the setting of the operation level of the vibration control unit 266, but it is also possible to set an operation level in a similar manner for the temperature control unit 267. Based on this setting, the vibration control unit 266 and the temperature control unit 267 generate a vibration control signal and a temperature control signal, respectively, for setting the amplitude and vibration frequency of the vibration module 153 and the temperature drop from room temperature of the electric heating module 154 so as to reproduce the wind speed and direction of the wind blowing in the virtual space for each set of tactile sensors.

[0075] In the example described above, the vibration control unit 266 and the temperature control unit 267 control the vibration and temperature of the tactile sensors in all areas of the fingerstall 15, but the areas to be controlled may be one or more arbitrary areas of the fingerstall 15.

[0076] Furthermore, the vibration control unit 266 may shift the timing at which the control shown in FIGS. 3B and 3C is started for each finger stall 15 depending on the wind speed information (fluid speed information) of the blowing wind, wind direction information (fluid direction information), and the positions of the multiple spaced apart finger stalls 15.

[0077] For example, assuming the state shown in FIG. 2A, the wind W blows on the right hand starting from the thumb and ending with the little finger. In FIG. 2A, the fingerstall 15a for the thumb of the right hand H and the fingerstall 15e for the little finger, which is separated from the fingerstall 15a, are separated by a distance d on the x-axis, and the wind speed of the wind W is v. The comparison unit 265 (acquisition unit) acquires the finger information and wind vector information for each fingerstall 15 from the fluid vector information generation unit 264. The comparison unit 265 outputs information about the wind blowing on each set and the distance d between the fingerstalls 15a and 15e (i.e., the distance from upwind to downwind) to the vibration control unit 266.

[0078] Based on the output information, the vibration control unit 266 starts the operation of each module in the fingerstall 15a, and then, after a time d / v has elapsed, starts the operation of each module in the fingerstall 15e. The operation of each module is as shown in FIGS. 2B and 3B. The timing of the start of operation of each module in the fingerstall 15b-d, as well as the fingerstall 15e, can be delayed from the start of operation of each module in the fingerstall 15a by a time difference corresponding to the distance from the corresponding fingerstall 15a and the wind speed v. Even when the wind W stops, the vibration control unit 266 can similarly control the timing of the stop of operation of each module in the fingerstall 15b-e from the stop of operation of each module in the fingerstall 15a by a time difference corresponding to the distance from the corresponding fingerstall 15a and the wind speed v.

[0079] In this way, the vibration control unit 266 can cause each module to operate with a time lag, so that the timing of starting or stopping operation in response to the wind is earlier for the fingerstall 15 that is the windward device, and the timing of starting or stopping operation in response to the wind is later for the fingerstall 15 that is the downwind device. Therefore, the user can perceive that the wind W has started to blow from the left direction in FIG. 2A in the virtual space, and can experience the wind W with a more realistic feeling.

[0080] Furthermore, the vibration control unit 266 and the temperature control unit 267 may shift the timing of starting the control shown in FIGS. 3B and 3C for each area unit of the fingerstall 15, depending on the wind speed information (fluid speed information) of the blowing wind, wind direction information (fluid direction information), and each position of the area of ​​the fingerstall 15.

[0081] For example, in the control shown in FIG. 3B, the tactile sensors in the areas where the operation level is "strong" are the first to start operating (i.e., vibration and temperature change are started) in each wind direction, followed by the tactile sensors in the areas where the operation level is "weak." The offset in timing at which the tactile sensors operate can be set by any method. For example, because the distance between the tactile sensors in each finger cot 15 is fixed, the vibration control unit 266 calculates the offset in timing by dividing the value stored in DB21 by the wind speed. The vibration control unit 266 shifts the operation timing of the tactile sensors in the areas where the operation level is "strong" and the tactile sensors in the areas where the operation level is "weak" by the calculated value. The temperature control unit 267 can also perform the process of shifting the operation timing in the same way as the vibration control unit 266.

[0082] However, the method of shifting the timing of starting control in each region is not limited to this method. Referring again to FIG. 3C , this method will be described. As described above, the vibration control unit 266 controls the operation levels of the vibration module 153 so that they increase in the order of region R, region F, and region L. The operation start timing is also controlled so that it is earlier in the order of region R, region F, and region L. Furthermore, if the angle θ is between 0° and 90°, the vibration control unit 266 sets the operation start timing to be earlier as the value of cosθ increases (as the absolute value of θ decreases). For example, in FIG. 3C , the vibration control unit 266 may set the operation start timing to be earliest when the angle θ is 0°, and start the operation of the vibration module 153 in region R at a timing delayed by r*sinθR / v (v is the wind speed of the wind W) from that timing. The vibration modules 153 in regions F and L also start operation at timings based on a similar calculation. Furthermore, when the angle θ is greater than 90° and equal to or less than 180°, the operation level can be set to 0 and the device can be set not to operate, as described above. The temperature control unit 267 can also perform processing to shift the operation timing in the same manner as the vibration control unit 266.

[0083] The above-described control of the operation start or stop timing can also be performed by the temperature control unit 267 instead of or in addition to the vibration control unit 266. In other words, the operation timing may be changed for either or both of the vibration module 153 and the electric heating module 154 in the fingerstall 15. For example, only the vibration timing of the vibration module 153 may be changed as described above. The vibration control unit 266 may also set the time difference for the operation start or stop of each fingerstall 15 to a value corresponding to at least one of the distance between the fingerstalls 15 and the wind speed v (for example, proportional to at least one of the distance and the wind speed v).

[0084] Furthermore, the direction of the wind experienced by the user can be changed according to the movement of the user's hand. When the user moves their hand, as described above, the finger information generation unit 263 updates and outputs finger information data in accordance with the movement in real space. This allows the vibration control unit 266 and the temperature control unit 267 to understand that the positional relationship of each finger stall 15 with respect to the wind has changed. Therefore, the above-described control can be performed according to the new positional relationship of each finger stall 15.

[0085] Furthermore, the finger information generation unit 263 can also acquire the velocity vector of each finger stall 15. As described above, the velocity vector is acquired by an infrared receiving sensor, a motion detection sensor (not shown), or analysis of captured video data provided in the VR device 20. The finger information generation unit 263 outputs information about the velocity vector of each finger to the comparison unit 265. The comparison unit 265 detects in the virtual space which location on the finger stall 15 the wind is blowing, in what direction, and at what speed, based on the finger information, wind vector information, and velocity vector information of each finger stall 15. At this time, if the velocity vector of a certain finger stall 15 has a component opposite to the wind vector blowing on that finger stall 15 (i.e., a canceling direction), the comparison unit 265 sets the wind speed blowing on the finger stall 15 so that the wind speed blowing on that finger stall 15 is increased by the amount of that component. On the other hand, when the velocity vector of a certain fingerstall 15 has the same component as the wind vector (i.e., is in an increasing direction), the comparison unit 265 sets the wind speed blowing on the fingerstall 15 so that the wind speed blowing on that fingerstall 15 decreases by the amount of that component. In other words, when the user moves their hand, the comparison unit 265 can reflect the relative speed of the finger with respect to the wind in the wind experienced by the user in the virtual space.

[0086] 2B and 3B based on the information on the wind speed determined as described above, the vibration control unit 266 and the temperature control unit 267 perform the control shown in Fig. 2B and 3B. This makes it possible to present to the user the perceived temperature of the wind that takes into account the relative speed of the fingers with respect to the wind.

[0087] The above-described control of the timing of starting or stopping operation may be similarly performed for tactile sensors provided at different positions on fingerstall 15, using information on the positional relationship between the tactile sensors in terms of wind direction and wind speed.

[0088] 4A and 4B are flowcharts showing an example of processing executed by the VR device 20. Below, the processing of the VR device 20 will be described with reference to Figs. 4A and 4B. Details of each process are as described above, and explanations will be omitted where appropriate.

[0089] First, the user turns on the power to the VR device 20 (step S11). In response to this, the control unit 23 resets the settings of the xyz coordinate space in the real space and the virtual space handled inside the VR device 20, which are stored in the DB 21, and prepares to start the subsequent processing (step S12).

[0090] Next, the finger information generation unit 263 determines whether or not the user's finger can be recognized based on the data generated by the device information generation unit 261 (step S13). If the user's finger cannot be recognized (No in step S13), the finger information generation unit 263 returns to step S13 and starts the process over again. If the user's finger can be recognized (Yes in step S13), the finger information generation unit 263 generates finger coordinate data in the xyz coordinate space of the real space and detects the posture (orientation) of the fingerstall 15. Then, the finger information generation unit 263 generates finger coordinate and posture data in the xyz coordinate space in the virtual space using the data of the fingers in the real space and the settings of the xyz coordinate space in the virtual space stored in the DB 21 (step S14).

[0091] The fluid vector information generation unit 264 refers to the data stored in DB21 and determines whether wind is flowing in the virtual space where the user is located (whether it is necessary to acquire wind vector information) (step S15). If wind is not flowing in the virtual space (No in step S15), the fluid vector information generation unit 264 returns to step S15 and starts the process over. If wind is flowing in the virtual space (Yes in step S15), the fluid vector information generation unit 264 acquires wind vector information for the virtual space (step S16).

[0092] Note that the processing of steps S13 to S14 and the processing of steps S15 to S16 may be executed first, or both may be executed in parallel. Furthermore, the virtual object information generation unit 262 may execute, at any timing, a process of placing a virtual object at an arbitrary position in the virtual space and outputting and displaying a 2D image of the virtual space including the virtual object to the image display unit 14.

[0093] The comparison unit 265 acquires and compares the finger information and wind vector information of each fingerstall 15, thereby detecting the direction and speed of the wind hitting the tactile sensor of each fingerstall 15 in the virtual space (step S17).

[0094] Then, vibration control unit 266 and temperature control unit 267 generate a vibration control signal and a temperature control signal, respectively, based on the information about the wind blowing through each set (comparison result) output from comparison unit 265, and transmit these to each finger stall 15 via wireless module 22. This controls the vibration and temperature of the tactile sensor associated with each finger stall 15 (step S18).

[0095] As described above, in the VR device 20 (tactile sensation generating device), the finger information generating unit 263 (posture information generating unit) generates posture information that represents the posture of the user's finger (attachment part) of the vibration module 153, and the fluid vector information generating unit 264 (fluid direction information acquiring unit) acquires wind direction information (fluid direction information) that represents the direction of wind flow in the virtual space. Then, the comparing unit 265 compares the finger posture information with the wind direction information, and the vibration control unit 266 controls the vibration of the vibration module 153 based on the result of the comparison by the comparing unit. This allows the user to perceive the wind in the virtual space as being closer to reality.

[0096] Furthermore, the fluid vector information generating unit 264 acquires wind speed information (fluid velocity information) that indicates the speed of wind flow in the virtual space, and the vibration control unit 266 can vibrate the first vibration module 153 (first vibration element) and then vibrate the second vibration module 153 (second vibration element) that is spaced apart from the first vibration module 153 with a time difference according to the wind speed information. Therefore, the VR device 20 can allow the user to experience, in the virtual space, that the wind is blowing faster on the upwind finger than on the downwind finger.

[0097] The present invention is not limited to the above-described embodiment, and modifications can be made as appropriate without departing from the spirit and scope of the present invention. For example, the wearable device may be a device that can be worn by wrapping it around any part of the body, such as the hand, arm, neck, torso, or foot, rather than the finger cot 15. In this case, similar to the finger cot 15, multiple tactile sensors each consisting of a vibration module 153 and an electric heating module 154 are provided in the circumferential direction of the device.

[0098] Instead of multiple vibration modules 153 and electric heating modules 154, only one set may be provided in the finger cot 15 (or other wearable device).

[0099] In the first embodiment, we described the generation of a tactile sensation experienced by a user when wind (gas) blows against a user's finger in a virtual space. However, the device according to the present invention is not limited to this. It can also generate a tactile sensation experienced when a user touches a liquid such as water. For liquids, the fluid vector information generator 264 acquires fluid direction information representing the direction of the fluid flowing through the virtual space and fluid velocity information representing the speed of the fluid flow, stored in the DB 21, as velocity vector (fluid vector) information for that flow in the virtual space. In this case, the comparator 265 first determines whether the fluid is in contact with the finger cot 15. If the fluid is in contact, the comparator 265 detects the direction and velocity of the fluid hitting each set of tactile sensors based on the finger posture. This allows the vibration controller 266 and the temperature controller 267 to perform control processing to provide the user with a tactile sensation corresponding to the fluid's velocity vector.

[0100] DB21 may also store data related to the viscosity of the liquid. Vibration control unit 266 may use this viscosity data to further adjust the vibration of vibration module 153 so that the user can experience the tactile sensation of the viscosity of the fluid. Furthermore, temperature control unit 267 may further adjust the temperature of electric heating module 154 based on the temperature setting of the liquid itself, similar to the control of the air temperature in the virtual space described in embodiment 1, so that the user can experience the temperature of the liquid. In this way, the device according to the present invention can generate the tactile sensation of any fluid, including gas and liquid.

[0101] As described above, one or more processors included in the VR system in the above-mentioned embodiments execute one or more programs including instructions for causing a computer to execute the algorithms described using the drawings. This processing enables the processing described in each embodiment to be realized.

[0102] The program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disk (DVD), Blu-ray disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals. The program may also take the form of, for example, an application program.

[0103] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0104] S1 VR System 11 imaging unit 12 depth detection unit 13 Motion detection unit 14 Image display unit 15 Finger Cots 151 Wireless Module 152 Microcontroller 153 Vibration module 154 Electric heating module 155 Battery 20 VR equipment 21 DB 22 Wireless Module 23 Control unit 24 Memory 25 I / O section 26 Information processing section 261 Device information generation unit 262 Virtual object information generation unit 263 Finger information generation unit 264 Fluid vector information generation unit 265 Comparison unit 266 Vibration control unit 267 Temperature control unit

Claims

1. a posture information generating unit that generates posture information representing a posture of a portion of a wearer where the vibration element is worn; a fluid direction information acquisition unit that acquires fluid direction information that indicates a direction of a fluid flow in a virtual space and fluid velocity information that indicates a speed of the fluid flow in the virtual space; a comparison unit that compares the posture information of the attachment portion with the fluid direction information; a vibration control unit that controls vibration of the vibration element based on the result of the comparison by the comparison unit; Equipped with the posture information generation unit acquires a velocity vector of the vibration element; The comparison unit compares the fluid direction information, the fluid velocity information, and the velocity vector of the vibration element. Tactile sensation generation device.

2. The vibration control unit vibrates a first vibration element, and then vibrates a second vibration element spaced apart from the first vibration element with a time difference corresponding to the fluid velocity information. The tactile sensation generating device according to claim 1 .

3. a posture information generating step of generating posture information representing a posture of a portion of a wearer wearing the vibration element; a fluid direction information acquisition step of acquiring fluid direction information representing a direction of a fluid flow in a virtual space and fluid velocity information representing a speed of the fluid flow in the virtual space; obtaining a velocity vector of the vibration element; a comparison step of comparing the posture information of the attachment portion, the fluid direction information, the fluid velocity information, and the velocity vector of the vibration element; a vibration control step of controlling vibration of the vibration element based on the comparison result; The tactile sensation generating method is executed by the tactile sensation generating device.

4. a posture information generating step of generating posture information representing a posture of a portion of a wearer wearing the vibration element; a fluid direction information acquisition step of acquiring fluid direction information representing a direction of a fluid flow in a virtual space and fluid velocity information representing a speed of the fluid flow in the virtual space; obtaining a velocity vector of the vibration element; a comparison step of comparing the posture information of the attachment portion, the fluid direction information, the fluid velocity information, and the velocity vector of the vibration element; a vibration control step of controlling vibration of the vibration element based on the comparison result; A program that causes a computer to execute the following.

Images