Information processing system, information processing method, and information processing program

Abstract

An information processing system according to the present invention comprises at least a user terminal and at least one control unit, each user terminal comprising a display unit capable of displaying at least one first object and at least one tactile sensation presentation unit that presents a tactile sensation to a user, wherein the control unit is configured to execute: a step for causing the display unit to display a state of the first object deforming in response to manipulation of the at least one tactile sensation presentation device; a step for calculating a virtual resistance force exerted by the deformed first object; and a step for causing the tactile sensation presentation unit to present to the user a tactile sensation that corresponds to the calculated resistance force.

Description

Information processing system, information processing method, and information processing program

[0001] The present invention relates to an information processing system, an information processing method, and an information processing program.

[0002] Patent Literature 1 discloses an information processing device having a controller used in virtual reality technology. This controller is configured to apply a virtual force to a virtual object by pressing a button, and is configured to change the resistance force from the virtual object based on the number of times the button is pressed.

[0003] Patent No. 7035090

[0004] However, in the above-mentioned information processing device, the resistance force is calculated based on a virtual external force applied to the object, and the virtual characteristics of the object itself are not taken into consideration. The present invention has been made to solve this problem, and an object of the present invention is to provide an information processing system, an information processing method, and an information processing program that can present a more realistic resistance force by taking into consideration the characteristics of the object itself.

[0005] Item 1. An information processing system comprising: at least one user terminal; and at least one control unit, wherein each of the user terminals comprises: a display unit capable of displaying at least one first object; and at least one tactile sense providing unit that provides a tactile sensation to the user, and wherein the control unit is configured to execute the steps of: causing the display unit to display a state in which the first object is deforming in response to an operation of the at least one tactile sense providing unit; calculating a virtual internal pressure of the deformed first object; and causing the tactile sense providing unit to provide a tactile sensation corresponding to the calculated internal pressure to the user.

[0006] Item 2. The information processing system according to Item 1, wherein the tactile sense providing unit includes at least a first tactile sense providing unit and a second tactile sense providing unit.

[0007] Item 3. The information processing system according to Item 2, wherein the first tactile sense presentation unit and the second tactile sense presentation unit are operated by a first user and a second user, respectively.

[0008] Item 4. The information processing system according to Item 3, wherein the control unit is configured to execute the steps of: causing the display unit to display how the first object is deforming in response to operation of the first tactile sense presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the first user; causing the display unit to display how the first object is deforming in response to operation of the second tactile sense presentation unit; causing at least the second tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the second user; and calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile sense presentation units.

[0009] Item 5. The information processing system according to Item 3, wherein the control unit is configured to execute the steps of: causing the display unit to display how the first object is deforming in response to operation of the first tactile sense presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the first user; causing the display unit to display how the first object is further deforming in response to operation of the second tactile sense presentation unit; calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile sense presentation units; and causing the first and second tactile sense presentation units to present tactile sensations corresponding to the calculated internal pressure to the first and second users.

[0010] Item 6. The information processing system according to any one of items 2 to 5, wherein the control unit is configured to execute the steps of: causing the display unit to display how the first object is deforming in response to operation of the first tactile sense presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the user; causing the display unit to display how the first object is deforming in response to operation of the second tactile sense presentation unit; causing at least the second tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the user; and calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile sense presentation units.

[0011] Item 7. The information processing system according to any one of items 2 to 5, wherein the control unit is configured to execute the steps of: causing the display unit to display how the first object is deforming in response to operation of the first tactile sense presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile sense presentation unit to present to the user a tactile sensation corresponding to the calculated internal pressure; causing the display unit to display how the first object is further deforming in response to operation of the second tactile sense presentation unit; calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile sense presentation units; and causing the first and second tactile sense presentation units to present to the user a tactile sensation corresponding to the calculated internal pressure.

[0012] Item 8. The information processing system according to any one of Items 2 to 7, wherein the control unit displays, on the display unit, a plurality of second objects that move in response to operation of the tactile sense presentation unit, the plurality of second objects are configured to move within the display unit in response to operation of the first tactile sense presentation unit and the second tactile sense presentation unit, and the control unit is configured to deform the first object in response to operation of at least one of the plurality of second objects.

[0013] Item 9. The information processing system according to any one of Items 1 to 8, wherein the control unit is configured to display, on the display unit, a second object that moves in response to an operation of the tactile sense providing unit, and to deform the first object in response to the movement of the second object.

[0014] Item 10. The information processing system according to any one of Items 1 to 9, wherein each of the tactile sense providing units has an operation unit operable by the user, and the control unit is configured to calculate the virtual internal pressure based on a total of displacement amounts of the operation units of the tactile sense providing units.

[0015] Item 11. The information processing system according to any one of Items 1 to 10, wherein each of the tactile sense providing units further includes a drive unit that changes at least the load that is output in accordance with an applied current, and is configured to provide the load output from the drive unit as the tactile sense to the user when the operation unit is operated by applying a current corresponding to the tactile sense to the drive unit.

[0016] Item 12. The information processing system according to Item 11, wherein the drive unit has a magnetorheological fluid device that contains a magnetorheological fluid, the viscosity of the magnetorheological fluid changes in response to the applied current, and the load changes in response to the viscosity of the magnetorheological fluid.

[0017] Item 13. The information processing system according to any one of Items 1 to 12, wherein the user terminal comprises: an information processing terminal having the display unit; and a tactile presentation device separated from the information processing terminal and having the tactile presentation unit, wherein the information processing terminal and the tactile presentation device are configured to be able to communicate with each other.

[0018] Item 14. The information processing system according to any one of Items 1 to 13, wherein the first object is an imitation of an object that generates resistance when subjected to external pressure.

[0019] Item 15. The information processing system according to Item 8 or 9, wherein the second object is a part of a human being or a machine capable of applying an external pressure to the first object.

[0020] Item 16. The information processing system according to any one of Items 1 to 15, wherein the display unit is configured by a display provided in a smartphone, a tablet computer, a personal computer, or a head-mounted display system.

[0021] Item 17. An information processing method for an information processing system comprising at least one user terminal and at least one control unit, wherein each of the user terminals comprises: a display unit capable of displaying at least one first object; and at least one tactile sense presentation unit that presents a tactile sensation to a user, the information processing method comprising: a step of causing the display unit to display an appearance of the first object being transformed in response to an operation of the at least one tactile sense presentation unit; a step of calculating a virtual internal pressure of the transformed first object; and a step of causing the tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the user.

[0022] Item 18. An information processing system comprising: at least one user terminal; and at least one control unit, wherein each of the user terminals comprises: a display unit capable of displaying at least one first object; and at least one tactile sense presentation unit that presents a tactile sensation to the user, the information processing system causing the control unit to execute the steps of: causing the display unit to display a state in which the first object is deforming in response to an operation of the at least one tactile sense presentation unit; calculating a virtual internal pressure of the deformed first object; and causing the tactile sense presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the user.

[0023] According to the present invention, by taking into consideration the characteristics of the object itself, it is possible to present a more realistic resistance force.

[0024] 6 is a schematic diagram of a tactile presentation system according to an embodiment of the present invention. FIG. 6 is a block diagram showing the hardware configuration of an example of an information processing terminal. FIG. 6 is an example of an object displayed on a touch panel display. FIG. 6 is an example of tactile presentation data. FIG. 6 is an example of tactile presentation data. FIG. 6 is a side view of an example of a tactile presentation device. FIG. 6 is a side view showing the tactile presentation device of FIG. 6 being held by a user. FIG. 6 is a block diagram showing the electrical configuration of an example of a tactile presentation device. FIG. 6 is a cross-sectional view showing an example of a magnetorheological fluid device of the tactile presentation device of FIG. 6. FIG. 6 is a flowchart showing a play process in the tactile presentation system of FIG. 1. FIG. 6 is a flowchart showing a play process in the tactile presentation system of FIG. 1. FIG. 6 is an example of a change in an object displayed on a touch panel display. FIG. 6 is an example of a change in an object displayed on a touch panel display. FIG. 6 is an example of a change in an object displayed on a touch panel display. FIG. 6 is a side view of another example of a tactile presentation device. FIG. 6 is a side view of another example of a tactile presentation device. FIG. 6 is a schematic diagram showing another example of a tactile presentation system according to an embodiment of the present invention.

[0025] An embodiment in which the information processing system of the present invention is applied to a tactile presentation system will be described below with reference to the drawings. FIG. 1 is a schematic diagram of the tactile presentation system. As shown in FIG. 1, the tactile presentation system 1 includes an information processing terminal 3, a first tactile presentation device 4, and a second tactile presentation device 6. The information processing terminal 3 is wirelessly connected to the first and second tactile presentation devices 4 and 6. The first tactile presentation device 4 and the second tactile presentation device 6 have the same configuration. A detailed description will be given below.

[0026] <1. Information Processing Terminal> Figure 2 is a block diagram showing the hardware configuration of an information processing terminal. As shown in Figure 2, the information processing terminal 3 is configured by a computer incorporating a control unit 31, a storage unit 32, a touch panel display 33, a speaker 34, a communication interface 35, and the like. This computer can be configured as a smartphone, a tablet computer, a personal computer, a dedicated computer, or the like, but the following description will be given assuming that it is configured as a smartphone. Note that in Figure 2, the communication interface 35 is referred to as a "communication I / F." This also applies to Figure 5, which will be described later.

[0027] The control unit 31 includes a CPU (processor), RAM, ROM, etc., and is configured to execute various information processes based on programs and data. The storage unit 32 is configured, for example, by an auxiliary storage device such as an HDD or SSD, and stores a program 321, play data 322, tactile presentation data 323, and various data 324 for driving the information processing terminal 3, etc. The program 321 is a program for executing a gameplay (described below), and various processes executed in the gameplay (described below) are performed based on this program. The play data 322 is data related to the gameplay, such as data related to objects displayed on the touch panel display, which is data necessary for the gameplay. The tactile presentation data 323 is information for presenting tactile sensations in the tactile presentation devices 4 and 6 (described below). The tactile presentation data 323 may be included in the play data 322 as part of the play data 322. Multiple pieces of tactile presentation data 323 are prepared for each tactile sensation to be presented. The various data 324 is various data, such as data for operating the information processing terminal 3. However, the destination where the data is stored is not particularly limited and can be changed as appropriate. For example, at least a part of the data stored in the storage unit 32 can be stored in a RAM or a ROM.

[0028] The touch panel display 33 functions as a display device that displays a play screen (described later) and also functions as an input device as needed. However, the display device and the input device may be separate. The speaker 34 outputs sound during play as needed.

[0029] The communication interface 35 includes, for example, a wireless device that is an interface for a wireless LAN or the like, and a mobile communication modem device that is an interface for a mobile phone network including a communication base station. The wireless device includes, for example, a wireless module for Wi-Fi (registered trademark) or Bluetooth (registered trademark). This allows the information processing terminal 3 to be connected to a network such as a wireless LAN, a public network such as the Internet, or a wireless network such as a mobile phone network including a communication base station. In this embodiment, the information processing terminal 3 transmits and receives various data to and from the tactile presentation devices 4 and 6 by any of the above-mentioned means.

[0030] Next, a description will be given of the display on the touch panel display 33. As shown in Fig. 3, in this embodiment, a vertically elongated balloon (first object) 71 as a first object, and a right hand 72 and a left hand 73 as second objects holding the balloon 71 are displayed on the touch panel display 33.

[0031] The balloon 71 has a virtual internal pressure S as an initial value, and is configured such that the internal pressure of the balloon 71 virtually increases when the right hand 72 and left hand 73 displayed on the touch panel display 33 grasp the balloon 71. The internal pressure increases by the sum of the movements of the right hand 72 and left hand 73 grasping the balloon 71. In this embodiment, when the operation unit 102 of the tactile presentation devices 4 and 6 (described later) is pressed, an animation is displayed on the touch panel display 33 showing the right hand 72 and left hand 73 grasping the balloon 71. At this time, when the angles θ1 and θ2 of the operation unit 102 are displaced by the pressing of the operation unit 102, the internal pressure of the balloon 71 increases in proportion to the sum of the displacement angles. Note that the angles θ1 and θ2 represent the displacement amounts of the operation unit 102 of the first tactile presentation device 4 and the operation unit 102 of the second tactile presentation device 6, respectively. Figure 4 shows this behavior, and is a graph showing the relationship between the total angle of the displacement amounts and the internal pressure of the balloon 71. As will be described later, each of the displacement amounts θ1 and θ2 varies between 0 and 90 degrees. The total angle of the displacement amounts θ1 + θ2 has a minimum value of 0 degrees and a maximum value of 180 degrees.

[0032] Once the internal pressure of the balloon 71 is determined, the tactile sensations felt by the right hand 72 and the left hand 73 in accordance with the internal pressure are presented to the tactile presentation devices 4 and 6. To this end, the information processing terminal 3 transmits current parameter information to the tactile presentation devices 4 and 6, which then input the received current parameter information to the magnetorheological fluid device 104. As a result, the tactile presentation devices 4 and 6 present a predetermined tactile sensation to the user. The current parameter information is, for example, parameters including a current value, a frequency, and a duty ratio. As shown in FIG. 5 , once the internal pressure of the balloon 71 is determined, the corresponding current parameters are determined. However, the balloon 71 is designed to burst if its internal pressure exceeds a predetermined value X. Therefore, X is the upper limit of the internal pressure. Among the current parameter information, the current value affects the tactile sensation obtained and the hardness of the object. In other words, the internal pressure of the balloon 71 can be changed by varying the current value.

[0033] In this embodiment, the tactile presentation data 323 is generated based on the graphs shown in Figures 4 and 5. That is, a table based on Figures 4 and 5 is stored in the storage unit as the tactile presentation data. Furthermore, such a table is stored for each object. For example, in this embodiment, a balloon is defined as the first object, but tactile presentation data for other objects is also prepared. However, the graphs shown in Figures 4 and 5 are merely examples, and graphs represented by other curves such as quadratic curves or straight lines can also be used.

[0034] 2. Tactile Presentation Device Fig. 6 is a side view of the tactile presentation device, Fig. 7 is a side view of the tactile presentation device showing a state in which the user is operating it, and Fig. 8 is a block diagram showing the electrical configuration of the tactile presentation device. As described above, the first and second tactile presentation devices 4, 6 have the same configuration, and therefore the following description will focus on the first tactile presentation device 4. The first tactile presentation device 4 presents to the user a tactile sensation corresponding to an object displayed on the touch panel display 33 of the information processing terminal 3. Furthermore, as shown in Fig. 7, the first tactile presentation device 4 is formed in a shape and size that fits in the user's hand and can be operated by the user's hand.

[0035] As shown in FIGS. 6 and 7, the first tactile presentation device 4 has an operation unit 102, a casing 105, and a support shaft 109.

[0036] <2-1. Electrical Configuration of the Tactile Presentation Device> The casing 105 has an internal housing 107. This housing 107 houses the control unit 51, memory unit 52, rotating shaft 103, magnetorheological fluid device 104, displacement amount detection unit 53, communication interface 55, and battery unit 56. The portion of the housing 107 that houses the magnetorheological fluid device (drive unit) 104 has a shape that allows the outer periphery of the magnetorheological fluid device 104 (a case 133 described later) to fit into it.

[0037] As shown in FIG. 8 , the control unit 51 includes a CPU (processor) and is configured to execute various information processes based on programs and data. The memory unit 52 is configured with a storage device such as a RAM or ROM, and stores a program 521, current parameter information 522, and various data 523 for driving the first tactile presentation device 4, etc. For example, the program and various data can be stored in the ROM, and the current parameter information 522 can be stored in the RAM. As will be described later, the information processing terminal 3 transmits predetermined current parameter information to the first tactile presentation device 4 based on the tactile presentation data 323 stored in the memory unit 32, and stores the information in the RAM. However, the storage destination of the data is not particularly limited and can be changed as appropriate. For example, an auxiliary storage device such as an SSD may be provided and the data may be stored therein.

[0038] The control unit 51 executes the program 521 to control the current input from the battery unit 56 to the magnetorheological fluid device 104, as will be described later. Specifically, the control unit 51 inputs current parameter information received from the first information processing terminal 3 to the magnetorheological fluid device 104. The control unit 51 controls the current input to the magnetorheological fluid device 104, thereby controlling the rotational resistance applied to the rotation shaft 103 of the magnetorheological fluid device 104, i.e., the displacement resistance of the operation unit 102. The control unit 51, memory unit 52, etc. of the tactile presentation device 4 shown in FIG. 8 can be configured using a microcomputer (microcontroller).

[0039] The communication interface 55 includes, for example, a wireless device that is an interface such as a wireless LAN. The wireless device includes, for example, a wireless module such as WiFi (registered trademark) or Bluetooth (registered trademark). This allows the first tactile presentation device 4 to be connected to a wireless network such as a wireless LAN or a public network such as the Internet. The first tactile presentation device 4 can be connected to the information processing terminal 3 via Bluetooth, for example.

[0040] The battery unit 56 supplies current to the magnetorheological fluid device 104, the control unit 51, the storage unit 52, the displacement amount detection unit 53, and the communication interface 55. The displacement amount detection unit 53 will be described later.

[0041] 2-2. Mechanical Configuration of the Tactile Presentation Device The casing 105 has a generally cylindrical outer shape, and a support arm 113 extends outward from a part of the casing 105 (to the right in FIG. 6) as shown in FIG. 6. The support arm 113 has the aforementioned support shaft 109 at its tip.

[0042] The operation unit 102 is disposed above the casing 105 and is configured to be displaced relative to the casing 105 by operation by the user. The operation unit 102 is formed in the shape of a curved plate, and one end thereof is rotatably supported on a support shaft 109. The operation unit 102 is configured to rotate around the support shaft 109 to move closer to or away from the upper part of the casing 105. When the operation unit 102 is rotated by an operating force of the user, the operating force is transmitted to the rotation shaft 103 of the magnetorheological fluid device 104 via a link mechanism 110.

[0043] A placement portion 106 on which a finger can be placed is provided at the bottom of the casing 105. As shown in Fig. 7, a user can hold the magnetorheological fluid device 104 by placing the index finger 125 along the operation portion 102 and the thumb along the placement portion 126. This allows the user to operate the operation portion 102 with the index finger 125 while holding the magnetorheological fluid device 104 in their hand.

[0044] The operation unit 102 is configured to be rotatable within a predetermined range. In this embodiment, the operation unit 102 is not provided with a return spring or the like. Therefore, when using the tactile presentation device 6, it is preferable to fasten the user's index finger 125 and the operation unit 102 with, for example, a hook-and-loop fastener fastener 112, as shown in FIG. 7 . This allows the user to push the operation unit 102 into the casing 105 with their fingertip and then pull it back to its original position.

[0045] The link mechanism 110 is configured to link the rotation of the operation unit 102 about the support shaft 109 with the rotation of the rotating shaft 103. Furthermore, a one-way clutch (not shown) is interposed in the link mechanism 110. When the user pushes the operation unit 102 toward the casing 105, the one-way clutch transmits the operating force transmitted from the operation unit 102 via the link mechanism 110 to the rotating shaft 103. On the other hand, when the user pulls the operation unit 102 back from the casing 105, the one-way clutch does not transmit the operating force transmitted from the operation unit 102 via the link mechanism 110 to the rotating shaft 103.

[0046] 9 is a cross-sectional view of the magnetorheological fluid device 104. As shown in FIG. 9, the magnetorheological fluid device 104 has a rotating shaft 103 that rotates in response to the displacement of the operation unit 102. The magnetorheological fluid device 104 is configured so that a magnetic field is applied to the magnetorheological fluid by an input current, thereby generating a rotational resistance in the rotating shaft 103. The magnetorheological fluid device 104 applies a rotational resistance (displacement resistance) to the operation unit 102 by applying a rotational resistance to the rotating shaft 103.

[0047] The magnetorheological fluid device 104 of this embodiment applies a magnetic field to the magnetorheological fluid, the strength of which corresponds to the magnitude of the current supplied from the battery unit 56 and the control unit 51, and thereby applies a rotational resistance to the rotating shaft 103 that corresponds to the magnitude of the current. The magnetorheological fluid device 104 has a substantially cylindrical outer shape, with the rotating shaft 103 protruding from one end face. The axis of the rotating shaft 103 and the axis of the support shaft 109 are substantially parallel to each other.

[0048] A displacement amount detection unit 53 that detects the rotation angle of the rotating shaft 103 is provided around the rotating shaft 103. The displacement amount detection unit 53 can be formed by a sensor such as an encoder. This allows the displacement amount detection unit 53 to detect the amount of displacement of the operating unit 102 that is linked to the rotating shaft 103 via a link mechanism 110. The displacement amount detection unit 53 can detect any amount of displacement in accordance with changes in the rotation angle of the rotating shaft 103.

[0049] 2-3. Structure of the magnetorheological fluid device Next, we will explain an example of the internal structure of the magnetorheological fluid device 104. As shown in Fig. 9, the magnetorheological fluid device 104 has a rotating shaft 103, a disk 128, yokes 129 and 130, a coil 131, a magnetorheological fluid 132, a case 133, etc., which constitute a rotating part.

[0050] The disk 128 is an internal rotor that rotates inside the magnetorheological fluid device 104 and is made of a magnetic material. The rotating shaft 103 is connected perpendicularly to the center of the back surface 128b of the disk 128. Therefore, the disk 128 and the rotating shaft 103 rotate together. The rotating shaft 103 is supported in a shaft hole 136 provided in the yoke 130 via a bearing 135. It is preferable that the rotating shaft 103 be made of a non-magnetic material.

[0051] The yoke is made up of a first yoke 129 and a second yoke 130. The first yoke 129 is made up of a disk-shaped member and faces the front surface 128a of the disk 128 via a small gap. The first yoke 129 is fitted into and fixed in a cylindrical case 133.

[0052] The second yoke 130 has an opposing surface 130a that faces the back surface 128b of the disk 128 via a small gap. The second yoke 130 is fitted and fixed inside a cylindrical case 133. The case 133, which is the outer periphery of the magnetorheological fluid device 104, is fixed so as not to rotate relative to the casing 105. In this embodiment, as shown in FIG. 6 , the case 133 is fixed to the casing 105 using a hollow set 127.

[0053] A space is formed by a recess formed in the center of the first yoke 129 and a recess formed in the center of the end face of the rotating shaft 103. A sphere 137 made of a non-magnetic material is accommodated in this space.

[0054] The coil 131 is disposed along an annular groove formed in the second yoke 130. A current is input to the coil 131 from the control unit 51.

[0055] The magnetorheological fluid 132 is sealed in the gap between the disk 128 and the first and second yokes 129 and 130. This magnetorheological fluid 132 is a liquid composed of magnetic particles dispersed in a dispersion medium. The magnetic particles can be, for example, nano-sized metal particles (metal nanoparticles). The magnetic particles are made of a magnetizable metal material. While there are no particular limitations on the metal material, soft magnetic materials are preferred. Examples of soft magnetic materials include alloys of iron, cobalt, nickel, and permalloy. The dispersion medium is not particularly limited, but hydrophobic silicone oil is an example. The amount of magnetic particles in the magnetorheological fluid can be, for example, 3 to 40 vol%. Various additives can also be added to the magnetorheological fluid to achieve various desired properties.

[0056] In the magnetorheological fluid device 104 configured as described above, when a current is applied to the coil 131, a magnetic path is formed within the disk 128, the first yoke 129, and the second yoke 130, for example, along the direction indicated by arrow P in FIG. 9 . This magnetic path penetrates the magnetorheological fluid 132 disposed in the gap between the surface 128a of the disk 128 and the first yoke 129, and the magnetorheological fluid 132 disposed in the gap between the back surface 128b of the disk 128 and the second yoke 130. As a result, the magnetorheological fluid 132 develops viscosity (shear stress) corresponding to the strength of the magnetic field, and the torque transmitted between the disk 128 and the yokes 129 and 130 increases in accordance with the strength of the magnetic field. In other words, the magnetorheological fluid device 104 applies rotational resistance to the rotating shaft 103 whose magnitude corresponds to the value of the current applied to the coil 131.

[0057] 2-4. Operation of the Tactile Presentation Device When using the first tactile presentation device 4 having the above configuration, as shown in FIG. 7 , the first tactile presentation device 4 is held between fingers, and the operation unit 102 is pushed in or pulled back with the index finger 125. When the operation unit 102 is pushed in, the operating force received by the operation unit 102 is transmitted to the magnetorheological fluid device 104 via the one-way clutch. This generates resistance to the pushing of the operation unit 102 according to the current input to the magnetorheological fluid device 104, and this resistance is presented to the user as a tactile sensation. Conversely, when pulling back the operation unit 102, the operating force received by the operation unit 102 is not transmitted from the one-way clutch to the magnetorheological fluid device 104. This means that almost no resistance is generated in the pulling back of the operation unit 102. This allows the user to pull back the operation unit 102 with a light force.

[0058] In this embodiment, the rotation axis 103 rotates 90 degrees from when the operation unit 102 is in the initial state until the operation unit 102 is fully pressed in. Therefore, the tactile presentation data indicates rotation angles θ1 and θ2 from 0 to 90 degrees.

[0059] 3. Play Processing by the Tactile Presentation System Next, the play processing executed by the tactile presentation system 1 will be described with reference to the flowcharts of FIGS. 10 and 11 and the screen displays of the information processing terminal shown in FIGS. 12 and 13. FIG.

[0060] First, the user wears the first tactile presentation device 4 on his right hand and the second tactile presentation device 6 on his left hand. Next, as shown in FIG. 10 , the user turns on the power of the first and second tactile presentation devices 4 and 6 and starts the program 321 on the information processing terminal 3. This allows the information processing terminal 3 and the first and second tactile presentation devices 4 and 6 to be communicatively connected, for example, via Bluetooth (step S101). That is, pairing is performed. At this time, a setting input is made as to whether the tactile presentation devices 4 and 6 are worn on the left or right hand. For example, the first tactile presentation device 4 is set to the right hand and the second tactile presentation device 6 is set to the left hand. This associates the tactile presentation devices 4 and 6 with the second objects, that is, the hands 72 and 73, respectively.

[0061] 3, the information processing terminal 3 displays a play screen on the touch panel display 33 (step S102). That is, the screen displays a balloon 71 and hands 72 and 73. In this state, when the user presses the operation unit 102 of the first tactile sense presentation device 4 (step S201), the first tactile sense presentation device 4 detects the displacement θ1 of the operation unit 102 in real time and transmits it to the information processing terminal 3 (step S202).

[0062] The information processing terminal 3 deforms the right hand 72 and the balloon 71 on the touch panel display 33 so that the right hand 72 grasps the balloon 71, as shown in Figure 12, in accordance with the displacement amount θ1 of the operation unit 102 transmitted from the first tactile presentation device 4 (step S103).

[0063] The information processing terminal 3 also calculates current parameter information from the received displacement θ1 (θ2 is set to 0) based on the tactile presentation data shown in Figures 4 and 5 described above, and transmits the calculated information to the first tactile presentation device 4 and the second tactile presentation device 6 (step S104). As a result, the first tactile presentation device 4 drives the magnetorheological fluid device 104 based on the received current parameters. As a result, the user can feel a tactile sensation in his or her right hand that corresponds to the displacement of the operation unit 102 (step S203).

[0064] The above process is performed until the user stops operating the operation unit 102 of the first tactile sense presentation device 4 (YES in step S105, YES in step S204). During this time, the information processing terminal 3 displays, like an animation, changes in the right hand 72 and the balloon 71 on the touch panel display 33 in accordance with the increase and decrease in the displacement amount θ1. Furthermore, the first tactile sense presentation device 4 changes the tactile sensation presented in accordance with the increase and decrease in the displacement amount θ1. This allows the user to feel a continuous change in the tactile sensation while pressing the operation unit 102. In other words, as the user presses the operation unit 102, the internal pressure of the balloon 71 increases, and the user can feel a gradual increase in the resistance to pressing the balloon 71. Note that the displacement amount θ1 is not transmitted to the information processing terminal 3 while the operation of the operation unit 102 of the first tactile sense presentation device 4 is stopped. This is also true for the second tactile sense presentation device 6, and the displacement amount θ2 is not transmitted to the information processing terminal 3 while the operation of the operation unit 102 of the second tactile sense presentation device 6 is stopped.

[0065] In the information processing terminal 3, the order of deformation of the object on the touch panel display 33 (step S103) and transmission of the current parameter information (step S104) is not particularly limited, and they can be performed simultaneously. This also applies to the processing by the second tactile presentation device 6 described next.

[0066] Next, the process of Fig. 10 will be continued with reference to Fig. 11. In the process of Fig. 11, the user presses the operation unit 102 of the second tactile presentation device 6 with the left hand while maintaining the pressing of the operation unit 102 of the first tactile presentation device 4 with the right hand (YES in step S204).

[0067] Next, when the user presses the operation unit 102 of the second tactile presentation device 6 (step S301), the second tactile presentation device 6 detects the displacement θ2 of the operation unit 102 in real time and transmits it to the information processing terminal 3 (step S302). At this time, information on the current parameters based on the operation of the first tactile presentation device 4 has already been transmitted to the second tactile presentation device 6 (step S104 in FIG. 10 ), so the initial tactile sensation felt by the user when operating the second tactile presentation device 6 corresponds to the displacement θ1 described above.

[0068] In accordance with the displacement amount θ2 of the operation unit 102 transmitted from the second tactile presentation device 6, the information processing terminal 3 deforms the left hand 73 and the balloon 71 on the touch panel display 33 so that, in addition to the right hand 72, the left hand 73 is grasping the balloon 71, as shown in Figure 13 (step S106).

[0069] The information processing terminal 3 also sums the displacement amount θ2 of the operation unit 102 transmitted from the second tactile presentation device 6 with the displacement amount θ1 of the operation unit 102 of the first tactile presentation device 4. The information processing terminal 3 then calculates current parameter information based on the tactile presentation data shown in FIGS. 4 and 5 described above, and transmits the calculated information to the first and second tactile presentation devices 4 and 6 (step S107). The first and second tactile presentation devices 4 and 6 then drive the magnetorheological fluid device 104 based on the received current parameters. As a result, the user can feel tactile sensations in both their right and left hands that correspond to the displacement amounts of the operation units 102 of the tactile presentation devices 4 and 6 (step S303).

[0070] The above process is performed until the user stops operating the operation unit 102 of the second tactile presentation device 6 (YES in step S108, YES in step S304), and during that time the information processing terminal displays, like an animation, changes in the left hand 73 and the balloon 71 on the touch panel display 33. Furthermore, as the user presses the operation unit 102 of the second tactile presentation device 6, the internal pressure of the balloon 71 increases, and the user can feel a gradual increase in resistance to pressing the balloon 71.

[0071] 13 is returned to the initial state by at least one of the right and left hands, the amount of displacement decreases, and the total angle of displacement θ1+θ2 also decreases. As a result, the internal pressure decreases, and the display of the balloon 71 and the hands 72 and 73 on the touch panel display 33 changes, and the tactile sensations presented by the tactile presentation devices 4 and 6 also change. As described above, by pressing in and returning the operation unit 102, the changes in the balloon 71 and the hands 72 and 73 can be visually recognized, and tactile sensations corresponding to the change in internal pressure can be felt by the tactile presentation devices 4 and 6.

[0072] <4. Features> According to this embodiment, the resistance force from the balloon 71 is presented as a tactile sensation in consideration of the internal pressure, which is a characteristic of the balloon 71, so that a more realistic resistance force can be felt. In particular, the internal pressure of the balloon 71 is calculated based on the sum of the displacement amounts of the operation units 102 of the first and second tactile presentation devices 4, 6, so that the resistance force from the balloon 71 obtained by pressing with both operation units 102 can be felt by the first and second tactile presentation devices 4, 6. In other words, a more realistic tactile sensation can be felt when performing a pressing operation with both hands.

[0073] 5. Modifications Although one embodiment of the present invention has been described above, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the spirit of the present invention. The following modifications can be combined as appropriate. Furthermore, the following modifications can also be combined as appropriate with the above-described embodiment.

[0074] (1) In the above embodiment, the internal pressure of the object (balloon 71) is calculated based on the displacement of the operation unit 102 of the tactile presentation devices 4 and 6, and current parameter information is generated based on this. However, this is not limited to this. For example, the internal pressure may be estimated from the state of the object, and current parameter information corresponding to this internal pressure may be transmitted to the tactile presentation devices 4 and 6. For example, using the formula "PV = constant (P: pressure, V: volume)" (1), if the volume of the object is halved as a result of image analysis, the internal pressure can be set to twice the initial value. For example, as shown in FIG. 14 , if a ball is displayed as an object on the touch panel display 33 and the ball is deformed in response to the operation of each tactile presentation device 4 and 6, the degree of deformation can be calculated by image analysis, and the internal pressure can be calculated based on this.

[0075] In addition to the above formula (1), the internal pressure may be changed according to a change in temperature using the formula "PV / T = constant (P: pressure, V: volume, T: temperature)." The temperature may be, for example, the temperature in the real space measured by the information processing terminal 3 or a temperature set in the virtual space in which the object is placed.

[0076] (2) In the above embodiment, a balloon and a hand are used as the objects, but this is not a limitation. In other words, any object whose resistance force changes virtually when operated by the tactile presentation device 4, 6 is not particularly limited. For example, the first object may be a container filled with gas or liquid, such as a plastic bottle with a closed lid, an unopened beverage can, a ball, or an inflated tire. Furthermore, the second object may be a representation of various human body parts other than a hand, or a machine or the like that can exert a force on the first object.

[0077] (3) In the above embodiment, the information processing terminal 3 and the first and second tactile presentation devices 4, 6 are configured as separate devices. However, these may also be integrated into a single device (user terminal).

[0078] (4) In the above embodiment, the current value was primarily used as the parameter information for the current. However, in addition to the current value, the frequency and duty ratio can also be used. Setting the frequency and duty ratio allows the current value to fluctuate at a predetermined cycle, for example, like a pulse wave. This affects the tactile characteristics of the object. Therefore, when dealing with an object whose surface tactile feel changes with changes in internal pressure, using the frequency and duty ratio as well can provide a more realistic tactile sensation.

[0079] (5) In the above embodiment, the magnetorheological fluid device 104 is used as an example of the tactile presentation devices 4 and 6 to present tactile sensations. However, this is not limiting. For example, a motor can be used to present different tactile sensations by varying the load on the rotation axis using current. In addition to the load resistance described above, the tactile presentation device can also present vibration, texture, or a warm / cold sensation, or a combination of these, depending on the internal pressure of the object. Therefore, the above-described tactile presentation data is merely an example, and the tactile presentation devices 4 and 6 may be configured to present different tactile sensations. In other words, in the above embodiment, the tactile presentation data 323 indicates the relationship between the rotation angle of the operation unit 102 and the current. However, any data may be used that allows the tactile sensation to be changed depending on the operation of the tactile presentation device.

[0080] (6) In the above-described embodiments, the tactile presentation devices 4, 6 are configured so that the user pinches the tactile presentation devices 4, 6 between two fingers and feels tactile sensations by manipulating the index finger. However, the structure of the tactile presentation devices 4, 6 is not limited to this. That is, the tactile presentation devices 4, 6 may be configured so that the user can operate the devices with multiple fingers and feel tactile sensations individually for each finger. Furthermore, the tactile presentation devices 4, 6 may be configured so that the user can feel tactile sensations by tracing the device with a finger instead of by pressing the operation unit 102. That is, the tactile presentation devices 4, 6 may be configured so that the user can feel tactile sensations with the fingers or hand (e.g., palm).

[0081] For example, the tactile presentation device 40 shown in FIG. 15 includes a magnetorheological fluid device 104 and a link member 81 connected to the rotation shaft 103 of the magnetorheological fluid device 104. The magnetorheological fluid device 104 is rotatably attached to the back of the hand by a hand attachment 82. Meanwhile, the end of the link member 81 opposite the rotation shaft 103 is rotatably attached to one of the fingers by a finger attachment 83. As a result, as shown in FIG. 16 , when the hand is clenched, the movement of the finger is transmitted to the rotation shaft 103 via the link member 81, causing the rotation shaft 103 to rotate. Therefore, when the hand is clenched, the tactile sensation (resistance when squeezing a balloon) can be felt as described above in accordance with the rotation of the rotation shaft 103. Note that although the examples of FIGS. 15 and 16 show the tactile presentation device 40 being attached to one finger, it may also be attached to multiple fingers. Furthermore, when tactile presentation devices 40 are attached to multiple fingers, the displacement amount transmitted to the information processing terminal 3 can be the sum of the displacement amounts of the rotation axes 103 of the magnetorheological fluid devices 104 in all tactile presentation devices 40. By setting which finger of the left or right hand each tactile presentation device 40 is attached to, it is possible to display the second objects, ie, the hands 72 and 73, moving on a finger-by-finger basis. Furthermore, when the magnetorheological fluid device 104 corresponding to the thumb does not displace, it can be determined that this is a non-gripping action, and control can be performed so that no tactile sensation (resistance) is imparted.

[0082] (7) In the above embodiment, two tactile presentation devices 4 and 6 are connected to one information processing terminal 3. However, three or more tactile presentation devices may be connected and the internal pressure of the object may be changed by operating the three or more tactile presentation devices. In this case, the internal pressure of the object is also determined by the total amount of displacement of the operation unit by the three or more tactile presentation devices.

[0083] It is also possible to connect one tactile presentation device 4 to one information processing terminal 3. Even in this case, the resistance force due to the change in the internal pressure of the object can be felt as a tactile sensation, so that a realistic tactile sensation can be obtained.

[0084] (8) In the above embodiment, the information processing terminal 5 and the first and second tactile presentation devices 4 and 6 are communicatively connected, but this is not limiting. For example, as shown in Fig. 17, an external server 2 may be communicatively connected to the information processing terminal 3. The external server 2 may be configured by a computer that can be connected to the information processing terminal 3 via a network such as the Internet.

[0085] In this case, parameter information of the current based on the tactile presentation data 323 can also be transmitted from the external server 2 to the tactile presentation devices 4 and 6 via the information processing terminal 3. Therefore, the control unit of the present invention can be included in the information processing terminal (user terminal) 3 as in the above embodiment, or can be included in the external server 2. Furthermore, multiple control units can be provided, and the devices in which they are located are not limited. That is, control units can be distributed and located in two or more of the information processing terminal 3, the tactile presentation devices 4 and 6, and the external server 2, and the processing for playing the above-mentioned game can be executed in a distributed manner.

[0086] (9) In the above embodiment, two tactile presentation devices 4, 6 are connected to one information processing terminal 3, and one user operates each tactile presentation device 4, 6 with his or her right and left hands. However, this is not limiting. For example, as shown in FIG. 18 , another information processing terminal 5 (hereinafter referred to as a second information processing terminal) is prepared, and one tactile presentation device 4, 6 is connected to each information processing terminal 3, 5. Two users then operate one information processing terminal and one tactile presentation device. The two information processing terminals 3, 5 are connected to each other via wired or wireless communication, and the displacement amounts of the operation units 102 caused by each tactile presentation device 4, 6 are transmitted to each other. At this time, avatars corresponding to each user can be displayed on each information processing terminal 3, 5, and each avatar (second object) can exert a force on a first object, such as a balloon. In this way, the two users can change the first object and feel the resistance caused by the internal pressure of the first object. This allows people in different locations to communicate with each other.

[0087] (10) External pressure may be applied to the first object separately by operating the two tactile presentation devices 4, 6. In this case, the number of users operating the two tactile presentation devices 4, 6 is not limited, and one person may operate the two tactile presentation devices 4, 6. Furthermore, external pressure may be applied to the first object simultaneously by operating the two tactile presentation devices 4, 6. In this case, the number of users operating the two tactile presentation devices 4, 6 is not limited, and one person may operate the two tactile presentation devices 4, 6.

[0088] (11) The present invention can be used as a virtual reality technology that uses a device such as a head-mounted display to operate a virtual object in a virtual space. That is, the above-described information processing terminal can be used as a head-mounted display. In this case, the head-mounted display displays a balloon or the like as a first object and the left and right hands of a user wearing the head-mounted display as second objects.

[0089] REFERENCE SIGNS LIST 1: Tactile presentation system (information processing system) 3: Information processing terminal 4: First tactile presentation device 6: Second tactile presentation device 31: Control unit 71: Balloon (first object) 72: Right hand (second object) 73: Left hand (second object) 102: Operation unit 104: Magnetorheological fluid device (drive unit) 321: Program

Claims

1. An information processing system comprising: at least one user terminal; and at least one control unit, wherein each of the user terminals comprises: a display unit capable of displaying at least one first object; and at least one tactile presentation unit that presents a tactile sensation to the user, and the control unit is configured to execute the steps of: causing the display unit to display the first object as it deforms in response to operation of at least one of the tactile presentation units; calculating a virtual internal pressure of the deformed first object; and causing the tactile presentation unit to present a tactile sensation to the user that corresponds to the calculated internal pressure.

2. The information processing system according to claim 1, wherein the tactile sense providing unit includes at least a first tactile sense providing unit and a second tactile sense providing unit.

3. The information processing system according to claim 2, wherein the first tactile sense presentation unit and the second tactile sense presentation unit are operated by a first user and a second user, respectively.

4. The information processing system of claim 3, wherein the control unit is configured to execute the steps of: displaying on the display unit how the first object is deformed in response to operation of the first tactile presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile presentation unit to present to the first user a tactile sensation corresponding to the calculated internal pressure; displaying on the display unit how the first object is deformed in response to operation of the second tactile presentation unit; causing at least the second tactile presentation unit to present to the second user a tactile sensation corresponding to the calculated internal pressure; and calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile presentation units.

5. The information processing system of claim 3, wherein the control unit is configured to execute the steps of: displaying on the display unit how the first object is deformed in response to operation of the first tactile presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the first user; displaying on the display unit how the first object is further deformed in response to operation of the second tactile presentation unit; calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile presentation units; and causing the first and second tactile presentation units to present a tactile sensation corresponding to the calculated internal pressure to the first and second users.

6. The information processing system of claim 2, wherein the control unit is configured to execute the steps of: displaying on the display unit how the first object is deformed in response to operation of the first tactile presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile presentation unit to present to the user a tactile sensation corresponding to the calculated internal pressure; displaying on the display unit how the first object is deformed in response to operation of the second tactile presentation unit; causing at least the second tactile presentation unit to present to the user a tactile sensation corresponding to the calculated internal pressure; and calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile presentation units.

7. The information processing system of claim 3, wherein the control unit is configured to execute the steps of: displaying on the display unit how the first object is deformed in response to operation of the first tactile presentation unit; calculating a virtual internal pressure of the deformed first object; causing at least the first tactile presentation unit to present to the user a tactile sensation corresponding to the calculated internal pressure; displaying on the display unit how the first object is further deformed in response to operation of the second tactile presentation unit; calculating a virtual internal pressure of the first object deformed by operation of the first and second tactile presentation units; and causing the first and second tactile presentation units to present to the user a tactile sensation corresponding to the calculated internal pressure.

8. An information processing system as described in any one of claims 4 to 7, wherein the control unit displays on the display unit a plurality of second objects that operate in response to operation of the tactile presentation unit, the plurality of second objects are configured to operate within the display unit by operation of the first tactile presentation unit and the second tactile presentation unit, and the first object is configured to be deformed by operation of at least one of the plurality of second objects.

9. An information processing system according to any one of claims 1 to 3, wherein the control unit is configured to display on the display unit a second object that moves in response to operation of the tactile presentation unit, and to deform the first object by the movement of the second object.

10. An information processing system as described in claim 1 or 2, wherein each of the tactile presentation units has an operation unit that can be operated by the user, and the control unit is configured to calculate the virtual internal pressure based on the sum of the displacement amounts of the operation units of each of the tactile presentation units.

11. The information processing system of claim 3, wherein each of the tactile presentation units further comprises a drive unit that changes at least the load output depending on the applied current, and is configured to apply a current corresponding to the tactile sensation to the drive unit, so that the load output from the drive unit when the operation unit is operated is presented to the user as the tactile sensation.

12. The information processing system according to claim 4, wherein the drive unit has a magnetorheological fluid device that contains a magnetorheological fluid, the viscosity of the magnetorheological fluid changes in response to the applied current, and the load changes in response to the viscosity of the magnetorheological fluid.

13. The information processing system described in claim 1 or 2, wherein the user terminal comprises: an information processing terminal having the display unit; and a tactile presentation device separated from the information processing terminal and having the tactile presentation unit, and the information processing terminal and the tactile presentation device are configured to be able to communicate with each other.

14. The information processing system according to claim 1 or 2, wherein the first object is an imitation of an object that generates resistance when subjected to external pressure.

15. The information processing system according to claim 8 or 9, wherein the second object is a part of a human being or a machine capable of exerting external pressure on the first object.

16. The information processing system according to claim 1 or 2, wherein the display unit is configured by a display provided in a smartphone, a tablet computer, a personal computer, or a head-mounted display system.

17. An information processing method in an information processing system comprising at least one user terminal and at least one control unit, each of the user terminals comprising: a display unit capable of displaying at least one first object; and at least one tactile presentation unit that presents a tactile sensation to the user, the information processing method comprising the steps of: causing the display unit to display how the first object is deforming in response to operation of at least one of the tactile presentation units; calculating a virtual internal pressure of the deformed first object; and causing the tactile presentation unit to present to the user a tactile sensation corresponding to the calculated internal pressure.

18. An information processing system comprising at least one user terminal and at least one control unit, each of the user terminals comprising: a display unit capable of displaying at least one first object; and at least one tactile presentation unit that presents a tactile sensation to the user. The information processing program causes the control unit to execute the following steps: causing the display unit to display how the first object is deforming in response to operation of at least one tactile presentation unit; calculating a virtual internal pressure of the deformed first object; and causing the tactile presentation unit to present a tactile sensation corresponding to the calculated internal pressure to the user.

Images