Virtual object tactile presentation device and program

Abstract

To provide the hand of a user with a tactile sense depending on a kind of a virtual object displayed on a head-mounted display.SOLUTION: A virtual object tactile sense presentation device includes a head-mounted display device 2, an HMD control device 3, a tactile sense presentation device 4, and a tactile sense information DB 22 for storing tactile sense information in association with a kind of a virtual object. The HMD control device 3 includes a display control unit 32 that allows a display 8 to display the virtual object and a user focus 28, and a tactile sense information transmission unit 35 that, when the user focus 28 is overlapped with a selectable virtual object 38, reads the tactile sense information associated with a kind of the virtual object 38 from the tactile sense information DB 22 to transmit the information to the tactile sense presentation device 4. The tactile sense presentation device 4 includes a displacement unit 102 to be displaced by an operator of a user, a rotation resistance generation unit 104 that gives displacement resistance to the displacement unit 102, and a power supply control unit (displacement resistance control unit) 202 that controls displacement resistance according to the received tactile sense information.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a virtual object tactile presentation device and a program for presenting the tactile sensation of a virtual object displayed on a display. [Background technology]

[0002] One of the currently commercially available head-mounted displays capable of displaying virtual objects is HOLOLENS (registered trademark) manufactured by Microsoft Corporation in the United States. This head-mounted display is equipped with a transparent display and can display virtual objects blended into the real world. The display can also display a user focus for selecting a virtual object. The user focus is displayed at the position where the user's line of sight, called "Gaze," intersects with the virtual object or the real world. When the user places the user focus on a selectable virtual object and performs a predetermined hand gesture (for example, a hand gesture called "air tap"), the virtual object is selected and a predetermined process associated with the virtual object, such as a change in the shape of the virtual object, generation of a sound effect, execution of a predetermined application program, etc., is started. The predetermined hand gesture is performed within the recognition range of a hand gesture sensor (infrared camera, depth camera, etc.) provided in HOLOLENS (registered trademark).

[0003] As a document related to this type of technology, for example, Patent Document 1 can be mentioned. In the device disclosed in Patent Document 1, a virtual object is displayed in the real world on a see-through head mounted display, and the virtual object can be selected using a user focus. The selected virtual object changes its shape. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2016-530600 Summary of the Invention [Problem to be solved by the invention]

[0005] The inventors of the present application considered that if a tactile sensation corresponding to a virtual object could be imparted to the user's hand when the user makes a hand gesture to select a virtual object, it would be possible to expand the uses of head-mounted displays and improve their convenience, playability, etc.

[0006] Therefore, an object of the present invention is to provide a virtual object tactile presentation device that can impart a tactile sensation to a user's hand that corresponds to the type of virtual object displayed on a display. [Means for solving the problem]

[0007] A virtual object tactile presentation device according to one aspect of the present invention includes a display device, a display control device for controlling a display ray of the display device, a tactile presentation device for presenting tactile sensations to a user, and a tactile information storage unit for storing tactile information associated with a type of selectable virtual object. The display control device includes a display control unit for displaying selectable virtual objects on the display and a virtual object selection means for selecting the virtual object by a user's operation, and a tactile information transmission unit for reading tactile information associated with the type of the virtual object from the tactile information storage unit and transmitting the tactile information to the tactile presentation device when a selectable virtual object is selected by the virtual object selection means. The tactile presentation device includes a displacement unit that is displaced by a user's operation, a displacement resistance generation unit that imparts a displacement resistance to the displacement unit, and a displacement resistance control unit that controls the displacement resistance imparted by the displacement resistance generation unit to the displacement unit in accordance with the tactile information received from the tactile information transmission unit.

[0008] A virtual object tactile presentation device having such a configuration can provide a tactile sensation to the user's hand that corresponds to the type of virtual object displayed on the display.

[0009] In a virtual object tactile presentation device having the above configuration, the display device may be a head-mounted display device, the virtual object selection means may be a user focus that can be moved by a user's operation, and the tactile information transmission unit may, when the user focus overlaps with the selectable virtual object, read out tactile information associated with the type of the virtual object from the tactile information storage unit and transmit the tactile information to the tactile presentation device.

[0010] In a virtual object tactile presentation device having the above-described configuration, the displacement resistance generating unit and the displacement unit of the tactile presentation device may form part of a gripper that a user can hold in his or her hand and move freely, the gripper may include a gripper position information detection unit that detects information regarding the position of the gripper, and the display control device may have a focus position determination unit that determines the display position of the user focus on the display based on the information regarding the position detected by the gripper position information detection unit.

[0011] In a virtual object tactile presentation device having the above configuration, the device may further include a hand state detection unit that detects information regarding the state of a user's hand, and the display control device may have a hand gesture recognition unit that recognizes whether or not the user has performed a predetermined hand gesture based on the information detected by the hand state detection unit, and the display control device may perform a predetermined process when the hand gesture recognition unit recognizes that the user has performed a predetermined hand gesture while a selectable virtual object is selected by the virtual object selection means.

[0012] In a virtual object tactile presentation device having the above-described configuration, the tactile presentation device may further have a displacement amount detection unit that detects information about the displacement amount of the displacement unit, and the display control device may perform a predetermined process when the displacement amount detected by the displacement amount detection unit is equal to or greater than a predetermined value when a selectable virtual object is selected by the virtual object selection means.

[0013] In the virtual object tactile presentation device having the above-described configuration, the predetermined process may be a predetermined display performed on a display by a display control unit of the display control device.

[0014] In a virtual object tactile presentation device having the above configuration, the tactile presentation device may further have a displacement amount detection unit that detects information on the displacement amount of the displacement unit, the tactile information storage unit may store tactile information in association with the type of selectable virtual object and the displacement amount of the displacement unit, and the tactile information transmission unit of the display control device may read out tactile information associated with the type of virtual object and the displacement amount of the displacement unit from the tactile information storage unit when a selectable virtual object is selected by the virtual object selection means, and transmit the tactile information to the tactile presentation device.

[0015] In a virtual object tactile presentation device having the above-described configuration, the displacement resistance generating unit of the tactile presentation device may include a rotating unit that rotates in conjunction with the displacement unit, and a rotational resistance generating unit that applies rotational resistance to the rotating unit, and the rotational resistance generating unit may have an internal rotor that rotates with the rotating unit, a magnetorheological fluid sealed in a gap formed around the internal rotor, and an electromagnet for applying a magnetic field to the magnetorheological fluid.

[0016] A program according to one aspect of the present invention is a program for causing a computer to function as the display control device in a virtual object tactile presentation device comprising a display device, a display control device for controlling a display ray of the display device, a tactile presentation device for presenting tactile sensations to a user, and a tactile information storage unit for storing tactile information associated with a type of selectable virtual object, wherein the tactile presentation device has a displacement unit that is displaced by user operation, a displacement resistance generating unit that imparts displacement resistance to the displacement unit, and a displacement resistance control unit that controls the displacement resistance imparted by the displacement resistance generating unit to the displacement unit in accordance with tactile information received from the tactile information transmitting unit, the program causing the display control device to function as a display control unit that displays selectable virtual objects on the display and a virtual object selection means for selecting the virtual object by user operation, and a tactile information transmitting unit that, when a selectable virtual object is selected by the virtual object selection means, reads tactile information associated with the type of virtual object from the tactile information storage unit and transmits it to the tactile presentation device. Effect of the Invention

[0017] According to the present invention, a tactile sensation corresponding to a virtual object can be imparted to a user's hand. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram illustrating an entire virtual object tactile presentation device. [Diagram 2] FIG. 1 is a schematic diagram of a head mounted display device. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of an HMD control device, etc. [Figure 4] 2 is a diagram showing functional blocks of the HMD control device and the tactile presentation device according to the first embodiment. FIG. [Diagram 5] FIG. 1 is a diagram showing an example of a real world and a virtual object displayed on a display. [Figure 6] 1A is a diagram showing an operator manipulating a gripping body of the tactile presentation device as viewed from the left side of the gripping body of the tactile presentation device, and a diagram showing a power supply unit. FIG. [Figure 7] 1A is a diagram showing a state in which an operator operates a gripping body of the tactile presentation device as viewed from the front of the gripping body of the tactile presentation device, and a diagram showing a power supply unit. FIG. [Figure 8] 1 is a bottom view of a gripping body of the tactile presentation device, showing a cross-section of a casing, a boss of a second link member, etc. FIG. [Figure 9] FIG. 2 is a left side view of a gripping body of the tactile presentation device. [Figure 10] FIG. 2 is a bottom view of a gripping body of the tactile presentation device. [Figure 11] FIG. 2 is a right side view of the gripper of the tactile presentation device. [Figure 12] FIG. 2 is a plan view of a gripping body of the tactile presentation device. [Figure 13] FIG. 2 is a left side view of a gripping body of the tactile presentation device. [Figure 14] 3A and 3B are cross-sectional views of a rotational resistance generating portion and a diagram showing a power supply unit. [Figure 15] 13 is a flowchart showing a processing operation executed in the virtual object tactile presentation device. [Figure 16] FIG. 11 is a diagram showing functional blocks of an HMD control device and a tactile presentation device according to a second embodiment. [Figure 17] FIG. 11 is a diagram showing functional blocks of an HMD control device and a tactile presentation device according to a third embodiment. [Figure 18] FIG. 13 is a bottom view of the gripping body of the tactile presentation device according to the third or fourth embodiment, showing a cross-section of the casing, the boss of the second link member, etc. FIG. [Figure 19] FIG. 13 is a diagram showing functional blocks of an HMD control device and a tactile presentation device according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] First Embodiment Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. As shown in Fig. 1 and Fig. 2, a virtual object tactile presentation device 1 according to this embodiment includes a head mounted display device 2, an HMD control device 3, a tactile presentation device 4, etc.

[0020] The head mounted display device 2 comprises a frame unit 7 to be worn on the head of a user 6, and a left eye display 8L and a right eye display 8R (hereinafter collectively referred to as "displays 8") attached to the frame unit 7. In this embodiment, the displays 8L and 8R are transmissive displays capable of displaying a composite image of the real world and a virtual object in the field of view of the user 6. The structure of this transmissive display may be similar to that of a well-known one disclosed in many documents (for example, the transmissive display disclosed in Patent Document 1).

[0021] The HMD control device 3 is a display control device that controls the displays 8L, 8R of the head mounted display device 2. In this embodiment, the HMD control device 3 is attached to a frame portion 7 of the head mounted display device 2. The HMD control device 3 can be configured, for example, by installing an OS (operating system) and an application program on a very small personal computer. For example, HOLOLENS (registered trademark) manufactured by Microsoft Corporation in the United States can be used as the head mounted display device 2 and the HMD control device 3 in this embodiment.

[0022] Fig. 3 shows an example of a schematic configuration of the HMD control device 3. Note that the hardware configuration of the HMD control device 3 shown in Fig. 3 is merely an example, and other hardware configurations can be adopted as long as they can achieve the functions of the HMD control device 3 described later.

[0023] The HMD control device 3 illustrated in Fig. 3 includes a processor 11, a main memory 12, a storage 13, a communication interface 14, and an input / output interface 15 for various devices. The above-mentioned units 11 to 15 are connected to each other via a bus 16. Note that the input / output interface 15 is provided for each of the various devices 8, 17 to 20 described below, but is illustrated collectively as one block in Fig. 3.

[0024] The processor 11 executes a program 10 (such as an OS or an application program) stored in the storage 13 to realize various functions, which will be described later.

[0025] In addition to the program 10, the storage 13 stores a tactile information database 22 (hereinafter also referred to as "tactile information DB 22"). As shown in FIG. 4, the tactile information DB 22 has tactile information for each type of virtual object. More specifically, it has parameter information of a current to be supplied to a rotational resistance generating unit 104 of the tactile presentation device 4 (described later) for each type of virtual object. As this parameter information, a current value, a frequency, and a duty ratio are set for each type of virtual object. Note that the current value, frequency, and duty ratio are examples of parameter information, and other parameter information can be set instead of or in addition to these.

[0026] A wireless communication device 24 is connected to the communication interface 14. The HMD control device 3 can wirelessly communicate with a wireless communication device 203 provided in the tactile presentation device 4 described later via the wireless communication device 24. In this embodiment, wireless communication is performed between both wireless communication devices 24, 203 using Bluetooth (registered trademark). Note that wired communication may be used instead of wireless communication. Also, wireless communication using another communication protocol may be used instead of wireless communication using Bluetooth (registered trademark).

[0027] As shown in FIG. 3, a display 8, an environment recognition camera 17, a depth camera 18, an infrared camera 19, and an inertial measurement unit 20 are connected to the HMD control device 3.

[0028] The display 8 displays virtual objects in the real world based on the image information received from the HMD control device 3.

[0029] The environment recognition camera 17 acquires image information of the spatial environment of the real world, and the depth camera 18 acquires depth of field information. The HMD control device 3 generates detailed three-dimensional spatial information of the real world based on the information acquired from the environment recognition camera 17 and the depth camera 18, and reflects the information in the image information of the displays 8L and 8R.

[0030] The infrared camera 19 acquires infrared information radiated from the surface of an object. The HMD control device 3 acquires information on the state of the user's hand from the infrared camera 19 and the depth camera 18. In this embodiment, the infrared camera 19 and the depth camera 18 configure a hand state detection unit that detects information on the state of the user's hand.

[0031] The inertial measurement unit 20 includes inertial sensors such as a three-axis magnetic sensor, a three-axis gyro sensor, and a three-axis acceleration sensor. The inertial measurement unit 20 senses the position, orientation, and acceleration of the head mounted display device 2. The HMD control device 3 identifies the direction of the user's line of sight based on the information acquired from the inertial measurement unit 20, and displays the user focus 28 on the display 8 at a position where the user's line of sight intersects with the real world or a virtual object (see FIG. 5). The display form of the user focus 28 differs depending on whether it overlaps with a selectable object or not. The user focus 28 of this embodiment has a doughnut shape when it overlaps with a selectable object, and has a semi-transparent circle shape when it does not overlap with a selectable object. In the display example of the display 8 shown in FIG. 5, only a plaster statue is displayed as the selectable object 38, and a table 39, a window 40, a wall 41, a floor 42, and the like are displayed as real world objects.

[0032] Next, a description will be given of functions realized in the HMD control device 3 by the processor 11 executing a program. By executing the program, the processor 11 functions as a focus position determination unit 31, a display control unit 32, an overlap determination unit 33, a hand gesture recognition unit 34, and a tactile information transmission unit 35, as shown in FIG.

[0033] Focus position determination unit 31 calculates the direction of the user's line of sight (actually, the direction in which the user's face faces) based on information acquired from inertial measurement unit 20, and determines the position where the real world or a virtual object intersects with the user's line of sight in the field of view of display 8 as the position of user focus 28. Of course, the position of user focus 28 moves as the direction of the user's line of sight changes.

[0034] 5, the display control unit 32 displays a virtual object 38 on the display 8, and also displays the user focus 28 at a position determined by the focus position determination unit 31. When the user performs a predetermined hand gesture with the user focus 28 overlapping the virtual object 38, a predetermined display is performed on the display 8. Examples of the predetermined display include deformation of the virtual object, disappearance of the virtual object, display of another virtual object, and the like.

[0035] The overlap determination unit 33 determines whether the user focus 28 overlaps with a selectable virtual object 38 or not.

[0036] The hand gesture recognition unit 34 recognizes whether or not the user has made a predetermined hand gesture based on information about the state of the user's hand detected by the infrared camera 19 and the depth camera 18 (detected by the hand state detection unit).

[0037] When the user focus 28 overlaps with a selectable virtual object 38, the tactile information transmission unit 35 reads out tactile information associated with the virtual object 38 from the tactile information DB 22 and transmits it to the tactile presentation device 4. When the hand gesture recognition unit 34 recognizes that the user has performed a predetermined hand gesture, the tactile information transmission unit 35 transmits a command to the tactile presentation device 4 to stop presenting a tactile sensation. Note that in this embodiment, the predetermined hand gesture is an action of tilting the index finger from an upright position (air tap).

[0038] -Explanation of the tactile presentation device- Next, a description will be given of the tactile presentation device 4. The tactile presentation device 4 in this embodiment includes a gripping body 100 that a user can hold in his / her hand and operate while moving it freely, and a power supply unit 200, as shown in FIG.

[0039] The grip body 100 is formed in a shape and size that fits in the user's hand, as shown in Fig. 6 or 7. Therefore, the user can operate the grip body 100 while holding it in his / her hand. The grip body 100 presents a tactile sensation to the user at the displacement portion 102 according to the value of the current supplied from the power supply unit 200.

[0040] 8 to 13, the gripping body 100 includes a displacement portion 102, a rotating shaft 103, a rotational resistance generating portion (displacement resistance generating portion) 104, and a casing 105. The gripping body 100 also includes two holding portions 106A and 106B.

[0041] The displacement part 102 is provided so as to be displaced relative to the casing 105 by the user's operation. In this embodiment, the displacement part 102 rotates about a rotation axis 108 provided at one end to move toward or away from the casing 105. One end of the displacement part 102 is supported by a support shaft 109 so as to be freely rotatable. As shown in FIG. 9, the displacement part 102 is configured to include a curved plate curved in an arc shape. When the displacement part 102 is rotated by the user's operating force, the operating force is transmitted via a link mechanism 110 to the rotating shaft 103, which is a rotating part.

[0042] As shown in Fig. 9 and Fig. 13, the displacement part 102 is configured to be rotatable within a predetermined range. In this embodiment, the displacement part 102 is not provided with a return spring or the like. Therefore, when using the grip 100, it is preferable to fasten the fingertip of the user to the displacement part 102 using, for example, a fastener fastener 112, as shown in Fig. 6 or 7. This allows the user to push the displacement part 102 into the casing 105 with his / her fingertip and then pull it back to the original position.

[0043] The link mechanism 110 is provided to link the displacement portion 102 and the rotating shaft 103. As shown in FIG. 9, the link mechanism 110 is composed of a first link member 114 and a second link member 115. One end of the first link member 114 is rotatably connected to the back surface of the displacement portion 102, and the other end is rotatably connected to the tip of the second link member 115. In this embodiment, as shown in FIG. 7, two first link members 114 are provided and rotatably connected to both sides of the second link member 115. As shown in FIG. 8, the base of the second link member 115 is connected to the rotating shaft 103 via a one-way clutch 116. A boss portion 115a is formed at the base of the second link member 115, and this boss portion 115a is supported in the casing 105 via a bearing 117. In addition, the tip of the rotating shaft 103 is supported inside the boss portion 115a via a bearing 120. The rotating shaft 103 is composed of a solid shaft 103a protruding from the rotational resistance generating part 104 and a hollow shaft 103b fitted onto the solid shaft 103a so as to rotate integrally with the solid shaft 103a, but the rotating shaft 103 may be composed of a single member. Hereinafter, only the "solid shaft 103a" may be simply referred to as the "rotating shaft 103".

[0044] When the user pushes the displacement portion 102 toward the casing 105, the one-way clutch 116 transmits the operation force transmitted from the displacement portion 102 via the link mechanism 110 to the rotating shaft 103 and the rotational resistance generating portion 104. Therefore, when the user pulls the displacement portion 102 back from the casing 105, the one-way clutch 116 does not transmit the operation force transmitted from the displacement portion 102 via the link mechanism 110 to the rotating shaft 103 and the rotational resistance generating portion 104.

[0045] The rotational resistance generating unit 104 applies a rotational resistance (displacement resistance) to the displacement unit 102 by applying a rotational resistance to the rotation shaft 103. In this embodiment, the rotational resistance generating unit 104 applies a magnetic field having a strength corresponding to the magnitude of a current supplied from the outside to a magnetorheological fluid, thereby applying a rotational resistance corresponding to the magnitude of the current to the rotation shaft 103. The outer shape of the rotational resistance generating unit 104 is formed into a substantially cylindrical shape, and the rotation shaft 103 protrudes from one end face 118 of the cylindrical shape. A specific example of the internal structure of the rotational resistance generating unit 104 will be described in detail later. As shown in FIG. 8, the axis 103a of the rotation shaft 103 and the rotation axis 108 of the displacement unit are substantially parallel to each other.

[0046] The casing 105 has a housing portion 107 therein. The housing portion 107 houses the rotating shaft 103, the rotational resistance generating portion 104, a boss portion 115a of the second link member 115, a bearing 117, etc. Of the housing portion 107, a portion 107a that houses the rotational resistance generating portion 104 has a shape that allows the outer circumferential portion 133a of the rotational resistance generating portion 104 to fit therein. A window 105a is formed on one side of the casing 105, and the other end surface 119 of the rotational resistance generating portion 104 is exposed from the window 105a. The other end surface 119 of the rotational resistance generating portion 104 is flat.

[0047] The outer shape of the casing 105 is formed in a substantially cylindrical shape as shown in Fig. 9, and a pair of support arms 113 extend outward from a part of the casing 105 as shown in Figs. 9 and 10. The aforementioned support shaft 109 is installed at the tip of the pair of support arms 113. The casing 105 is also provided with an opening 105b for avoiding interference with the link mechanism 110 (see Figs. 7 and 10). The casing 105 in this embodiment is constructed by fastening two casing members 105A and 105B with bolts 134 as shown in Fig. 10.

[0048] The gripping portions 106A and 106B are portions of the gripping body 100 that a user can pinch and hold with his / her fingers while operating the displacement portion 102. The gripping body 100 according to this embodiment is provided with at least two gripping portions 106A and 106B.

[0049] The first holding portion 106A is formed on both sides of the casing 105. In this embodiment, as shown in FIG. 12, both sides of the casing 105 exist on both sides of the direction in which the center line 102a of the displacement portion 102 rotates. In other words, both sides of the casing 105 exist on both sides of a plane including the trajectory drawn by the center line 102a of the displacement portion 102 when the displacement portion 102 is displaced. In addition, flat surfaces are provided on both sides of the casing 105. One flat surface is formed only by the side surface 121 of the casing, and the other flat surface is formed including the other end surface 119 of the rotational resistance generating portion 104. Both the side surface 121 of the casing 105 and the other end surface 119 of the rotational resistance generating portion 104 are approximately parallel to the direction in which the center line 102a of the displacement portion 102 rotates.

[0050] As shown in FIG. 7, a user can hold the gripping portion 106A between thumb 123 and middle finger 124 and place the index finger 125 along the displacement portion 102, thereby operating the displacement portion 102 with the index finger 125 while holding the gripping body 100 in the hand.

[0051] 6, the second holding portion 106B is formed by the displacement portion 102 and a side portion 126 of the casing 105 opposite the displacement portion 102. In this case, the user can pinch and hold the grip body 100 by placing the thumb 123 along the side portion 126 of the casing 105 opposite the displacement portion 102 and the index finger 125 along the displacement portion 102. In this case as well, the user can operate the displacement portion 102 with the index finger 125 while holding the grip body 100 in the hand.

[0052] Next, a configuration example of the rotational resistance generating part 104 will be described with reference to Fig. 14. The rotational resistance generating part 104 shown in Fig. 14 is made up of a disk 128, yokes 129 and 130, a coil 131, a magnetorheological fluid 132, a case 133, and the like.

[0053] The disk 128 is an internal rotor that rotates inside the rotational resistance generating unit 104 and is made of a magnetic material. The rotating shaft 103 is connected vertically to the center of the back surface 128b of the disk 128. Therefore, the disk 128 and the rotating shaft 103 rotate together. The rotating shaft 103 is supported in a shaft hole 136 provided in the yoke 130 via a bearing 135. It is preferable that the rotating shaft 103 is made of a non-magnetic material.

[0054] The yoke is composed of a first yoke 129 and a second yoke 130. The first yoke 129 is opposed to the surface 128a of the disk 128 via a small gap and is composed of a disk-shaped member. The first yoke 129 is fitted into and fixed in a cylindrical case 133.

[0055] The second yoke 130 has an opposing surface 130a that faces the rear surface 128b of the disk 128 via a small gap. The second yoke 130 is fitted and fixed inside a cylindrical case 133. The case 133, which is the outer periphery of the rotational resistance generating part 104, is fixed so as not to rotate with respect to the casing 105 (see FIG. 8). In this embodiment, as shown in FIG. 9, the case 133 is fixed to the casing 105 by using an enamel set 127.

[0056] A sphere 137 made of a non-magnetic material is accommodated in the space defined by a recess formed in the center of first yoke 129 and a recess formed in the center of the end face of rotating shaft 103 .

[0057] The coil 131 is disposed along an annular groove formed in the second yoke 130. A current is supplied to the coil 131 from a power supply control section 202, which will be described later.

[0058] The magnetorheological fluid 132 is sealed in the gap between the disk 128 and the first and second yokes 129 and 130. The magnetorheological fluid 132 is a liquid in which magnetic particles are dispersed in a dispersion medium. For example, nano-sized metal particles (metal nanoparticles) can be used as the magnetic particles. The magnetic particles are made of a magnetizable metal material, and although there is no particular limitation on the metal material, a soft magnetic material is preferable. Examples of the soft magnetic material include alloys such as iron, cobalt, nickel, and permalloy. Although there is no particular limitation on the dispersion medium, an example of the soft magnetic material is hydrophobic silicone oil. The blending amount of the magnetic particles in the magnetorheological fluid can be, for example, 3 to 40 vol%. It is also possible to add various additives to the magnetorheological fluid in order to obtain various desired characteristics.

[0059] In the rotational resistance generating unit 104 having the above configuration, when a current is applied to the coil 131, a magnetic path is formed in the disk 128, the first yoke 129, and the second yoke 130 along the direction indicated by the arrow P in FIG. 14, for example. This magnetic path penetrates the magnetorheological fluid 132 disposed in the gap between the front surface 128a of the disk 128 and the first yoke 129, and the magnetorheological fluid 132 disposed in the gap between the back surface 128b of the disk 128 and the second yoke 130. As a result, the magnetorheological fluid 132 exhibits a viscosity (shear stress) according to the strength of the magnetic field, and the torque transmitted between the disk 128 and the yokes 129, 130 increases according to the strength of the magnetic field. That is, the rotational resistance generating unit 104 applies a rotational resistance to the rotating shaft 103 whose magnitude corresponds to the value of the current applied to the coil 131.

[0060] Next, the power supply unit 200 will be described. The power supply unit 200 is preferably a portable power supply unit that can be attached to a part of the user's body (the waist in the example shown in FIG. 1), for example. As shown in FIG. 6, the power supply unit 200 includes a battery 201, a power supply control unit 202, and a wireless communication device 203. The power supply control unit 202 controls the value of the current supplied from the battery 201 to the coil 131 of the rotational resistance generating unit 104 based on the tactile information (current parameter information) of the virtual object received from the HMD control device 3 via the wireless communication device 203. The power supply control unit 202 functions as a displacement resistance control unit that controls the displacement resistance applied to the displacement unit 102 by controlling the value of the current supplied to the coil 131.

[0061] Based on tactile information (current parameter information) received from the HMD control device 3, the power supply control unit 202 controls the current value supplied to the coil 131 to vary between 0 A and the current value specified in the parameter information in accordance with the frequency and duty ratio specified in the parameter information.

[0062] When using the tactile presentation device 4 having the above configuration, as shown in FIG. 6 or FIG. 7, the user presses or pulls back the displacement unit 102 with the index finger 125 while pinching the gripper 100 with his or her fingers. When the displacement unit 102 is pressed, the operating force received by the displacement unit 102 is transmitted to the rotational resistance generating unit 104 via the one-way clutch 116, so that resistance is generated against the movement of pressing the displacement unit 102 according to the current value applied to the coil 131, and the resistance is presented to the user as a tactile sensation. Conversely, when pulling back the displacement unit 102, the operating force received by the displacement unit 102 is not transmitted from the one-way clutch 116 to the rotational resistance generating unit 104 side, so that almost no resistance is generated against the movement of pulling back the displacement unit 102. Therefore, the user can pull back the displacement unit 102 with a light force.

[0063] Next, the processing operations executed in the virtual object tactile presentation device 1 will be described with reference to the flowchart of FIG.

[0064] When the display control unit 32 displays a virtual object on the display 8 by blending it with the real world (S1) and displays the user focus 28 (S2), the overlap determination unit 33 repeatedly determines whether the user focus 28 overlaps with a selectable virtual object 38 (S3).

[0065] When the overlay determination unit 33 determines that the user focus 28 overlaps with a selectable virtual object 38 (S3: YES), the tactile information transmission unit 35 reads out current parameter information, which is tactile information associated with the virtual object 38, from the tactile information DB 22 and transmits it to the tactile presentation device 4 via the wireless communication device 24 (S4).

[0066] On the other hand, when the parameter information of the current transmitted by the tactile information transmitting unit 35 is received by the power supply control unit 202 via the wireless communication device 203 on the tactile presentation device 4 side (T1), the power supply control unit 202 (displacement resistance control unit) controls the current value supplied from the battery 201 to the coil 131 of the rotational resistance generating unit 104 according to the parameter information of the current (T2). As a result, the rotational resistance generating unit 104 applies a magnetic field of a strength according to the current value supplied from the power supply control unit 202 to the magnetorheological fluid. As a result, the rotational resistance generating unit 104 applies a rotational resistance according to the current value to the rotating shaft 103, and the rotational resistance appears as a displacement resistance in the displacement unit 102 via the link mechanism 110. Then, the displacement resistance is presented as a tactile sensation when the user presses the displacement unit 102. The tactile sensation presented when the displacement unit 102 is pressed in this way varies depending on the current value, frequency, duty ratio, etc. of the current supplied to the coil 131 of the rotational resistance generating unit 104.

[0067] Following the process of S4, on the HMD control device 4 side, the hand gesture recognition unit 34 repeatedly recognizes whether or not the user has performed a predetermined hand gesture (air tap), and when the hand gesture recognition unit 34 recognizes that the predetermined hand gesture has been performed (S5: YES), the display control unit 32 performs a predetermined display process on the display 8 (S6). Examples of the predetermined display process include deformation of the selected virtual object, disappearance of the selected virtual object, and additional display of another virtual object. After that, the tactile information transmission unit 35 transmits a command to the tactile presentation device 4 to stop presenting the tactile sensation (S7).

[0068] On the other hand, when the command to stop the presentation of the tactile sensation transmitted by the tactile information transmission unit 35 is received by the power supply control unit 202 via the wireless communication device 203 on the tactile presentation device 4 side (T3), the power supply control unit 202 (displacement resistance control unit) stops the power supply to the coil 131 of the rotational resistance generation unit 104 (T4). This causes the presentation of the tactile sensation in the displacement unit 102 to be stopped.

[0069] According to the virtual object tactile presentation device 1 described above, a tactile sensation corresponding to the virtual object can be presented to the user's hand by the user placing the user focus 28 on the virtual object 38 and pressing the displacement unit 102 of the tactile presentation device 4. In addition, since the operation of pressing the displacement unit 102 of the tactile presentation device 4 is the same as a predetermined hand gesture (air tap), the user can receive a tactile sensation while selecting and operating a virtual object.

[0070] <Second embodiment> A second embodiment of the present invention will be described below. In the second embodiment, differences from the first embodiment will be mainly described, and descriptions of configurations and functions similar to those of the first embodiment will be omitted.

[0071] 16, the virtual object tactile presentation device according to the second embodiment includes a gripper position detection unit 150 that detects information related to the position of the gripper 100 in the gripper 100. As the gripper position detection unit 150, for example, an inertial measurement unit similar to the inertial measurement unit 20 described above can be used. In addition, the power supply unit 200 is further provided with a position information transmission unit 250 that transmits information detected by the gripper position detection unit 150 to the HMD control device 3 side via a wireless communication device 203.

[0072] The focus position determination unit 31A of the HMD control device 3 determines the position of the user focus 28 on the display 8 based on information about the position of the grip body 100 acquired from a grip body position detection unit 150 included in the grip body 100. That is, in this embodiment, the user focus 28 can be moved by moving the hand of the user holding the grip body 100, rather than by moving the user's line of sight.

[0073] <Third embodiment> A third embodiment of the present invention will be described below. In the third embodiment, differences from the first embodiment will be mainly described, and descriptions of configurations and functions similar to those of the first embodiment will be omitted.

[0074] 17, the virtual object tactile presentation device according to the third embodiment is provided with a displacement amount detection unit 111 that detects information on the amount of displacement of the displacement unit 102 from a predetermined position (in this embodiment, the position where the displacement unit 102 is pulled back the most) on the gripper 100 of the tactile presentation device 4. Also, the power supply unit 200 is provided with a displacement amount information transmission unit 251 that transmits information on the amount of displacement detected by the displacement amount detection unit 111 to the HMD control device 3 side via the wireless communication device 203.

[0075] The processor 11 of the HMD control device 3 also functions as a displacement amount determination unit 36 ​​that determines whether or not the displacement amount is equal to or greater than a predetermined value, based on information on the displacement amount of the displacement unit 102 of the tactile presentation device 4 acquired from the displacement amount detection unit 111. Then, when the displacement amount determination unit 36 ​​determines that the displacement amount of the displacement unit 102 is equal to or greater than a predetermined value with the user focus 28 overlapping with a selectable virtual object 38, the display control unit 32 performs a predetermined display process on the display 8. Examples of the predetermined display process include deformation of the selected virtual object, disappearance of the selected virtual object, and additional display of another virtual object.

[0076] A specific example of the displacement amount detection unit 111 is shown in Fig. 18. For example, an encoder, a variable resistor, or the like can be used as the displacement amount detection unit 111. The displacement amount detection unit 111 shown in the figure is fitted into a predetermined portion 107b of the housing portion 107 of the casing 105, and its input portion is connected to a boss portion 115a of the second link member 115 via a connecting member 122. The boss portion 115a of the second link member 115 rotates in conjunction with the displacement of the displacement portion 102, so the displacement amount detection unit 111 detects the amount of rotation, thereby detecting the amount of displacement of the displacement portion 102.

[0077] <Fourth embodiment> A fourth embodiment of the present invention will be described below. In the fourth embodiment, differences from the first embodiment will be mainly described, and descriptions of configurations and functions similar to those of the first embodiment will be omitted.

[0078] 19, the virtual object tactile presentation device according to the fourth embodiment is provided with a displacement amount detection unit 111 that detects information on the amount of displacement of the displacement unit 102 from a predetermined position (in this embodiment, the position where the displacement unit 102 is pulled back the most) on the gripper 100 of the tactile presentation device 4. Also, the power supply unit 200 is provided with a displacement amount information transmission unit 251 that transmits information on the amount of displacement detected by the displacement amount detection unit 111 to the HMD control device 3 side via the wireless communication device 203 (the above points are the same as those in the third embodiment).

[0079] 19, the tactile information DB 22A stores parameter information of the current supplied to the rotational resistance generating unit 104 of the tactile presentation device 4 in association with the type of selectable virtual object and the displacement amount of the displacement unit 102. As this parameter information, a current value, a frequency, and a duty ratio are set for each type of virtual object and each displacement amount. In this embodiment, three types of displacement amounts, current values, frequencies, and duty ratios are stored for each type of virtual object.

[0080] When the user focus 28 overlaps with a selectable virtual object 38, the tactile information transmission unit 35A of the HMD control device 3 reads tactile information (current parameter information) associated with the type of selected virtual object 38 and the information on the displacement amount of the displacement unit 102 sent from the tactile presentation device 4 from the tactile information DB 22A, and transmits it to the tactile presentation device 4.

[0081] According to this embodiment, three types of current parameter information are set for each type of virtual object, so that when the user presses the displacement portion 102, three types of haptics are sequentially presented depending on the amount of pressing.

[0082] <Other Modifications> In this embodiment, a transmissive display is used that can display a composite image of the real world and virtual objects in the user's field of vision, but it is also possible to use a non-transmissive display that does not transmit images of the real world and only displays virtual objects.

[0083] Moreover, a normal stationary display device may be adopted instead of the head mounted display device 2 in this embodiment. The display of this stationary display device may be a transmissive display that can display a composite image of the real world and virtual objects in the user's field of vision, or a non-transmissive display that does not transmit images of the real world and displays only virtual objects.

[0084] When a normal installed display is employed, information on the user's line of sight (the direction in which the user's face faces), which is necessary information for displaying the user focus 28 (virtual object selection means) on the display, can be obtained from the inertial measurement unit 20 by mounting the inertial measurement unit 20 on the head of the user 6 by some mounting means. Note that the method of obtaining information on the user's line of sight is not limited to the above, and can be obtained by various known methods. [Explanation of symbols]

[0085] 1. Virtual object tactile presentation device 2 Head-mounted display device (display device) 3 HMD control device (display control device) 4. Tactile presentation device 8,8L,8R Display 10 Programs 11 Processors 13. Storage 17. Environment Recognition Camera 18 Depth Camera 19 Infrared Camera 20 Inertial Measurement Unit 22 Tactile Information Database 28 User Focus 31 Focus position determination unit 32 Display control section 33 Overlap determination unit 34 Hand Gesture Certification Department 35 Tactile information transmission unit 36 Displacement determination section 38 Virtual Objects 100 Grip 102 Displacement section 103 Rotating shaft (rotating part) 104 Rotational resistance generating part 111 Displacement detection unit 128 Disc (inner rotor) 132 Magnetorheological Fluid 150 Grip position detection unit 200 Power Supply Unit 250 Location information transmission unit 251 Displacement information transmission unit

Claims

1. A display device; a display control device for controlling a display of the display device; A tactile presentation device that presents a tactile sensation to a user; a haptic information storage unit that stores haptic information in association with a type of selectable virtual object; A virtual object tactile presentation device comprising: The display control device includes: a display control unit that displays selectable virtual objects on the display and displays a user focus that does not exist in the real world and that can be moved by the user's operation as a virtual object selection means for selecting the virtual object through a user's operation; and a tactile information transmission unit that, when the user focus overlaps with the selectable virtual object and the selectable virtual object is selected, reads tactile information associated with the type of the virtual object from the tactile information storage unit and transmits the tactile information to the tactile presentation device; having The tactile presentation device comprises: A displacement unit that is displaced by a user's operation; a displacement resistance generating section that applies a displacement resistance to the displacement section; a displacement resistance control unit that controls the displacement resistance applied to the displacement unit by the displacement resistance generation unit in accordance with the tactile information received from the tactile information transmission unit; having the displacement resistance generating unit and the displacement unit of the tactile presentation device constitute a part of a gripping body that can be held in a user's hand and moved freely, The gripper includes a gripper position information detection unit that detects information regarding a position of the gripper in a three-dimensional space, the display control device includes a focus position determination unit that determines a display position of the user focus on the display based on information about a position in a three-dimensional space detected by the gripper position information detection unit, The grip does not include a means for detecting the movement of the grip on a flat surface such as a desk, which is included in a mouse that is an input device for a computer; the tactile presentation device further includes a displacement amount detection unit that detects information on a displacement amount of the displacement unit, the display control device performs a predetermined process when the displacement amount detected by the displacement amount detection unit is equal to or greater than a predetermined value in a state in which a selectable virtual object is selected by the user focus. A virtual object tactile presentation device comprising:

2. A display device; a display control device for controlling a display of the display device; A tactile presentation device that presents a tactile sensation to a user; a haptic information storage unit that stores haptic information in association with a type of selectable virtual object; A virtual object tactile presentation device comprising: The display control device includes: a display control unit that displays selectable virtual objects on the display and displays a user focus that does not exist in the real world and that can be moved by the user's operation as a virtual object selection means for selecting the virtual object through a user's operation; and a tactile information transmission unit that, when the user focus overlaps with the selectable virtual object and the selectable virtual object is selected, reads tactile information associated with the type of the virtual object from the tactile information storage unit and transmits the tactile information to the tactile presentation device; having The tactile presentation device comprises: A displacement unit that is displaced by a user's operation; a displacement resistance generating section that applies a displacement resistance to the displacement section; a displacement resistance control unit that controls the displacement resistance applied to the displacement unit by the displacement resistance generation unit in accordance with the tactile information received from the tactile information transmission unit; having the displacement resistance generating unit and the displacement unit of the tactile presentation device constitute a part of a gripping body that can be held in a user's hand and moved freely, The gripper includes a gripper position information detection unit that detects information regarding a position of the gripper in a three-dimensional space, the display control device includes a focus position determination unit that determines a display position of the user focus on the display based on information about a position in a three-dimensional space detected by the gripper position information detection unit, The grip does not include a means for detecting the movement of the grip on a flat surface such as a desk, which is included in a mouse that is an input device for a computer; the tactile presentation device further includes a displacement amount detection unit that detects information on a displacement amount of the displacement unit, the haptic information storage unit stores haptic information in association with a type of selectable virtual object and a displacement amount of the displacement unit; a tactile information transmission unit of the display control device, when a selectable virtual object is selected by the user focus, reads out tactile information associated with the type of the virtual object and the displacement amount of the displacement unit from the tactile information storage unit, and transmits the tactile information to the tactile presentation device. A virtual object tactile presentation device comprising:

3. A display device; a display control device for controlling a display of the display device; A tactile presentation device that presents a tactile sensation to a user; a haptic information storage unit that stores haptic information in association with a type of selectable virtual object; A virtual object tactile presentation device comprising: The display control device includes: a display control unit that displays selectable virtual objects on the display and displays a user focus that does not exist in the real world and that can be moved by the user's operation as a virtual object selection means for selecting the virtual object through a user's operation; and a tactile information transmission unit that, when the user focus overlaps with the selectable virtual object and the selectable virtual object is selected, reads tactile information associated with the type of the virtual object from the tactile information storage unit and transmits the tactile information to the tactile presentation device; having The tactile presentation device comprises: A displacement unit that is displaced by a user's operation; a displacement resistance generating section that applies a displacement resistance to the displacement section; a displacement resistance control unit that controls the displacement resistance applied to the displacement unit by the displacement resistance generation unit in accordance with the tactile information received from the tactile information transmission unit; having the displacement resistance generating unit and the displacement unit of the tactile presentation device constitute a part of a gripping body that can be held in a user's hand and moved freely, The gripper includes a gripper position information detection unit that detects information regarding a position of the gripper in a three-dimensional space, the display control device includes a focus position determination unit that determines a display position of the user focus on the display based on information about a position in a three-dimensional space detected by the gripper position information detection unit, The grip does not include a means for detecting the movement of the grip on a flat surface such as a desk, which is included in a mouse that is an input device for a computer; The displacement resistance generating unit of the tactile presentation device is A rotating part that rotates in conjunction with the displacement part; a rotational resistance generating unit that applies a rotational resistance to the rotating unit; Including, The rotational resistance generating portion includes: an inner rotor that rotates together with the rotating portion; a magnetorheological fluid sealed in a gap formed around the inner rotor; an electromagnet for applying a magnetic field to the magnetorheological fluid; having A virtual object tactile presentation device comprising:

4. The virtual object tactile presentation device according to any one of claims 1 to 3, The display device is a head-mounted display device. A virtual object tactile presentation device comprising:

5. The virtual object tactile presentation device according to any one of claims 1 to 4, A virtual object tactile presentation device, wherein the gripper position information detection unit is an inertial measurement unit.

Images