Tactile presentation system, tactile presentation method, and program

Abstract

The present invention applies a stimulus suitable for an operation in a virtual space to the sole of a foot. This tactile presentation system comprises: a left stimulation unit and a right stimulation unit each including a plurality of movable contact portions provided apart from each other and an actuator that individually moves the plurality of movable contact portions, each of the plurality of movable contact portions having a surface configured to contact any portion of the sole and moving in a direction along the surface; and a control unit that, when a motion for moving a user in a virtual space is started, controls movement of the plurality of movable contact portions on the basis of control information stored in association with a type of the motion.

Description

Haptic Presentation System, Haptic Presentation Method, and Program

[0001] The present invention relates to a haptic presentation system, a haptic presentation method, and a program.

[0002] For example, in the user interface in VR, the provision of vision and hearing is central. Therefore, the feeling of walking or running has been expressed by images and sounds. In order to further express the feeling of walking or running, devices that stimulate the tactile sensation of the sole of the foot have been studied.

[0003] In the following literature, it is described that the tactile sensation of the sole of the foot is stimulated in accordance with walking by applying a shearing force to the sole of the foot.

[0004] G. Kato, Y. Kuroda, K. Kiyokawa and H. Takemura, "Force Rendering and its Evaluation of a Friction Based Walking Sensation Display for a Seated User," in IEEE Transactions on Visualization and Computer Graphics, vol. 24, no. 4, pp. 1506 1514, April 2018, doi: 10.1109 / TVCG.2018.2793641.

[0005] By applying a shearing force to the sole of the foot, it is possible to present a tactile sensation without moving the foot body very much, but the variations of the tactile sensation given are few. For example, with a conventional device that applies a shearing force, it is only possible to control the presence or absence and intensity of the stimulus at the timing when the entire foot touches the ground, and it is difficult to adjust the stimulus according to the operation in the virtual space.

[0006] One of the objects of the present disclosure is to provide a technology that enables a stimulus suitable for an operation in a virtual space to be applied to the sole of the foot.

[0007] (1) A tactile presentation system comprising: a left stimulator and a right stimulator, each including a plurality of movable contact parts that are spaced apart from each other and each having a surface configured to contact any part of the sole of the foot and move in a direction along said surface, and actuators that move the plurality of movable contact parts individually; and a control unit that controls the movement of the plurality of movable contact parts based on control information stored in association with the type of motion when a motion that moves a user in a virtual space is started.

[0008] (2) In (1), the actuator moves each of the plurality of movable contact parts in two different directions along the surface, a tactile presentation system.

[0009] (3) A tactile presentation system in (1) or (2), wherein the plurality of movable contact parts include at least a portion of a first contact part configured to contact the toes, a second contact part configured to contact the base of the big toe, a third contact part configured to contact the base of a toe other than the big toe, and a fourth contact part configured to contact the heel.

[0010] (4) In (3), the control unit, when the type of motion to be started is running, moves the fourth contact part forward in response to the heel making contact with the ground in the virtual space, then moves the second contact part backward and then moves the second contact part forward, a tactile presentation system.

[0011] (5)(4) The control unit, when the type of motion to be started is running, moves the fourth contact part forward in response to the heel making contact with the ground in the virtual space, then moves the second contact part backward, moves the second contact part forward, and moves the third contact part inward and diagonally forward, a tactile presentation system.

[0012] (6) In any of (1) to (5), the control unit moves at least a portion of the plurality of movable contact parts forward before the foot leaves the virtual space when the type of motion to be initiated is a forward jump, and moves at least a portion of the plurality of movable contact parts backward in response to the foot landing in the virtual space, a haptic presentation system.

[0013] (7) In any of (1) to (6), the control unit moves at least a portion of the plurality of movable contacts included in the left stimulator and the right stimulator in a direction corresponding to the direction of the curve, and the movement of the plurality of movable contacts in the right stimulator and the left stimulator that corresponds to the inside of the curve is greater than the movement in the one that corresponds to the outside of the curve, a tactile presentation system.

[0014] (8) In any of (1) to (7), the control unit moves at least a portion of the plurality of movable contact parts included in the left stimulator and the right stimulator that corresponds to the foot making contact, in a direction along the direction of movement after the reversal, in response to the foot making contact with the ground in the virtual space, when the motion to be started includes a reversal of the direction of movement in the lateral direction.

[0015] (9) In any of (1) to (8), the control unit moves at least a portion of the plurality of movable contacts included in one of the left stimulator and the right stimulator toward the other of the left stimulator and the right stimulator, and then moves at least a portion of the plurality of movable contacts included in the other of the left stimulator and the right stimulator toward the one of the left stimulator and the right stimulator, in the case that the type of motion to be started is a tennis forehand stroke, a tactile presentation system.

[0016] In (10) and (9), the control unit, when the type of motion to be started is a tennis forehand stroke, moves at least a portion of the plurality of movable contacts included in one of the left stimulator and the right stimulator in a first direction toward the other of the left stimulator and the right stimulator, and moves the plurality of movable contacts included in the other of the left stimulator and the right stimulator that correspond to the toes in the first direction, and then moves at least a portion of the plurality of movable contacts included in the other of the left stimulator and the right stimulator in a second direction toward the one of the left stimulator and the right stimulator, and moves the plurality of movable contacts included in one of the left stimulator and the right stimulator that correspond to the thumb in the first direction, a tactile presentation system.

[0017] (11) A tactile presentation system in which, in any of (1) to (10), each of the left stimulator and the right stimulator further includes a mechanism that moves the plurality of movable contact parts up and down individually based on the control of the control unit.

[0018] (12)(11) The control unit, when the type of motion to be started is walking on mud, moves the plurality of movable contact parts along the surface and lowers them in accordance with the contact of the foot in the virtual space, a tactile presentation system.

[0019] (13)(11) The control unit raises a portion of the plurality of movable contact parts in response to the contact of a portion of the foot in the virtual space when the type of motion to be started is walking on a rock, a tactile presentation system.

[0020] (14)(13) The control unit raises the movable contact portion of the plurality of movable contact portions that corresponds to the portion of the foot that contacts the protrusion in the virtual space when the type of motion to be started is walking on a rock, a tactile presentation system.

[0021] (15) A tactile presentation method comprising a left stimulator and a right stimulator, each comprising a plurality of movable contact parts provided spaced apart from each other and each having a surface configured to contact any part of the sole of the foot and moving in a direction along said surface, and actuators for individually moving the plurality of movable contact parts, the tactile presentation method comprising the steps of: acquiring control information stored in association with the type of motion when a motion that moves a user in a virtual space is started, and controlling the movement of the plurality of movable contact parts based on the acquired control information.

[0022] (16) A program that causes a computer to execute a process in which, when a motion that moves a user in a virtual space is initiated, it acquires control information stored in association with the type of motion, and based on the acquired control information, controls the movement of a plurality of movable contact parts, each having a surface that is spaced apart from each other and configured to contact any part of the sole of the foot, and which moves in a direction along that surface, by using actuators that move each of the plurality of movable contact parts included in the left stimulation part and the right stimulation part individually.

[0023] This figure shows an example of the configuration of a tactile feedback system according to an embodiment of the present invention. This figure schematically shows an example of a right foot stimulator. This figure shows an example of the arrangement of multiple movable contact parts in a right foot stimulator. This is a flowchart showing an example of the processing of the tactile feedback system. This figure shows an example of control information. This figure illustrates the movement of the movable contact parts at the start of walking or running. This figure illustrates the movement of the movable contact parts in the middle of walking. This figure illustrates the movement of the movable contact parts in the middle of running. This figure illustrates the movement of the movable contact parts at the stop of walking. This figure illustrates the movement of the movable contact parts when running on a right curve. This figure illustrates the movement of the movable contact parts when the foot applies a strong lateral force to the ground. This figure illustrates the movement of the movable contact parts in a tennis forehand stroke. This figure illustrates the movement of the movable contact parts when jumping forward. This figure illustrates the movement of the movable contact parts when walking on mud. This figure illustrates the movement of the movable contact parts when walking on rocks. This figure shows an example of the arrangement of xy stages that operate by screw drive.

[0024] Embodiments of the present invention will be described below with reference to the drawings. Components having the same function will be denoted by the same reference numeral, and their descriptions will be omitted.

[0025] Figure 1 shows an example of the configuration of a tactile presentation system according to an embodiment of the present invention. The tactile presentation system according to the present invention includes a control device 10, a display device 20, a left foot stimulator 30, and a right foot stimulator 31. The tactile presentation system draws a three-dimensional image based on the position and orientation of the user object in the virtual space, as well as information of other objects, and displays the image on the display device 20. The left foot stimulator 30 and the right foot stimulator 31 also provide tactile feedback to the soles of the feet according to the state of the user object in the virtual space.

[0026] The control device 10 controls the display on the display device 20 and further controls the presentation of tactile sensations to the soles of the feet by the left foot stimulator 30 and the right foot stimulator 31. The control device 10 is a computer, such as a personal computer or a home game console. The control device 10 includes a processor 11, a storage 12, and a communication unit 13.

[0027] The processor 11 operates according to a program stored in the storage 12 and controls the communication unit 13 and the display device 20. The program may be provided by another computer via communication through the communication unit 13, or it may be provided to another computer by being stored in a computer-readable storage medium such as flash memory or an optical disc.

[0028] The storage 12 is composed of at least some of the following: RAM (e.g., DRAM), non-volatile memory, and a secondary storage device (e.g., a hard disk drive, a solid-state drive). The storage 12 stores the above-mentioned program. The storage 12 also stores information and calculation results input from the processor 11, the communication unit 13, etc.

[0029] The communication unit 13 consists of integrated circuits, terminals, antennas, etc., for communicating with other devices. The communication unit 13 is configured to communicate with other devices (e.g., computers) according to protocols such as Ethernet, Wi-Fi®, or Bluetooth®. The communication unit 13 may include, for example, a network interface card. Based on the control of the processor 11, the communication unit 13 inputs information received from other devices to the processor 11 and storage 12, and transmits the information to the other devices.

[0030] The display device 20 is a device that displays images to the user. The display device 20 may be a flat panel display, a head-mounted display, or it may be built into the control device 10.

[0031] The left foot stimulator 30 is a device that provides tactile sensations to the sole of the left foot, and the right foot stimulator 31 is a device that provides tactile sensations to the sole of the right foot. Each of the left foot stimulator 30 and the right foot stimulator 31 has a plurality of movable contact parts 32 and a plurality of actuators 33. The plurality of movable contact parts 32 are spaced apart from each other and each has a surface configured to contact a part of the sole of the foot. The plurality of movable contact parts 32 are movable in a direction along their surfaces. The plurality of actuators 33 move the plurality of movable contact parts 32 individually. The plurality of actuators 33 also move each of the plurality of movable contact parts 32 in two different directions along their surfaces.

[0032] Figure 2 is a schematic diagram showing an example of a right foot stimulator 31. The right foot stimulator 31 includes a base plate 36, actuators 33a to 33d, movable contact parts 32a to 32d, a foot fixing part 34, and a strap 35. The actuators 33a to 33d are arranged on the base plate 36, and the movable contact parts 32a to 32d are each arranged and driven above the actuators 33a to 33d. The upper surfaces of the multiple movable contact parts 32a to 32d are configured to contact the sole of the foot. The actuators 33a to 33d allow the movable contact parts 32a to 32d to move in a direction along the upper surfaces of the multiple movable contact parts 32a to 32d. In the example of Figure 2, the movable contact parts 32a to 32d are each provided corresponding to the actuators 33a to 33d and are operated by the corresponding actuators 33a to 33d. Here, one movable contact part 32a to 32d may correspond to multiple actuators 33.

[0033] The foot fixing part 34 is fixed to the base plate 36 via an intermediate member 37, and actuators 33a to 33d and movable contact parts 32a to 32d are arranged between the foot fixing part 34 and the base plate 36. In the example shown in Figure 3, two straps 35 are connected to the foot fixing part 34, and the two straps 35 are positioned to hold down the toes and ankles, respectively. The foot fixing part 34 and the straps 35 prevent the foot from moving forward, backward, left, or right. To accommodate differences in foot size, the foot fixing part 34 may include a mechanism to adjust the connection with the straps 35.

[0034] Figure 3 shows an example of the arrangement of multiple movable contact parts 32 in the right foot stimulator 31. Figure 3 shows the arrangement of the movable contact parts 32a to 32d, the foot fixing part 34, and the base plate 36 from a top view of the right foot stimulator 31. The foot fixing part 34 has a shape that corresponds to the shape of the sole of the foot in a plan view. On the other hand, since the multiple movable contact parts 32a to 32d are configured to contact the sole of the foot, the foot fixing part 34 does not need to have a surface for contacting the sole of the foot. If the foot fixing part 34 has a surface for contacting the sole of the foot, that surface is provided in a range that does not interfere with at least the multiple movable contact parts 32a to 32d.

[0035] The movable contact portion 32a is configured to contact at least a portion of the toes. The movable contact portion 32b is configured to contact the base of the big toe, known as the ball of the foot. The movable contact portion 32c is configured to contact the base of the toes other than the big toe (for example, the ring finger and little finger). The movable contact portion 32d is configured to contact the heel. The movable contact portions 32a to 32d are positioned to contact corresponding parts of the foot within a predetermined size range. The right foot stimulator 31 may also include a mechanism to adjust the positions of the movable contact portions 32a to 32d so that they contact the corresponding parts mentioned above.

[0036] In a plan view, the movable contact portion 32a overlaps the front of the foot fixing portion 34, the movable contact portion 32d overlaps the rear of the foot fixing portion 34, and the movable contact portions 32b and 32c are provided between the movable contact portions 32a and 32d when viewed in the front-rear direction, with the centers of the movable contact portions 32b and 32c located in front of the center of the foot fixing portion 34 in the front-rear direction. When viewed in the front-rear direction, the center of the movable contact portion 32c may be located behind the center of the movable contact portion 32b. In a plan view, the movable contact portion 32b overlaps the left (inner) end of the foot fixing portion 34, and the movable contact portion 32c is located to the right (outer) of the movable contact portion 32b when viewed in a plan view.

[0037] Here, the left foot stimulator 30 has a mirror-symmetric configuration with the right foot stimulator 31, except for the detailed components such as the inside of the actuator 33. Therefore, a detailed explanation of the left foot stimulator 30 will be omitted. In the left foot stimulator 30, the right side corresponds to the medial side, and the left side corresponds to the lateral side.

[0038] In the examples shown in Figures 2 and 3, the number of movable contact parts 32 provided on the left foot stimulator 30 and the right foot stimulator 31 is four, but only some of these may be present. For example, instead of movable contact parts 32b and 32c, one movable contact part 32 corresponding to both may be provided, or movable contact parts 32a or 32c may not be provided at all. The number of movable contact parts 32 may also be greater. For example, additional movable contact parts 32 that contact some of the toes (e.g., the ring finger and little finger) may be provided.

[0039] When a motion that moves the user in the virtual space is initiated, the control device 10 controls the movement of the multiple movable contact parts 32 based on control information stored in association with the type of motion. The movement of the multiple movable contact parts 32 presents tactile sensations to the soles of the feet.

[0040] Figure 4 is a flowchart showing an example of the processing of the tactile feedback system, and the processing shown in Figure 4 is mainly executed by the control device 10. More specifically, the processor 11 included in the control device 10 executes the instruction code of the control program stored in the storage 12, and controls the communication unit 13, the display device 20, the left foot stimulator 30, or the right foot stimulator 31.

[0041] First, the control device 10 determines the type of motion for the user object (S101). The types of motion may include, for example, starting to walk, starting to run, walking in the middle, running in the middle, stopping walking, stopping running, jumping forward, running around a curve, moving laterally, a tennis forehand stroke, walking on a rock, walking on mud, and tilting the body to the left or right.

[0042] The control device 10 may determine the type of motion based on instructions input from the user via an input device. The control device 10 may also determine the type of motion based on the relationship (e.g., contact / proximity) between the user's object in the virtual space and other objects in its surroundings, or it may determine a type associated with the type of motion that was just completed. An example of the latter is determining an intermediate walking motion when the type of motion of the just-finished motion was the start of walking. Here, the control device 10 may use the determined time as the start time.

[0043] Once the type of motion is determined, the control device 10 obtains position and orientation information of the user object from the motion data corresponding to the type, and further obtains control information of the movable contact part 32 corresponding to the type (S102). Here, motion data and control information exist for each type of motion, and these are stored in the storage 12 in association with the corresponding type of motion.

[0044] The position and orientation information of the user object is information indicating the position and orientation of the user object in the virtual space. The information indicating the orientation may be information indicating the relative positions and angles of elements constituting the user object, for example, a plurality of elements constituting the feet, hands, head, and torso respectively. The information indicating the position may be information indicating the moving speed of the representative position of the user object. The control information is information indicating the movement of each of the plurality of movable contact portions 32 according to the elapsed time from the start time. The control information may be information indicating the direction and magnitude of the shearing force applied by the movable contact portion 32 to the corresponding part of the sole of the foot. The control information may also be information indicating the position, moving speed, and direction of the movable contact portion 32. When the movable contact portion 32 moves up and down, the control information may further include the vertical position of the movable contact portion 32.

[0045] FIG. 5 is a diagram showing an example of control information. FIG. 5 shows the control information stored in the storage 12 in association with a certain type of motion. In this control information, a two-dimensional shearing force output by the movable contact portion 32 is stored in association with time. In the example of FIG. 5, the values of the shearing force are set for each of the left-right direction and the front-back direction. Also, the shearing force in the control information is set for each of the movable contact portions 32 corresponding to the toe, ball of the thumb, ball of the little finger, and heel in the left foot stimulation device 30 and each of the movable contact portions 32 corresponding to the toe, ball of the thumb, ball of the little finger, and heel in the right foot stimulation device 31. In the example of FIG. 5, the control information is set such that the time is the start time and the output is maintained until the next time.

[0046] When the position and orientation information is acquired, the control device 10 determines the position and orientation of the user object in the virtual space (S 103). In S 103, the control device 10 may execute a process of correcting the position information in the local coordinate system indicated by the position and orientation information to the coordinate system of the virtual space, and a process of correcting the positions and angles of the elements constituting the user object, for example, by IK (Inverse Kinematics) according to the state of the ground. In S 103, the control device 10 executes a process of detecting the contact between the sole of the user object's foot and an object other than the ground.

[0047] And when the control device 10 detects contact between the sole of the user object and the object, it modifies the control information of the movable contact part 32 (S104). Here, for example, when contact is detected for an object protruding from the ground such as a rock, the control device 10 may modify the control information. Details of the process will be described later. Note that the process of S104 may not be performed.

[0048] Next, the control device 10 controls the actuator 33 based on the control information to control the movement of the movable contact part 32 (S105). The control device 10 may control the actuator 33 to generate a shearing force in the magnitude and direction indicated by the control information. For example, a correspondence table between the shearing force and the displacement from the reference position is stored in the storage 12 in advance, and the control device 10 controls the actuator 33 corresponding to the movable contact part 32 so that the displacement corresponding to the shearing force occurs in the movable contact part 32 in the indicated direction. The direction of the shearing force and the direction from the reference position of the movable contact part 32 are the same. When the movable contact part 32 outputs a shearing force in a certain direction, the control device 10 controls the actuator 33 so that the movable contact part 32 moves to the position where the displacement in that direction occurs. The relationship between the motion and the control of the movement of the movable contact part 32 will be described later.

[0049] The control device 10 draws a display image based on the position and orientation of the user object in the virtual space in parallel with the control of the actuator 33 (S106). The control device 10 sends an image signal indicating the image generated by the drawing to the display device 20, causing the display device 20 to output the display image. Note that this drawing may be performed using a general three-dimensional image drawing method. A detailed description of the drawing method will be omitted.

[0050] As the control of the movable contact part 32 and the rendering of the display image progress, the control device 10 determines whether to terminate the determined motion (S107). Here, the control device 10 may determine termination when an instruction is input from the user via an input device, or when a collision with a specific type of object in the surroundings is detected. If termination is determined (Y in S107), the process is repeated from S101. On the other hand, if it is determined that termination is not possible (N in S107), the process from S102 onwards is repeated. The time since disclosure changes with each repetition, and the operation of the movable contact part 32 is controlled by control information corresponding to that time.

[0051] The following section will explain how to control the movement of the movable contact part 32 for each type of motion.

[0052] Figure 6 illustrates the movement of the movable contact portion 32 during walking or running. The waveform diagrams shown below indicate the magnitude and duration of the shear force output by the movable contact portion 32 corresponding to the heel, little toe ball, big toe ball, and toe, respectively. The numbered rectangular regions 62 at the top of Figure 6 are connected to the waveform diagrams by dashed lines. These regions 62 also include a left region 60 and a right region 61. The circles in the left region 60 and the right region 61 correspond to the movable contact portion 32 in the left foot stimulator 30 and the right foot stimulator 31, respectively. The circles at the top, left, right, and bottom in the right region 61 correspond to the movable contact portions 32a, 32b, 32c, and 32d, respectively, and the circles at the top, right, left, and bottom in the left region 60 correspond to the movable contact portions 32a, 32b, 32c, and 32d, respectively.

[0053] The arrows extending from the circle indicate the direction and magnitude of the force acting on the corresponding movable contact part 32 from the sole of the foot at the time indicated by the dashed line connected to the region 62 containing the circle. This force is the reaction force to the shear force that the movable contact part 32 outputs to the sole of the foot. The direction of the shear force is opposite to that of the arrow, and the magnitude of the shear force is the same as that of the force described above. Unless otherwise specified, the direction of the shear force remains the same in the waveform diagram from when the waveform value rises until when it falls. The direction of movement of the movable contact part 32 is the same as the direction of the shear force, and the amount of movement from the reference position increases as the shear force increases.

[0054] Here, the storage 12 stores control information (see Figure 5) indicating the movement shown in Figure 6, associated with the type of motion (walking or starting to run). When the motion to start walking or running is determined in S101, the control device 10 controls the multiple movable contact parts 32 to output a shear force corresponding to the time, based on the control information and the current time.

[0055] When walking or running motion begins, the control device 10 controls the actuator 33 that drives the movable contact parts 32 in the following order. The item numbers correspond to the numbers in region 62 in Figure 6. (1) The actuator 33 is controlled so that the movable contact part 32d corresponding to the heel of the right foot outputs a forward shear force. (2) Subsequently, when the shear force of the movable contact part 32d corresponding to the heel becomes sufficiently small, the actuator 33 is controlled so that the movable contact parts 32c and 32b corresponding to the little toe ball and the big toe ball of the right foot output a stronger forward shear force than the previous movable contact part 32d. (3) Subsequently, when the shear force output by the movable contact parts 32c and 32b corresponding to the little toe ball and the big toe ball becomes sufficiently small, the actuator 33 is controlled so that the movable contact part 32a corresponding to the toes of the right foot outputs a stronger forward shear force than the previous movable contact parts 32b and 32c.

[0056] When this motion ends, the control device 10 initiates a motion of the type that is either an intermediate walking motion or an intermediate running motion. As described here, the tactile presentation system according to this embodiment can apply shear force to each part of the sole of the foot at different timings, allowing the user to feel a more realistic tactile sensation.

[0057] Figure 7 illustrates the movement of the movable contact part 32 during walking. Similar to Figure 6, the waveform diagram shows the magnitude and duration of the shear force output by the movable contact part 32, and the circles and arrows within the numbered rectangular areas 62 at the top indicate the direction and magnitude of the movable contact part 32 and the reaction force to the shear force exerted by the movable contact part 32. The storage 12 stores control information indicating the movement shown in Figure 7, associated with the type of motion. Control information indicating the movement shown in the figures is also stored in association with the type of motion for other types of motions described in subsequent figures.

[0058] When the motion in the middle of walking begins, the control device 10 controls the movable contact parts 32 in the following order: (1) Control the actuator 33 so that the movable contact part 32d corresponding to the heel of the right foot outputs a forward shear force. (2) Next, while weakening the forward shear force of the movable contact part 32d, control the actuator 33 so that the movable contact part 32c corresponding to the little toe ball outputs a forward shear force. (3) Next, when the shear force of the movable contact part 32d becomes sufficiently small, control the actuator 33 so that the movable contact part 32b corresponding to the big toe ball outputs a forward shear force. (4) Control the actuator 33 so that the direction in which the movable contact part 32c outputs shear force is inward and diagonally forward. (5) When the shear force of the movable contact part 32c becomes sufficiently small, control the actuator 33 so that the movable contact part 32b corresponding to the big toe ball outputs a stronger forward shear force. (6) The actuator 33 is controlled so that the movable contact portion 32a corresponding to the toes outputs a forward shear force, and the forward shear force of the movable contact portion 32b is weakened. (7) The actuator 33 is controlled so that the shear force of the movable contact portion 32b is sufficiently reduced, and the movable contact portion 32a corresponding to the toes outputs a stronger forward shear force. (8) The actuator 33 is controlled so that the shear force of the movable contact portion 32a is sufficiently weakened.

[0059] Control (1) is performed in response to the heel of the user object's right foot making contact with the ground in the virtual space, and control (8) is performed in response to the right foot leaving the ground. During the motion in the middle of walking, the direction in which the movable contact part 32c corresponding to the ball of the little toe outputs shear force changes from forward to inward and diagonally forward. In the haptic presentation system according to this embodiment, shear force can be applied at different timings depending on the part of the sole of the foot, and shear force can be applied not only in the front-to-back direction but also in the left-to-right direction. As a result, the user can feel a tactile sensation that is closer to reality.

[0060] Although Figure 7 only shows the control of the movable contact part 32 for the right foot, the same control as in Figure 7 is executed after the waveform diagram in Figure 7, in accordance with the ground contact of the left foot in the virtual space. Since the left foot is a mirror image symmetrical to the right foot, it is the same as in Figure 7 except that the left and right positions and movements are different. Therefore, a detailed explanation is omitted.

[0061] Figure 8 illustrates the movement of the movable contact part 32 during travel. The meaning of the waveform diagram and the arrows within region 62 is the same as in Figure 7.

[0062] When motion during travel begins, the control device 10 controls the movable contact parts 32 in the following order: (1) The actuator 33 is controlled so that the movable contact part 32d corresponding to the heel of the right foot outputs a forward shear force. (2) Next, the actuator 33 is controlled so that the movable contact part 32c corresponding to the ball of the little toe outputs a forward shear force while the forward shear force of the movable contact part 32d is weakened. (3) Next, when the shear force of the movable contact part 32d becomes sufficiently small, the actuator 33 is controlled so that the movable contact part 32c and the movable contact part 32b corresponding to the ball of the big toe output a rearward shear force. Here, more precisely, the movable contact part 32b is controlled to output a shear force slightly diagonally inward from the rear. (4) The actuator 33 is controlled so that the direction in which the movable contact part 32b outputs a shear force is forward, and the direction in which the movable contact part 32c outputs a shear force is inward and diagonally forward. (5) The actuator 33 is controlled so that the shear force of the movable contact portion 32c becomes sufficiently small and the movable contact portion 32b corresponding to the ball of the foot outputs a stronger forward shear force. (6) The actuator 33 is controlled so that the movable contact portion 32a corresponding to the toes newly outputs a forward shear force and the forward shear force of the movable contact portion 32b weakens. (7) The actuator 33 is controlled so that the shear force of the movable contact portion 32b becomes sufficiently small and the movable contact portion 32a corresponding to the toes outputs a stronger forward shear force. (8) The actuator 33 is controlled so that the shear force of the movable contact portion 32a weakens sufficiently.

[0063] Here, the control device 10 may make the magnitude of the shear force proportional to the user's movement speed in the virtual space, and the period for presenting the shear force may be a linear function of the movement speed, that is, the sum of the value obtained by multiplying the movement speed by a constant and the minimum operating time t.

[0064] The control device 10 controls the movable contact part 32d to move forward in response to the heel of the user object's right foot touching the ground in (1), then moves the movable contact parts 32b and 32c backward in (3), and further controls the movable contact part 32b to move forward and the movable contact part 32c to move inward and diagonally forward in (4). The control in (8) is performed in response to the right foot leaving the ground. These controls allow the user to feel a tactile sensation closer to the sensation of running.

[0065] Although Figure 8 only shows the control of the movable contact part 32 for the right foot, similar control is performed after the waveform diagram in Figure 8, in accordance with the ground contact of the left foot in the virtual space. Since the left foot is a mirror image symmetrical to the right foot, a detailed explanation is omitted.

[0066] Figure 9 illustrates the movement of the movable contact part 32 when walking stops. The meaning of the waveform diagram and the arrows within region 62 is the same as in Figure 7.

[0067] When the motion to stop walking begins, the control device 10 controls the movable contact parts 32 in the following order: (1) The actuator 33 is controlled so that the movable contact part 32d corresponding to the heel of the right foot outputs a rearward shear force. (2) Next, the actuator 33 is controlled so that the rearward shear force of the movable contact part 32d is sufficiently reduced, and the movable contact parts 32a, 32b, and 32c corresponding to the toes, ball of the foot, and little toe ball output a rearward shear force. Here, the direction of the shear force of the movable contact part 32a may be rearward from the inside, and the direction of the shear force of the movable contact part 32c may be rearward from the outside. (3) Next, the control device 10 controls the movable contact parts 32a, 32b, and 32c so that the shear forces are sufficiently reduced, and the movable contact part 32d outputs a forward shear force. (4) The actuator 33 is controlled so that the movable contact parts 32a, 32b, and 32c output a rearward shear force, and then the actuator 33 is controlled so that the forward shear force of the movable contact part 32d is sufficiently reduced. The direction of the shear force on the movable contact parts 32a, 32b, and 32c may be the same as in (2). (5) Control the actuator 33 so that the shear force on the movable contact parts 32a to 32d is sufficiently small.

[0068] The control in (1) is performed in accordance with the contact of the user object's right heel with the ground in the virtual space. In the motion of stopping walking, a rearward shear force is output from the movable contact part 32d etc. in (1) and (2), but in (3), the movable contact part 32d outputs a forward shear force. This allows the user to feel a tactile sensation that is closer to reality. Similar control may be performed for the left foot, and similar control may be performed when stopping running.

[0069] Figure 10 illustrates the movement of the movable contact portion 32 during right-hand curve travel. During right-hand curve travel, the timing of movement of the movable contact portion 32 is the same as the motion during travel as described in Figure 8, but the amount of shear force output by the left foot stimulator 30 and the right foot stimulator 31 is different. In addition, in order to counteract centrifugal force, the shear force includes a component in the direction corresponding to the direction of the curve (inward direction of the curve). Therefore, the control device 10 controls the actuator 33 so that at least a part of the movable contact portion 32 also moves in the direction corresponding to the direction of the curve.

[0070] Furthermore, the control device 10 controls the actuator 33 so that the shear force output by the movable contact portion 32 of the right foot stimulator 31, which corresponds to the inside of the curve, is greater than the shear force output by the left foot stimulator 30, which corresponds to the outside of the curve. Here, not only the magnitude of the shear force but also the duration for which the shear force is applied may be increased. This allows the user to feel the sensation of driving around a curve through touch. In the case of a left curve, the direction of the shear force is reversed left to right, with the left foot stimulator 30 corresponding to the inside of the curve, and the directions corresponding to the inside and outside of the curve being reversed left to right.

[0071] Note that the control of the movable contact parts 32 in Figures 6 to 10 is merely an example, and the time and direction of movement of each movable contact part 32 may differ, or the direction and time of movement of each movable contact part 32 may be customized according to the user's habits.

[0072] Figure 11 illustrates the movement of the movable contact part 32 when the foot applies a strong lateral force to the ground. The example in Figure 11 corresponds, for example, to cases where the type of motion being reproduced involves the body moving laterally and then reversing its direction of movement in ball games such as tennis or basketball, or simply involves a motion of tilting the body to the side. Region 62 is similar to that shown in Figure 6, etc., and the arrows extending from the circle indicate the direction and magnitude of the reaction force of the shear force output by the movable contact part 32 corresponding to that circle.

[0073] When the direction of body movement reverses, the control device 10 controls the actuator 33 so that at least some of the multiple movable contact parts 32 included in the left foot stimulator 30 and the right foot stimulator 31 corresponding to the foot making contact with the ground, output a shear force (inward-facing shear force) along the direction of body movement after the reversal, according to the ground contact of the foot on the side of that direction of movement in the virtual space at the timing immediately before the reversal. As shown in Figure 11, the direction of the shear force output by the multiple movable contact parts 32 does not need to perfectly coincide with the direction of movement, and may be shifted somewhat, for example, in the front-back direction.

[0074] Figure 12 illustrates the movement of the movable contact part 32 in a tennis forehand stroke. In the example in Figure 12, in the virtual space, the user's right hand object moves the racket to the right side of the body, then moves the racket towards the left side of the body while hitting the ball, and then the upper body object twists to the left. In the example in Figure 12, multiple regions 62 are arranged vertically, and time progresses from top to bottom. The ball is hit back at timings (2) to (3), and at timing (4), the right hand has moved to the left side relative to the upper body, and the right foot is on tiptoe.

[0075] When the forehand stroke motion begins, the control device 10 controls the movable contact parts 32 in the following order: (1) The actuator 33 is controlled so that the movable contact parts 32a to 32d included in the right foot stimulator 31 output a shear force to the left, and the movable contact part 32a included in the left foot stimulator 30, which corresponds to the toes, outputs a weaker shear force to the left. (2) The shear force output by the movable contact parts 32a to 32d included in the right foot stimulator 31 is reduced. (3) The actuator 33 is controlled so that the movable contact parts 32a to 32d included in the left foot stimulator 30 output a shear force to the right. The actuator 33 is also controlled so that the shear force of the movable contact parts 32c and 32d included in the right foot stimulator 31 is sufficiently reduced, and the movable contact parts 32a and 32b included in the right foot stimulator 31 continue to output a shear force to the left. (4) The shear force of the movable contact parts 32a to 32d included in the left foot stimulator 30 is not changed, the shear force of the movable contact part 32b included in the right foot stimulator 31 is sufficiently reduced, and the actuator 33 is controlled so that the movable contact part 32a included in the right foot stimulator 31 continuously outputs a shear force in the left direction.

[0076] This is the case when the racket is held with the right hand. In this case, the control device 10 moves at least a portion of the movable contact portion 32 of the right foot stimulator 31 corresponding to the right foot in the first direction in (1), and further moves the movable contact portion 32a corresponding to the toes in the first direction. Then, in (3), while moving at least a portion of the movable contact portion 32 of the left foot stimulator 30 corresponding to the left foot in the second direction, the movable contact portion 32a of the right foot stimulator 31 corresponding to the toes is moved in the first direction. Here, the first direction is the direction of the left foot stimulator 30 (left) as seen from the right foot stimulator 31, and the second direction is the direction of the right foot stimulator 31 (right) as seen from the left foot stimulator 30. When the racket is held with the left hand, the left and right are reversed.

[0077] In this example, by applying shear force in the lateral direction according to the timing of the foot's weight shift, the user can experience a more realistic tactile sensation.

[0078] Figure 13 illustrates the movement of the movable contact part 32 when jumping forward. The example in Figure 13 describes a type of motion in which both feet jump forward. In the example in Figure 13, multiple regions 62 are arranged vertically, and time progresses from top to bottom. (1) corresponds to the state when the knee begins to bend, (2) corresponds to the state when the knee is most bent, (3) corresponds to the state when the knee is extended and pushing off the ground, and (4) corresponds to the state when landing after the jump.

[0079] When the motion of jumping forward with both feet begins, the control device 10 controls the movable contact parts 32 in the following order: (1) Control the actuator 33 so that the movable contact parts 32a to 32d included in both the left foot stimulator 30 and the right foot stimulator 31 output a forward shear force. (2) Control the actuator 33 so that the shear force output by the movable contact parts 32a to 32d included in both the left foot stimulator 30 and the right foot stimulator 31 increases. (3) Sufficiently weaken the shear force of the movable contact part 32d included in both the left foot stimulator 30 and the right foot stimulator 31 that corresponds to the heel. (4) Control the actuator 33 so that the movable contact parts 32a to 32d included in both the left foot stimulator 30 and the right foot stimulator 31 output a backward shear force.

[0080] In this motion, the control device 10 controls at least some of the multiple movable contact parts 32 to output a forward shear force before the foot leaves the virtual space, and at least some of the multiple movable contact parts 32 to output a backward shear force in response to the foot landing in the virtual space. Furthermore, before the foot leaves the virtual space, the control device 10 sequentially controls the movable contact parts 32a to 32d corresponding to the heel, toes, ball of the foot, and little toe to output shear forces, and the movable contact part 32 corresponding to the heel to not output shear forces. This type of control allows the user to experience a tactile sensation that is closer to reality.

[0081] Here, the left foot stimulator 30 and the right foot stimulator 31 may include a mechanism to individually move up and down a plurality of movable contact parts 32. More specifically, a linear actuator may be provided below the actuators 33a to 33d to move up and down together with the actuators 33 that move forward, backward, left, and right, or a linear actuator may be provided between the actuators 33 that move forward, backward, left, and right and the movable contact part 32 to move up and down the movable contact part 32. Hereafter, when simply referred to as actuator 33, it will also include the linear actuator for moving up and down.

[0082] The movable contact portion 32 moves up and down, making it possible to provide a foot sensation that corresponds to the type of ground. Examples of this will be explained below, such as walking on mud, walking on rocks, and walking on gravel.

[0083] Figure 14 illustrates the movement of the movable contact portion 32 when walking on mud. Figure 14 shows the movement of the movable contact portion 32 of the left foot stimulator 30 in a motion of walking on mud. Multiple sets of movable contact portions 32 shown in Figure 14 are numbered from 1 to 9, and the movement of the movable contact portion 32 changes in the order of these numbers. In Figure 14, the dashed arrows indicate the vertical movement of the movable contact portion 32, and the solid arrows indicate the direction of the reaction force of the shear force output by the movable contact portion 32.

[0084] In the example in Figure 14, when walking on mud, the part of the foot that touches the ground sinks in. In the example in Figure 14, the toes are in contact at (1), the ball of the big toe at (3), the ball of the little toe at (4), and the heel at (5). The control device 10 outputs a shear force by causing the movable contact part 32 corresponding to the part of the foot that touches the ground to move along the surface of the movable contact part 32, and also controls the movable contact part 32 to descend.

[0085] More specifically, when the motion of walking on mud begins, the control device 10 controls the movable contact parts 32 in the following order: (1) Control the actuator 33 so that the movable contact part 32a corresponding to the toes outputs a rearward shear force and moves downward. (2) Continue the control in (1). (3) While maintaining the shear force and downward movement of the movable contact part 32a, control the actuator 33 so that the movable contact part 32b corresponding to the ball of the big toe outputs a rearward shear force and moves downward. (4) End the output of shear force and downward movement of the movable contact part 32a, and while maintaining the shear force and downward movement of the movable contact part 32b, control the actuator 33 so that the movable contact part 32c corresponding to the ball of the little toe outputs a rearward shear force and moves downward. (5) End the output of shear force and downward movement of the movable contact part 32b, and while maintaining the shear force and downward movement of the movable contact part 32c, control the actuator 33 so that the movable contact part 32d corresponding to the heel moves downward. (6) The actuator 33 is controlled so that the output of shear force and downward movement of the movable contact part 32c ends, and the movable contact part 32d outputs a rearward shear force and moves downward. (7) The actuator 33 is controlled so that the output of shear force and downward movement of the movable contact part 32d ends. (8) The movable contact parts 32a to 32d are raised. This corresponds to the timing when the feet leave the mud in the virtual space. (9) The movable contact parts 32a to 32d return to the same height as in (1).

[0086] By lowering the movable contact part 32 in conjunction with demonstrating the shear force in this way, it is possible to express the feeling of your foot sinking into the mud when stepping on it.

[0087] Figure 15 illustrates the movement of the movable contact part 32 when walking on a rock. Figure 15 shows the movement of the movable contact part 32 of the left foot stimulator 30 in a motion of walking on a rock. Multiple sets of the movable contact part 32 shown in Figure 15 are numbered from 1 to 7, and the movement of the movable contact part 32 changes in the order of these numbers. In Figure 15, the dashed arrows and solid arrows have the same meaning.

[0088] When walking on rocks, the control device 10 controls a portion of the movable contact area 32 that corresponds to the part of the foot that is in contact with the protrusions of the rock in the virtual space to rise. In the example in Figure 15, the ball of the foot is in contact with the protrusions of the rock in (2) and (3), so this part is raised.

[0089] More specifically, in the process of S104 in Figure 4, when contact between the sole of the foot in the virtual space and the object of the rock's protrusion is detected, the control device 10 identifies the movable contact part 32 corresponding to the sole of the foot, modifies the control information so that the identified movable contact part 32 rises, and after a certain period of time, the movable contact part 32 lowers to its original position. Alternatively, when contact between the sole of the foot in the virtual space and the object of the rock's protrusion is detected, the control information may be modified so that a predetermined movable contact part 32 (for example, movable contact part 32b) rises and then lowers to its original position without identifying the movable contact part 32.

[0090] In the example shown in Figure 15, the control device 10 controls the movable contact portion 32 in the following order: (1) Control the actuator 33 so that the movable contact portion 32a outputs a rearward shear force. (2) Control the actuator 33 so that in addition to the movable contact portion 32a, the movable contact portion 32b outputs a rearward shear force, and the movable contact portion 32b rises. (3) Control the actuator 33 to terminate the output of the shear force from the movable contact portion 32a, and instead cause the movable contact portion 32c to output a rearward shear force, while maintaining the shear force and upward movement of the movable contact portion 32b. (4) Control the actuator 33 to terminate the output of the shear force from the movable contact portion 32b while maintaining the rearward shear force from the movable contact portion 32c, and cause the movable contact portion 32b to descend. (5) Control the actuator 33 to terminate the output of the shear force from the movable contact portion 32c and cause the movable contact portion 32d to output a rearward shear force. (6) Terminate the output of the shear force from the movable contact portion 32d.

[0091] This type of control allows for a more precise representation of the sensations felt on the soles of the feet when walking on rocks.

[0092] Walking on gravel may also be represented by vertical movement. In this case, the control device 10 may randomly move the movable contact part 32 that outputs shear force up and down, and control the actuator 33 to return it to its original position when the output of shear force ends.

[0093] Although various embodiments of the invention have been described so far, the technical scope of the invention is not limited to these and can be modified in various ways in accordance with its gist.

[0094] A general xy stage operated by screw drive may be used as the actuator 33 in the left foot stimulator 30 and the right foot stimulator 31. Figure 16 shows an example of the arrangement of the actuator 33 operated by screw drive. Figure 16 shows the arrangement of the actuator 33 (in this case, the xy stage) when viewed from above in the left foot stimulator 30. The area 70 enclosed by the solid line shows the shape of the left foot.

[0095] The xy stage includes a first mechanism 71, shown by a dashed line, and a second mechanism 72, shown by a solid line. A movable contact portion 32 is positioned on top of the second mechanism. The first mechanism 71 moves the second mechanism 72, which includes the movable contact portion 32, in a first direction by turning a screw with a motor 73. The second mechanism 72 is positioned on top of the first mechanism 71 and moves the movable contact portion 32 in a second direction by turning a screw with a motor 74. The second direction is perpendicular to the first direction, and the first direction is either forward / backward or left / right. This makes it possible to move the movable contact portion 32 forward / backward and left / right. In plan view, much of the xy stage extends beyond the movable contact portion 32, but as shown in Figure 16, it is possible to arrange them without interference through ingenuity.

Claims

1. A tactile presentation system comprising: a left stimulator and a right stimulator, each including a plurality of movable contact parts spaced apart from each other and having surfaces configured to contact any part of the sole of the foot, and moving in a direction along said surfaces, and actuators for individually moving the plurality of movable contact parts; and a control unit that controls the movement of the plurality of movable contact parts based on control information stored in association with the type of motion when a motion that moves a user in a virtual space is initiated.

2. A tactile presentation system according to claim 1, wherein the actuator moves each of the plurality of movable contact parts in two mutually different directions along the surface.

3. A tactile presentation system according to claim 1, wherein the plurality of movable contact parts include at least a portion of a first contact part configured to contact the toes, a second contact part configured to contact the base of the big toe, a third contact part configured to contact the base of a toe other than the big toe, and a fourth contact part configured to contact the heel.

4. A tactile presentation system according to claim 3, wherein the control unit, when the type of motion to be initiated is running, moves the fourth contact portion forward in response to the heel making contact with the ground in the virtual space, then moves the second contact portion backward and then moves the second contact portion forward.

5. A tactile presentation system according to claim 4, wherein, when the type of motion to be initiated is running, the control unit moves the fourth contact portion forward in response to the heel making contact with the ground in the virtual space, then moves the second contact portion backward, moves the second contact portion forward, and moves the third contact portion inward and diagonally forward.

6. A tactile presentation system according to claim 1, wherein the control unit moves at least a portion of the plurality of movable contact parts forward before the foot leaves the virtual space when the type of motion to be initiated is a forward jump, and moves at least a portion of the plurality of movable contact parts backward in response to the foot landing in the virtual space.

7. A tactile presentation system according to claim 1, wherein, when the type of motion to be initiated is travel along a curve, the control unit moves at least a portion of the plurality of movable contacts included in the left stimulator and the right stimulator in a direction corresponding to the direction of the curve, and the movement of the plurality of movable contacts in the right stimulator and the left stimulator corresponding to the inside of the curve is greater than the movement in the part corresponding to the outside of the curve.

8. A tactile presentation system according to claim 1, wherein the control unit moves at least a portion of the plurality of movable contact portions included in the left stimulation portion and the right stimulation portion corresponding to the foot that is in contact with the ground, in a direction along the direction of movement after reversal, in response to the foot on the side of the direction of movement in the virtual space when the motion to be started includes a reversal of the direction of movement in the lateral direction.

9. A tactile presentation system according to claim 1, wherein, when the type of motion to be initiated is a tennis forehand stroke, the control unit moves at least a portion of the plurality of movable contacts included in one of the left stimulator and the right stimulator toward the other of the left stimulator and the right stimulator, and then moves at least a portion of the plurality of movable contacts included in the other of the left stimulator and the right stimulator toward the one of the left stimulator and the right stimulator.

10. A tactile presentation system according to claim 9, wherein, when the type of motion to be started is a tennis forehand stroke, the control unit moves at least a portion of the plurality of movable contacts included in one of the left stimulator and the right stimulator in a first direction toward the other of the left stimulator and the right stimulator, and after moving the plurality of movable contacts included in the other of the left stimulator and the right stimulator that correspond to the toes in the first direction, moves at least a portion of the plurality of movable contacts included in the other of the left stimulator and the right stimulator in a second direction toward the one of the left stimulator and the right stimulator, and moves the plurality of movable contacts included in one of the left stimulator and the right stimulator that correspond to the thumb in the first direction.

11. A tactile presentation system according to claim 1, wherein each of the left stimulator and the right stimulator further includes a mechanism that individually moves the plurality of movable contact parts up and down based on the control of the control unit.

12. A tactile presentation system according to claim 11, wherein the control unit moves the plurality of movable contact parts downward along the surface in response to the contact of the foot in the virtual space when the type of motion to be started is walking on mud.

13. A tactile presentation system according to claim 11, wherein the control unit raises a portion of the plurality of movable contact portions in response to the contact of a portion of the foot in the virtual space when the type of motion to be started is walking on a rock.

14. A tactile presentation system according to claim 13, wherein the control unit raises, among the plurality of movable contact parts, a movable contact part corresponding to the part of the foot that contacts a protrusion in the virtual space when the type of motion to be started is walking on a rock.

15. A tactile presentation method comprising a left stimulator and a right stimulator, each including a plurality of movable contact parts spaced apart from each other and having surfaces configured to contact any part of the sole of the foot, and moving in a direction along said surfaces, and actuators for individually moving the plurality of movable contact parts, the method comprising: acquiring control information stored in association with the type of motion when a motion that moves a user in a virtual space is initiated; and controlling the movement of the plurality of movable contact parts based on the acquired control information.

16. A program that causes a computer to execute a process in which, when one of several motions that move a user in a virtual space is initiated, it acquires control information stored in association with the type of motion, and based on the acquired control information, controls the movement of a plurality of movable contact parts, each having a surface that is spaced apart from each other and configured to contact any part of the sole of the foot, and moving in a direction along that surface, by actuators that move the plurality of movable contact parts included in the left stimulation part and the right stimulation part, respectively, individually.

Images