Vibration perception position control device, vibration perception position control method, and vibration perception position control program

Abstract

This vibration sensing position control apparatus comprises: a sound image localization setting unit; a vibration signal generation unit; a synchronization control unit; a guide control unit; a sound control unit; and a vibration control unit. The sound image localization setting unit executes setting for localizing a sound image in a driving signal of an acoustic device on the basis of information indicating the direction of a sound source of each sound included in sound data. The vibration signal generation unit generates a vibration signal on the basis of information indicating vibration of a vibration device which applies vibration to the palm surface of a user. The synchronization control unit executes control for synchronizing the timings of generating sound and vibration with each other. The guide control unit controls presentation of guide information for guiding the direction and the angle of the palm surface. The sound control unit controls the acoustic device in response to the driving signal. The vibration control unit controls the vibration of the vibration device in response to the vibration signal.

Description

Technical Field

[0001] One aspect of the present invention relates to a vibration perception position control device, a vibration perception position control method, and a vibration perception position control program.

Background Art

[0002] Existing devices such as smartphones and game controllers are equipped with only about one or two vibrators, and are limited to applying vibration stimulation to the entire surface of the skin that comes into contact with the device. In order to change the perceived position of vibration on the contact surface, as many vibrators as the number of positions, or a special device, are required.

[0003] For example, Non-Patent Document 1 realizes the presentation of vibration for each of the five fingers by attaching a vibrator to each fingertip using a glove-type device.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the conventional technology, there is a problem that only one vibration source can be felt for one vibrator. If only one vibration source can be felt for one vibrator, for example, in order to transmit vibrations to a plurality of positions, regions, and / or directions on the surface of the palm, the same number of vibrators are required.

[0006] The present invention has been made in view of the above circumstances, and an object thereof is to provide a vibration perception position control device, a vibration perception position control method, and a vibration perception position control program capable of making a user feel a plurality of vibration sources on the palm even with a single vibrator.

Means for Solving the Problems

[0007] In order to solve the above problems, a vibration perception position control device according to an aspect of the present invention includes an audio-visual localization setting unit, a vibration signal generation unit, a synchronization control unit, a guidance control unit, a sound control unit, and a vibration control unit. The audio-visual localization setting unit performs settings for localizing audio-visuals in the drive signal of the sound device based on information indicating the direction of the sound source of each sound included in the sound data. The vibration signal generation unit generates a vibration signal based on information indicating the vibration of a vibration device that gives vibration to the palm surface of the user. The synchronization control unit performs control to synchronize the timing of generating sound and vibration. The guidance control unit controls the presentation of guidance information for guiding the orientation and angle of the palm surface. The sound control unit controls the sound device according to the drive signal. The vibration control unit controls the vibration of the vibration device according to the vibration signal.

Effects of the Invention

[0008] According to an aspect of the present invention, it is possible to provide a vibration perception position control device, a vibration perception position control method, and a vibration perception position control program capable of making a user feel a plurality of vibration sources on the palm even with a single vibrator by presenting vibration in synchronization with sound on the palm and controlling the localization of sound for auditory presentation.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 10A

Figure 10B

Figure 11

Figure 12

Figure 13

Figure 14

MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. [First Embodiment] FIG. 1 is a diagram showing an example of the appearance of a vibration perception position control device 10 according to the first embodiment of the present invention. The vibration perception position control device 10 is a device for making a user perceive vibration when the user holds it with the palm. An xyz coordinate system for explaining the processing described later is defined as shown in FIG. 1.

[0011] FIG. 2 is a diagram showing an example of a gripping method of the vibration perception position control device 10. In the example of FIG. 2, the user places the vibration perception position control device 10 on the palm of the left hand and holds it. The vibration perception position control device 10 uses the illusion of a person by emitting sound and vibration by the processing described later, and makes the user perceive that an arbitrary position on the palm surface is vibrating. For example, the vibration perception position control device 10 can also make the user perceive that the base of the ring finger on the palm is vibrating, or that the area extending from the base of the thumb to the base of the index finger is vibrating. Note that the above-described gripping method is an example, and the vibration perception position control device 10 may be in contact with the user's palm via a medium.

[0012] As shown in FIGS. 1 and 2, the vibration perception position control device 10 has a spherical shape. In this embodiment, the vibration perception position control device 10 is a perfect sphere, but it may be an ellipsoid. A direction mark M indicating the direction when the user places it on the palm and holds it is marked on the vibration perception position control device 10. In this embodiment, the direction mark M is an arrow, and the user is defined to hold the vibration perception position control device 10 in a state where this direction mark M is on the upper surface and the arrow points to the fingertip.

[0013] FIG. 3 is a diagram showing an example of the hardware configuration of the vibration perception position control device 10. The vibration perception position control device 10 includes a CPU 101, a memory 102, a communication device 103, an acoustic device 104, and a vibration device 105. These CPU 101, memory 102, communication device 103, acoustic device 104, and vibration device 105 are interconnected by a bus 106.

[0014] The memory 102 is a storage that uses a combination of a non-volatile memory such as a ROM and a volatile memory such as a RAM as a storage medium. The memory 102 stores programs necessary for the CPU 101 to execute various processes. The programs include the vibration perception position control program according to the first embodiment. The memory 102 further stores data acquired and created in the process of the CPU 101 performing various processes.

[0015] The CPU 101 may be a multi-core multi-thread type and can execute a plurality of processes in parallel.

[0016] The communication device 103 is a device for transmitting and receiving signals to and from other devices. The communication may be performed either by wire or wirelessly. As the wireless method, for example, a mobile phone communication system such as 4G, 5G, etc., a wireless LAN, a small power wireless data communication standard such as Bluetooth (registered trademark), etc. can be used.

[0017] The acoustic device 104 is, for example, a speaker capable of stereo reproduction, and is a device that receives a drive signal from the CPU 101 and generates sound. Note that the vibration perception position control device 10 does not necessarily have to include the acoustic device 104. In that case, the vibration perception position control device 10 transmits a drive signal to an external acoustic device such as headphones, earphones, a speaker, etc. through the communication device 103.

[0018] The vibration device 105 is a device that receives a vibration signal from the CPU 101 and generates vibration. The vibration device 105 may be composed of one vibrator or may be composed of a plurality of vibrators. That is, the vibration perception position control device 10 includes at least one vibrator.

[0019] FIG. 4 is a functional configuration diagram of the vibration perception position control device 10. The vibration perception position control device 10 includes a localization information acquisition unit 11, a vibration parameter information acquisition unit 12, a guidance parameter information acquisition unit 13, an audio-visual localization setting unit 14, a vibration signal generation unit 15, a synchronization control unit 16, an audio control unit 17, a vibration control unit 18, and a guidance control unit 19. Each of these processing function units is realized by a CPU 101 and a memory 102 that constitute a computer. Specifically, each processing function unit is realized by the CPU 101 executing the processes described in the vibration perception position control program stored in the memory 102. Note that these processing function units may be realized in various other forms including integrated circuits such as ASICs and FPGAs.

[0020] The localization information acquisition unit 11 acquires localization information. The localization information is information indicating the direction of the sound source of each sound included in the sound data. Specifically, the localization information is information associating a value indicating the time when each sound included in the sound data is generated and a value indicating the direction of the sound source at an angle with the user's front direction as 0 degrees. The sound data is generated by another system based on the content provided by the service providing side, and the localization information is generated by another system based on this sound data. Thus, the localization information can be set on the service providing side. The localization information acquisition unit 11 acquires this localization information via a communication device 103 or the like.

[0021] The vibration parameter information acquisition unit 12 acquires vibration parameter information. The vibration parameter information is information indicating the vibration of the vibration device 105. When the vibration device 105 is composed of a plurality of vibrators, the vibration parameter information is information indicating the vibration of each vibrator. The vibration parameter information can include the frequency, length, etc. of the vibration. The vibration parameter information acquisition unit 12 acquires, for example, the vibration parameter information generated by another system based on the content provided by the service providing side via a communication device 103 or the like. Thus, the vibration parameter information can also be set on the service providing side.

[0022] The induction parameter information acquisition unit 13 acquires body movement induction parameter information. The body movement induction parameter information is information indicating the user's body movement. For example, the body movement induction parameter information can include gripping force such as strongly gripping the vibration perception position control device 10 with the entire palm or lightly gripping it after releasing the force, the direction of the palm, that is, the arm, the inclination angle of the z-axis direction in the xyz coordinates of the palm with respect to the gravity direction, etc. Specifically, the body movement induction parameter information may be voice data indicating the instruction content. The induction parameter information acquisition unit 13 acquires, for example, the body movement induction parameter information generated by another system based on the content provided by the service providing side via a communication device 103 or the like. Thus, the body movement induction parameter information can also be set by the service providing side.

[0023] The sound image localization setting unit 14 makes settings for localizing the sound image in the drive signal of the sound device 104 based on the localization information. When the sound device 104 is an external device such as headphones or earphones, for example, it is assumed that the relative positional relationship between the user and the vibration perception position control device 10, such as the gripping posture of the vibration perception position control device 10, is assumed in advance. For example, as shown in FIG. 2, the user can stipulate that by gripping the vibration perception position control device 10 with one hand with the palm facing upward, the positive x-axis direction corresponds to the right direction and the positive y-axis direction corresponds to the front direction of the body. In such a gripping state, for example, when the angle of the sound source is θ and the x-axis direction (angle θ = 90 degrees) is the direction of the sound source, it is assumed that the x-axis direction is on the right side of the user in the relative positional relationship between the user and the vibration perception position control device 10. Therefore, in such a case, the sound image localization setting unit 14 sets to emit sound only from the earphone in contact with the right ear. The settings for localizing the sound image may be a known method, for example, settings in which the volume of a plurality of speakers or the like is adjusted.

[0024] The vibration signal generation unit 15 generates a vibration signal based on the vibration parameter information. When the vibration device 105 includes a plurality of vibrators, the vibration signal generation unit 15 generates the vibration signals of the respective vibrators. For example, the vibration signal generation unit 15 generates a vibration signal in a form that attenuates a sine wave of 200 Hz within 0.15 seconds.

[0025] The synchronization control unit 16 performs control to synchronize the timing at which sound and vibration are generated. Note that the localization information, body movement guidance parameter information, and vibration parameter information are information specified in time series, and the time at which sound and vibration are generated is specified. Therefore, the synchronization control unit 16 performs synchronization control according to the time specified in the localization information, body movement guidance parameter information, and vibration parameter information.

[0026] The sound control unit 17 controls the acoustic device 104 according to the drive signal at the timing controlled by the synchronization control unit 16.

[0027] The vibration control unit 18 controls the vibration of the vibration device 105 according to the vibration signal at the timing controlled by the synchronization control unit 16.

[0028] The guidance control unit 19 controls the communication device 103 at the timing controlled by the synchronization control unit 16, transmits a guidance voice signal to an external presentation device such as a speaker, and presents the instruction content by voice.

[0029] Next, the processing operation of the vibration perception position control device 10 configured as described above will be described. FIG. 5 is a flowchart showing an example of the flow of vibration perception position control processing in the vibration perception position control device 10. The vibration perception position control device 10 starts the vibration perception position control processing when the power is turned on or an instruction is received via the communication device 103 or the like. Note that the vibration perception position control device 10 may store in advance in the memory 102 data indicating the pitch of the sound for each time, or may receive it via the communication device 103 or the like.

[0030] The positioning information acquisition unit 11 acquires positioning information via a communication device 103 or the like (step S11). Specifically, the positioning information is information including a combination of time and angle θ in time series. FIG. 6 is a schematic diagram for explaining the angle in the positioning information. As shown in FIG. 6, the angle θ is the angle of the sound source with the front direction of the user as the reference 0 degrees. The right ear direction of the user is θ = 90 degrees, and the left ear direction of the user is θ = -90 degrees. An example of the positioning information is ((t1, -90), (t2, 0), (t3, 90)). In this case, the acoustic device 104 emits a sound with θ = -90 degrees as the sound source at time t1, emits a sound with θ = 0 degrees as the sound source at time t2, and emits a sound with θ = 90 degrees as the sound source at time t3.

[0031] Next, the audio-visual positioning setting unit 14 performs settings for positioning the audio-visual based on the positioning information (step S12).

[0032] Subsequently, the induction parameter information acquisition unit 13 acquires body movement induction parameter information via a communication device 103 or the like (step S13). Specifically, the body movement induction parameter information is information including a combination of time and voice data indicating the instruction content regarding the user's body movement in time series.

[0033] Next, the vibration parameter information acquisition unit 12 acquires vibration parameter information via a communication device 103 or the like (step S14). Specifically, the vibration parameter information is information including a combination of time and an identifier indicating a vibrator (hereinafter referred to as a vibrator ID) in time series. When there is one vibrator, the vibrator ID may not be present. An example of the vibration parameter information is ((t1, v2), (t2, v1, v2)). v1 and v2 are examples of vibrator IDs. In this case, the vibration device 105 vibrates the vibrator v2 at time t1, and vibrates the vibrators v1 and v2 at time t2.

[0034] Subsequently, the vibration signal generation unit 15 generates a vibration signal based on the vibration parameter information (step S15).

[0035] Here, when the vibration device 105 is composed of a plurality of vibrators, the vibration signal generation unit 15 assigns vibrations to each vibrator (step S16). That is, the vibration signal generation unit 15 generates a vibration signal for each vibrator. Note that when the vibration device 105 is composed of a single vibrator, the vibration signal generation unit 15 does not have to execute the process of this step S16.

[0036] Next, the synchronization control unit 16 synchronizes the timings of the sound and the vibration (step S17). Specifically, the synchronization control unit 16 performs control for synchronization according to the specification of the time indicated in the localization information, the body movement guidance parameter information, and the vibration parameter information.

[0037] Subsequently, the guidance control unit 19 controls the guidance of the user's body movement at the timing controlled by the synchronization control unit 16 (step S18). Specifically, the guidance control unit 19 controls the communication device 103 at the timing controlled by the synchronization control unit 16 to transmit a guidance voice signal to an external presentation device such as a speaker, and causes the presentation device to output a sound for guiding the user's body movement.

[0038] Next, the sound control unit 17 and the vibration control unit 18 control the sound and the vibration at the timing controlled by the synchronization control unit 16 (step S19). Specifically, the sound control unit 17 controls the acoustic device 104 according to the drive signal at the timing controlled by the synchronization control unit 16. Also, the vibration control unit 18 controls the vibration of the vibration device 105 according to the vibration signal at the timing controlled by the synchronization control unit 16.

[0039] FIG. 7 is a diagram for explaining an example of the localization information and the vibration parameter information. The localization information corresponding to FIG. 7 is, for example, ((t1, -90), (t2, 0), (t3, 90), (t4, 0), (t5, -90)). Also, the vibration parameter information is, for example, ((t1, v1), (t2, v1, v2), (t3, v2), (t4, v1, v2), (t5, v1, v2)).

[0040] Based on such orientation information and vibration parameter information, at time t1, the acoustic device 104 emits sound so as to make the user perceive the sound source at an azimuth of -90 degrees. The vibrator v1 included in the vibration device 105 vibrates, and the vibrator v2 does not vibrate. At time t2, the acoustic device 104 emits sound so as to make the user perceive the sound source at an azimuth of 0 degrees. The vibrators v1 and v2 included in the vibration device 105 vibrate. At time t3, the acoustic device 104 emits sound so as to make the user perceive the sound source at an azimuth of 90 degrees. The vibrator v1 included in the vibration device 105 does not vibrate, and the vibrator v2 vibrates. At time t4, the acoustic device 104 emits sound so as to make the user perceive the sound source at an azimuth of 0 degrees. The vibrators v1 and v2 included in the vibration device 105 vibrate. Then, at time t5, the acoustic device 104 emits sound so as to make the user perceive the sound source at an azimuth of -90 degrees. The vibrators v1 and v2 included in the vibration device 105 vibrate.

[0041] Note that a part of the above-mentioned various types of information may be in a format compliant with the MIDI (Musical Instrument Digital Interface) standard.

[0042] FIG. 8 is a diagram for explaining another example of the orientation information and the vibration parameter information. The sound source and the vibration presented synchronously are not limited to the discrete presentation as shown in FIG. 7, and may be a continuous presentation as shown in FIG. 8.

[0043] When the sound and the vibration are synchronized as described above, even if there is only one vibrator included in the vibration device 105, in the vibration perception position control device 10, it is possible to make the user have an illusion that the part corresponding to the azimuth of each sound source vibrates.

[0044] FIG. 9A is a schematic diagram for explaining the pressure sensation stimulation region R and the vibration perception position P when the vibration perception position control device 10 is held in a state where the force is released. Further, FIG. 9B is a schematic diagram for explaining the pressure sensation stimulation region R and the vibration perception position P when the vibration perception position control device 10 is strongly gripped with the entire palm. The pressure sensation stimulation region R is a region where the magnitude of the pressure sensation stimulation is equal to or greater than a certain level. When the user holds the vibration perception position control device 10 in a state where the force is released as shown in FIG. 9A, the pressure sensation stimulation region R depends on gravity, that is, it depends on the inclination angle of the palm with respect to the direction of gravity. Hereinafter, this inclination angle of the palm with respect to the direction of gravity will be referred to as the "angle of the palm". When the user strongly grips the vibration perception position control device 10 with the entire palm as shown in FIG. 9B, the pressure sensation stimulation region R will cover almost the entire palm. In this case, the pressure sensation stimulation region R has less dependence on the angle of the palm.

[0045] FIG. 10A is a schematic diagram for explaining the vibration perception position P when the palm is in the vertical direction. Further, FIG. 10B is a schematic diagram for explaining the vibration perception position P when the palm is in the horizontal direction. The vibration perception position P is the position where the user feels vibration. The vibration perception position P changes according to the sound emitted by the sound device 104, that is, the direction of the sound source. When the palm is in the vertical direction as shown in FIG. 10A, that is, in a direction where the fingertips of the palm face the front direction of the user, and the user holds the vibration perception position control device 10, as indicated by the double-headed arrows in the same figure as well as in FIGS. 9A and 9B, by changing the direction of the sound source, the vibration perception position P can be changed in the left-right direction of the palm. For example, when the sound device 104 emits a sound so as to make the user feel that the sound source is at an azimuth of -90 degrees and vibrates the vibrator included in the vibration device 105, the user can be made to have the illusion that vibration is occurring on the left side of the palm. Further, when the sound device 104 emits a sound so as to make the user feel that the sound source is at an azimuth of 90 degrees and vibrates the vibrator included in the vibration device 105, the user can be made to have the illusion that vibration is occurring on the right side of the palm. Thus, even though the same vibrator generates the same vibration, the user can be made to have the illusion that the portion corresponding to the direction of the sound source is vibrating.

[0046] Also, as shown in FIG. 10B, when the user holds the vibration perception position control device 10 with the palm facing sideways, that is, in a direction where the fingertips point in the left direction of the user, the vibration perception position P can be changed in the vertical direction (fingertip / wrist direction) of the palm by changing the direction of the sound source, as indicated by the double-headed arrow in the figure. For example, when the sound device 104 emits a sound so as to make the user feel that the sound source is at a direction of -90 degrees and the vibrator included in the vibration device 105 is vibrated, the user can be made to have an illusion that it is vibrating on the upper side (base of the finger) of the palm. Also, when the sound device 104 emits a sound so as to make the user feel that the sound source is at a direction of 90 degrees and the vibrator included in the vibration device 105 is vibrated, the user can be made to have an illusion that it is vibrating on the lower side (near the wrist of the palm) of the palm. In this way, even when the orientation of the palm is changed, the user can be made to have an illusion that a location corresponding to the direction of the sound source is vibrating on the extension line of the vector connecting the two points of both ears.

[0047] Therefore, by changing the way of holding the vibration perception position control device 10, the angle and orientation of the palm, even though the same vibrator generates the same vibration, the user can be made to perceive the vibration at various positions and in various size ranges of the palm. Thus, in addition to presenting the sound source and the vibration in synchronization, the vibration perceived by the user can be controlled by guiding the user's body movement indicating how to hold the vibration perception position control device 10, the angle and orientation of the palm.

[0048] As described in detail above, the vibration perception position control device 10 according to the first embodiment of the present invention utilizes the effect of cross-modal perception, which is a phenomenon in which the perception in touch changes by complementing the tactile information with the simultaneously presented auditory information. The vibration perception position control device 10 presents vibration in synchronization with sound on the palm according to the localization information and vibration parameter information provided by the system side based on the content provided by the service providing side, and controls the localization of the sound for auditory presentation. As a result, the vibration perception position P can be changed on the surface of the palm in contact with the vibration perception position control device 10 and on the extension line of the vector connecting both ears. The vibration perception position control device 10 can change the perception position of the vibration within a pressure sensation stimulation region R, which is a region where the pressure sensation stimulation depending on the way of holding the vibration perception position control device 10 with the palm is above a certain level. Then, the vibration perception position control device 10 controls the change direction and the change region range of the vibration perception position P by inducing the way of holding the palm, the orientation, the angle, etc. according to the induction parameter information provided by the system side based on the content provided by the service providing side.

[0049] In addition, the audio-visual localization setting unit 14 performs setting for localizing the audio-visual so as to make the user perceive that the vibration perception position P changes on the extension line of the vector connecting the user's both ears on the palm surface in contact with the vibration device 105. Specifically, the audio-visual localization setting unit 14 performs setting for localizing the audio-visual in the drive signal of the acoustic device 104 so that the position on the palm surface where the user wants to perceive the vibration from the vibration device 105 is set as the direction of the sound source. As a result, the user can be made to feel that the position of the palm is vibrating.

[0050] In addition, the induction control unit 19 induces the orientation and the angle of the palm surface in contact with the vibration device 105 so as to make the user perceive that the vibration perception position P changes within the pressure sensation stimulation region R, which is a region where the pressure sensation stimulation on the palm surface in contact with the vibration device 105 is above a certain level. As a result, the vibration perception position P can be controlled.

[0051] Specifically, the induction control unit 19 controls the change direction of the vibration perception position P, or the change direction and the change area range, by inducing the direction of the palm surface in contact with the vibration device 105, or the direction and the angle. In this way, by performing induction to cause the user to change the direction of the palm surface, or the direction and the angle, the change direction of the vibration perception position P, or the change direction and the change area range can be changed.

[0052] [Second Embodiment] Next, a second embodiment of the present invention will be described. In the following description, for parts that are the same as those in the first embodiment, the same reference numerals as those used in the first embodiment are given, and the description thereof will be omitted.

[0053] FIG. 11 is a diagram showing a hardware configuration example of the vibration perception position control device 10 according to the second embodiment of the present invention. In the second embodiment, a sensor 107 is further provided with respect to the hardware configuration of the vibration perception position control device 10 according to the first embodiment. The sensor 107 is, for example, an acceleration sensor, a gyro sensor for detecting an angular velocity, or the like.

[0054] FIG. 12 is a functional configuration diagram of the vibration perception position control device 10 according to the second embodiment. In the second embodiment, it further includes a sensor information acquisition unit 20, a center of gravity estimation unit 21, a direction angle estimation unit 22, and a video superimposition position determination unit 23 with respect to the functional configuration of the vibration perception position control device 10 according to the first embodiment.

[0055] The sensor information acquisition unit 20 acquires sensor information that is the detection data of the sensor 107.

[0056] Based on the sensor information acquired by the sensor information acquisition unit 20, the center of gravity estimation unit 21 estimates the center of gravity of the palm, which is the body part in contact with the vibration perception position control device 10. At this time, the center of gravity estimation unit 21 can estimate the center of gravity by using the induction parameter information acquired by the induction parameter information acquisition unit 13. As described in the first embodiment, since the induction parameter information is information indicating the content of the instruction for the user's body movement, this information can be used to know how the palm has moved so far and how it will move in the future. That is, the orientation and angle of the palm at the time when the sensor information is acquired can be assumed based on this induction parameter information. Therefore, based on the assumed orientation and angle of the palm and the actual movement of the palm indicated by the sensor information, it is possible to estimate where the center of gravity of the palm is at the current time.

[0057] Based on the sensor information acquired by the sensor information acquisition unit 20, the direction angle estimation unit 22 estimates the orientation and angle of the palm, which is the target part.

[0058] Based on the estimation result of the center of gravity by the center of gravity estimation unit 21 and the estimation result of the orientation and angle of the palm by the direction angle estimation unit 22, the video superimposition position determination unit 23 calculates the superimposition position at which a video for guiding the user's body movement should be superimposed and displayed on the palm, which is the body part, or in the virtual space. The induction parameter information, which is information indicating the content of the instruction for the user's body movement acquired by the induction parameter information acquisition unit 13, can be induction video content including instruction content in the form of sound and video.

[0059] Therefore, in this second embodiment, the synchronization control unit 16 performs control to synchronize the timing of generating sound and vibration with the timing of generating video. That is, the synchronization control unit 16 controls the timing of the drive signal of the video based on the information on the superimposition position calculated by the video superimposition position determination unit 23. The guidance control unit 19 controls the communication device 103 at the timing controlled by the synchronization control unit 16 to transmit a video content signal to a presentation device such as a head-mounted display for AR (augmented reality) or VR (virtual reality), and present guidance video content indicating the instruction content by video and sound. That is, the presentation device superimposes and displays a video on the user's palm by AR, or superimposes and displays a video at an arbitrary position in the virtual space by VR.

[0060] FIG. 13 is a flowchart showing an example of the flow of vibration perception position control processing in the vibration perception position control device 10 according to the second embodiment. Similar to the first embodiment, the localization information acquisition unit 11 acquires localization information via the communication device 103 or the like (step S11). Next, the audio-visual localization setting unit 14 performs setting for localizing audio-visual based on the localization information (step S12).

[0061] Subsequently, the sensor information acquisition unit 20 acquires sensor information from the sensor 107 (step S21).

[0062] Next, the direction angle estimation unit 22 estimates the direction and angle of the palm at the current time based on the sensor information acquired by the sensor information acquisition unit 20 (step S22).

[0063] Subsequently, similar to the first embodiment, the guidance parameter information acquisition unit 13 acquires body movement guidance parameter information via the communication device 103 or the like (step S13).

[0064] Next, the center of gravity estimation unit 21 estimates the center of gravity of the palm, which is the body part in contact with the vibration perception position control device 10, based on the guidance parameter information acquired by the guidance parameter information acquisition unit 13 and the sensor information acquired by the sensor information acquisition unit 20 (step S23).

[0065] Subsequently, the video superimposition position determination unit 23 determines the video superimposition position where the video of the guidance video content should be superimposed based on the center of gravity of the palm estimated by the center of gravity estimation unit 21 and the orientation and angle of the palm estimated by the orientation angle estimation unit 22 (step S24).

[0066] Next, the vibration parameter information acquisition unit 12 acquires vibration parameter information via the communication device 103 or the like (step S14). Subsequently, the vibration signal generation unit 15 generates a vibration signal based on the vibration parameter information (step S15). Here, when the vibration device 105 is composed of a plurality of vibrators, the vibration signal generation unit 15 assigns vibrations to each vibrator (step S16).

[0067] Next, the synchronization control unit 16 synchronizes the timings of the video, sound, and vibration (step S25). Specifically, the synchronization control unit 16 performs synchronization control according to the designation of the time indicated in the localization information and the vibration parameter information, the time indicated in the guidance parameter information, and the video superimposition position determined by the video superimposition position determination unit 23.

[0068] Subsequently, the sound control unit 17, the vibration control unit 18, and the guidance control unit 19 control the video, sound, and vibration at the timings controlled by the synchronization control unit 16 (step S26). Specifically, the sound control unit 17 controls the acoustic device 104 according to the drive signal at the timings controlled by the synchronization control unit 16. Also, the vibration control unit 18 controls the vibration of the vibration device 105 according to the vibration signal at the timings controlled by the synchronization control unit 16. Further, the guidance control unit 19 controls the communication device 103 at the timings controlled by the synchronization control unit 16 to transmit a video content signal to an external presentation device and output guidance video content for guiding the user's body movements to the presentation device.

[0069] As described in detail above, in the vibration perception position control device 10 according to the second embodiment of the present invention, there are provided a direction angle estimation unit 22 that estimates the orientation and angle of the palm surface, a center of gravity estimation unit 21 that estimates the center of gravity of the contact surface with the 105 vibration device on the palm surface, and a video superimposition position determination unit 23, a synchronization control unit 16, and a guidance control unit 19 that serve as a superimposition unit that superimposes video content on an arbitrary position on the user's palm or in the virtual space based on the estimation results of the orientation and angle of the palm surface and the estimation result of the center of gravity. Therefore, in addition to sound, it is possible to guide the user's body movements also by video.

[0070] [Other Embodiments] Note that the present invention is not limited to the above-described embodiments.

[0071] For example, the vibration perception position control device 10 does not have to be a housing having a spherical shape as shown in FIG. 1, and may be a quadrangular prism. Specifically, the vibration perception position control device 10 may be, for example, a mobile terminal such as a smartphone.

[0072] When the vibration perception position control device 10 is a mobile terminal such as a smartphone, each of the above-described processes may be defined in an application program installed in the smartphone or the like. That is, in the first and second embodiments, the vibration perception position control device 10 acquires induction parameter information and the like provided by the system side based on the content provided by the service providing side via the communication device. However, the vibration perception position control device 10 can generate such induction parameter information and the like based on the content. That is, the vibration perception position control device 10 may be a system that plays content.

[0073] Also, the vibration perception position control device 10 may be configured such that each part constituting the vibration perception position control device 10 is dispersed and housed in a plurality of housings. FIG. 14 is a diagram showing a hardware configuration example when the vibration perception position control device 10 according to the second embodiment is configured by two units, a control unit 10A and a vibration unit 10B. The control unit 10A includes a CPU 101, a memory 102, a communication device 103A, and an acoustic device 104, and the vibration unit 10B includes a communication device 103B, a vibration device 105, and a sensor 107. The vibration unit 10B can have a housing having a spherical shape as shown in FIG. 1. The acoustic device 104 may also be a separate unit. Similarly, the vibration perception position control device 10 according to the first embodiment can be dispersed into a plurality of units.

[0074] Also, the flow of each process described with reference to the flowcharts of FIGS. 5 and 13 is not limited to the described procedure, and the order of some steps may be interchanged, some steps may be performed simultaneously, or the processing content of some steps may be modified.

[0075] Also, the vibration signal generation unit 15 may generate a vibration signal such that the intensity of the vibration corresponding to each sound repeats between strong and weak. As a result, since it is possible to make the user feel that something has changed for each sound, it becomes easier to create an illusion that the location corresponding to the direction of each sound source is vibrating.

[0076] In addition, the methods described in each embodiment can be stored as a processing program (software means) to be executed by a computer on a recording medium such as a magnetic disk (e.g., a floppy (registered trademark) disk, a hard disk, etc.), an optical disk (e.g., a CD-ROM, a DVD, an MO, etc.), or a semiconductor memory (e.g., a ROM, a RAM, a flash memory, etc.), and can also be transmitted and distributed through a communication medium. Note that the program stored on the medium side includes a setting program for configuring software means (including not only an execution program but also tables and data structures) to be executed by a computer in the computer. The computer that realizes this device reads the program recorded on the recording medium, and in some cases, constructs software means using the setting program, and executes the above-described processing by having its operation controlled by this software means. Note that the recording medium referred to in this specification includes not only a medium for distribution but also storage media such as a magnetic disk and a semiconductor memory provided inside a computer or in a device connected via a network.

[0077] In short, the present invention is not limited to the above-described embodiments as they are, and at the implementation stage, components can be modified and embodied without departing from the gist thereof. Also, various inventions can be formed by appropriately combining a plurality of components disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

Explanation of Reference Numerals

[0078] 10…Vibration perception position control device 10A…Control unit 10B…Vibration unit 11…Positioning information acquisition unit 12…Vibration parameter information acquisition unit 13…Induction parameter information acquisition unit 14…Audio-visual positioning setting unit 15…Vibration signal generation unit 16…Synchronization control unit 17…Sound control unit 18…Vibration control unit 19…Induction control unit 20…Sensor information acquisition unit 21…Center of gravity estimation unit 22…Direction angle estimation unit 23…Video overlay position determination unit 101…CPU 102…Memory 103, 103A, 103B…Communication device 104…Audio device 105…Vibration device 106…Bus 107…Sensor M…Direction mark P…Vibration perception position R…Pressure sensation stimulation area

Claims

**Claim 1**: An apparatus that, even when the position for applying vibration to the user's palm is the same, utilizes the human illusion to make the user perceive that an arbitrary position on the palm surface is vibrating by synchronously emitting sound and vibration. An audio localization setting unit that sets the position on the palm surface where vibration is to be perceived by the user as the direction of the sound source and performs settings for localizing the sound image in the drive signal of the audio device. A vibration signal generation unit that generates a vibration signal for the vibrator of the vibration device that applies vibration to the user's palm surface. A synchronization control unit that performs control to synchronize the timing of generating sound and vibration. A sound control unit that controls the audio device according to the drive signal. A vibration control unit that controls the vibration device according to the vibration signal. A vibration perception position control device comprising the above. **Claim 2** The audio localization setting unit performs settings for localizing the sound image so that the perceived position of the vibration changes on the extension line of the vector connecting the user's both ears on the palm surface in contact with the vibration device. The vibration perception position control device according to claim 1. **Claim 3**: Further comprising a guidance control unit that controls the presentation of guidance information by voice or video for guiding the orientation and angle of the palm surface. The guidance control unit guides the orientation and angle of the palm surface in contact with the vibration device so that the perceived position of the vibration is perceived to change within a region where the pressure sensation stimulus on the palm surface in contact with the vibration device is above a certain level. The vibration perception position control device according to claim 1 or 2. **Claim 4**: Further comprising a guidance control unit that controls the presentation of guidance information by voice or video for guiding the orientation and angle of the palm surface. The guidance control unit controls the changing direction of the perceived position of the vibration by guiding the orientation of the palm surface in contact with the vibration device. The vibration perception position control device according to claim 1 or 2. **Claim 5**: A guidance control unit that controls the presentation of guidance information by video for guiding the orientation and angle of the palm surface. A direction angle estimation unit that estimates the orientation and angle of the palm surface; A center of gravity estimation unit that estimates the center of gravity of the contact surface with the vibration device on the palm surface; An overlapping unit that overlaps video content at an arbitrary position on the user's palm or in virtual space based on the estimation result of the orientation and angle of the palm surface and the estimation result of the center of gravity; further comprising; The synchronization control unit performs control to synchronize the timing of generating sound and vibration with the video content. The vibration perception position control device according to claim 1.

6. A method executed by a device that utilizes the human illusion to make the user perceive that an arbitrary position on the palm surface is vibrating by synchronizing sound and vibration even when the position where vibration is applied to the user's palm is the same, a step of setting the position on the palm surface where the user is to perceive vibration as the direction of the sound source and performing settings for localizing the sound image in the drive signal of the sound device; a step of generating a vibration signal of a vibrator of a vibration device that applies vibration to the user's palm surface; a step of performing control to synchronize the timing of generating sound and vibration; a step of controlling the sound device according to the drive signal; a step of controlling the vibration device according to the vibration signal; comprising a vibration perception position control method.

7. A vibration perception position control program that causes a computer to execute the processing by each part of the vibration perception position control device according to any one of claims 1 to 5.

Images