Vibration signal generation device

Abstract

Even in the case of sound that lacks low-frequency components, it generates vibrations based on the sound. [Solution] Envelope information relating to the envelope of an audio signal is derived, and frequency modulation and amplitude modulation based on the envelope information are applied to a fundamental signal which is a wave with constant frequency and amplitude to generate a vibration signal for vibrating a vibration generating device.

Description

Technical Field

[0001] The present invention relates to a vibration signal generation device.

Background Art

[0002] By applying vibrations generated based on sound to the body, it is possible to make the body feel the sound. Since the frequency range of vibrations that humans can feel is the low-frequency range among the frequency ranges that humans can hear, conventionally, only the low-frequency range of a sound signal has been extracted, and vibrations have been generated based on the signal in the extracted low-frequency range (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the method as described above, in the case of a sound having no component in the low-frequency range, vibrations based on the sound cannot be generated.

[0005] As an example of the problem to be solved by the present invention, it is possible to generate vibrations based on sound even in the case of a sound having no component in the low-frequency range.

Means for Solving the Problems

[0006] In order to solve the above problems, the invention according to claim 1 is a vibration signal generation device that generates a vibration signal for vibrating a vibration generation device, and includes an envelope information derivation unit that derives envelope information regarding an envelope of a sound signal, and a vibration signal generation unit that performs frequency modulation and amplitude modulation based on the envelope information on a basic signal that is a wave having a constant frequency and amplitude to generate a vibration signal.

[0007] The invention described in claim 8 is a vibration signal generation method performed by a computer for generating a vibration signal to vibrate a vibration generating device, comprising: an envelope information derivation step of deriving envelope information relating to the envelope of an audio signal; and a vibration signal generation step of performing frequency modulation and amplitude modulation based on the envelope information on a fundamental signal which is a wave with constant frequency and amplitude to generate a vibration signal.

[0008] The invention described in claim 9 involves having a computer execute the vibration signal generation method described in claim 8.

[0009] The invention described in claim 10 stores the vibration signal generation program described in claim 9. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows a vibration generation system according to one embodiment of the present invention. [Figure 2] This diagram illustrates the envelope E(t) of an audio signal. [Figure 3] This diagram illustrates the relationship between envelope level and frequency, and the resulting vibrational signal. [Figure 4] This diagram illustrates the relationship between envelope level and frequency, and the resulting vibrational signal. [Figure 5] This figure shows an example of processing operation in a vibration signal generation device 100 according to one embodiment of the present invention. [Figure 6] This figure shows an audio output and vibration generation system according to one embodiment of the present invention. [Figure 7] This diagram illustrates the relationship between the level and frequency of the envelope. [Modes for carrying out the invention]

[0011] A vibration signal generating device according to one embodiment of the present invention is a vibration signal generating device that generates a vibration signal for vibrating a vibration generator, and comprises: an envelope information derivation unit that derives envelope information relating to the envelope of an audio signal; and a vibration signal generating unit that generates a vibration signal by performing frequency modulation and amplitude modulation based on the envelope information on a fundamental signal which is a wave with constant frequency and amplitude. Therefore, in this embodiment, it is possible to generate a vibration signal based on audio even in the case of audio that does not have components in the low frequency range. Furthermore, in this embodiment, frequency modulation is performed based on the level of the envelope of the audio signal. Therefore, the frequency of the generated vibration signal changes according to the level of the envelope. Thus, in this embodiment, it is possible to generate non-monotonous vibrations based on audio even in the case of audio that does not have components in the low frequency range.

[0012] The vibration signal generating device may further include an extraction unit that extracts low-frequency components from the audio signal, and a mixing unit that mixes the extracted low-frequency component signals with the vibration signal. In this way, if the audio contains low-frequency components, it becomes possible to deliver vibrations of the low-frequency components of the audio signal to the listener of the audio, resulting in a vibration with an attack that follows the audio to the user.

[0013] The vibration signal generation unit may perform frequency modulation such that the frequency increases as the level of the envelope increases. In this way, it becomes possible to emphasize high-frequency vibrations more than low-frequency vibrations.

[0014] The vibration signal generation unit may perform frequency modulation such that the frequency decreases as the level of the envelope increases. In this way, it becomes possible to emphasize low-frequency vibrations more than high-frequency vibrations.

[0015] The information processing device may further include a mode determination unit that determines one mode from a plurality of modes, and the vibration signal generation unit may perform frequency modulation based on the mode determined by the mode determination unit. In this way, for example, if a plurality of modes relating to changes in the physical and mental state are provided, it becomes possible to provide the user with vibrations appropriate to their physical and mental state.

[0016] The aforementioned multiple modes include a first mode intended for relaxation, and the vibration signal generation unit may perform frequency modulation such that the frequency decreases as the level of the envelope increases, provided that the determined mode is the first mode. In this way, lower frequency vibrations are emphasized more than higher frequency vibrations. This makes it possible to make the user feel lower frequency vibrations more strongly and relax the user.

[0017] The aforementioned multiple modes include a second mode intended for an awakening effect, and the vibration signal generation unit may perform frequency modulation such that the frequency increases as the level of the envelope increases, provided that the determined mode is the second mode. In this way, vibrations with higher frequencies are emphasized more than vibrations with lower frequencies. This makes it possible to make the user feel the higher frequency vibrations more strongly and awaken the user.

[0018] Also, a vibration signal generation method according to an embodiment of the present invention is a vibration signal generation method executed by a computer to generate a vibration signal for vibrating a vibration generator, the method including an envelope information derivation step of deriving envelope information regarding an envelope of an audio signal, and a vibration signal generation step of performing frequency modulation and amplitude modulation based on the envelope information on a basic signal that is a wave with a constant frequency and amplitude to generate a vibration signal. Therefore, in this embodiment, even in the case of an audio signal without components in the low-frequency region, it is possible to generate a vibration signal based on this audio signal. Also, in this embodiment, frequency modulation is performed based on the level of the envelope of the audio signal. Therefore, the frequency of the generated vibration signal changes according to the level of the envelope. Thus, in this embodiment, even in the case of an audio signal without components in the low-frequency region, it is possible to generate non-monotonic vibrations based on this audio signal.

[0019] Also, a vibration signal generation program according to an embodiment of the present invention causes a computer to execute the above-described vibration signal generation method. Therefore, in this embodiment, it is possible to use a computer to generate a vibration signal based on an audio signal even in the case of an audio signal without components in the low-frequency region.

[0020] Also, a computer-readable storage medium according to an embodiment of the present invention stores the above-described vibration signal generation program. Therefore, in this embodiment, the above-described vibration signal generation program can be distributed alone in addition to being incorporated into a device, and it is possible to easily perform version updates and the like.

Example

[0021] <Vibration Generation System> FIG. 1 is a diagram showing a vibration generation system according to an embodiment of the present invention. The vibration signal generation system includes a vibration signal generation device 100 and a vibration generation device 200. The vibration signal generation device 100 generates a vibration signal based on an audio signal, and the vibration generation device 200 generates vibrations based on the vibration signal generated by the vibration signal generation device 100.

[0022] The vibration signal generation device 100 is composed of a computer having a CPU and the like, and includes an audio signal acquisition unit 110, an envelope information derivation unit 120, and a vibration signal generation unit 130.

[0023] The audio signal acquisition unit 110 acquires an audio signal. For example, the audio signal acquisition unit 110 may acquire an audio signal by receiving an audio signal output from another device, or it may acquire audio data stored in a storage device, a storage medium such as a CD, or the cloud, and acquire an audio signal by generating an audio signal from this acquired audio data.

[0024] The envelope information derivation unit 120 derives envelope information relating to the envelope of the audio signal acquired by the audio signal acquisition unit 110. The envelope information signal 120 derives the envelope of the audio signal as a function of time E(t), for example, as shown in Figure 2. Methods for deriving the envelope of a signal include, for example, peak hold processing and absolute value averaging.

[0025] The vibration signal generation unit 130 generates a vibration signal by applying frequency modulation and amplitude modulation to a fundamental signal, which is a wave with constant frequency and amplitude, based on the envelope information acquired by the envelope information derivation unit 120, and outputs it to the vibration generator 200. At this time, the vibration signal generation unit 130 performs frequency modulation so that the frequency of the generated vibration signal is in the low-frequency range (for example, 20-100 Hz).

[0026] The fundamental vibration is, for example, a sine wave (Asin(ωt)) with constant frequency ω and amplitude A. The vibration signal generation unit 130 generates a vibration signal (E(t)sin(Ω(E(t))t)) by, for example, changing the frequency ω of this fundamental vibration based on the level of the envelope (E(t)) (frequency modulation based on envelope information: ω=Ω(E(t))) and changing the amplitude A to match the envelope (amplitude modulation based on envelope information: A=E(t)).

[0027] In this case, the vibration signal generation unit 130 may perform frequency modulation such that the frequency increases as the envelope level increases (i.e., dω / dE>0), as shown in Figure 3(A), or it may perform frequency modulation such that the frequency decreases as the envelope level increases (i.e., dω / dE<0), as shown in Figure 4(A). Note that in Figures 3(A) and 4(A), the relationship between the envelope level and frequency is linear, but the relationship between the envelope level and frequency is not limited to linear.

[0028] When frequency modulation is performed so that the frequency increases as the envelope level increases, the vibration signal generated by the vibration signal generation unit 130 becomes a dense signal where the envelope level is high and a sparse signal where the envelope level is low, as shown in Figure 3(B). On the other hand, when frequency modulation is performed so that the frequency decreases as the envelope level increases, the vibration signal generated by the vibration signal generation unit 130 becomes a sparse signal where the envelope level is high and a dense signal where the envelope level is low, as shown in Figure 4(B).

[0029] The vibration generator 200 is a device that generates vibrations based on an input vibration signal, and provides the user with vibrations based on the vibration signal. The vibration generator 200 can be embedded in a seat on which the user can sit, for example. Alternatively, the vibration generator 200 may be installed inside a cushion. The user can feel the vibrations by sitting on this cushion or by placing it between their back and the backrest of the seat. The vibration generator 200 may also be installed inside a pouch. The user can feel the vibrations by placing this pouch against a part of their body such as their hands, stomach, chest, or feet.

[0030] Thus, in this embodiment, a vibration signal is generated based on the envelope of the audio signal. Therefore, even in the case of audio that lacks low-frequency components, it is possible to generate a vibration signal based on this audio. Furthermore, in this embodiment, frequency modulation is performed based on the level of the audio signal's envelope. Therefore, the frequency of the generated vibration signal changes according to the level of the envelope. Thus, in this embodiment, even in the case of audio that lacks low-frequency components, it is possible to generate non-monotonous vibrations based on this audio.

[0031] For example, in this embodiment, even those with hearing impairments can perceive sound through vibrations. For instance, in this embodiment, during musical performances, even those with hearing impairments can receive vibrations that correspond to changes in the volume of the music, allowing them to fully appreciate the atmosphere of the performance. Furthermore, in this embodiment, during theatrical performances, even those with hearing impairments can receive vibrations that reproduce the intonation of the lines spoken by the actors, allowing them to more fully experience the atmosphere of the play.

[0032] Figure 5 shows an example of processing operation in a vibration signal generation device 100 according to one embodiment of the present invention. The audio signal acquisition unit 110 acquires an audio signal (step S501). The envelope information derivation unit 120 derives envelope information relating to the envelope of the audio signal acquired by the audio signal acquisition unit 110 (step S502). The vibration signal generation unit 130 generates a vibration signal by performing frequency modulation and amplitude modulation on a fundamental signal, which is a wave with constant frequency and amplitude, based on the envelope information acquired by the envelope information derivation unit 120 (step S503).

[0033] <Audio Output / Vibration Generation System> Figure 6 shows an audio output and vibration generation system according to one embodiment of the present invention. The audio output and vibration generation system shown in Figure 7 further includes an audio signal output device 300 and a speaker 400, in addition to a vibration signal generator 100 and a vibration generator 200. In this audio output and vibration generation system, the audio output device 300 outputs the sound of an audio signal through the speaker 300, the vibration signal generator 100 generates a vibration signal based on this audio signal, and the vibration generator 200 generates vibrations based on this vibration signal. Therefore, in this embodiment, it is possible to allow the user to hear audio and to receive vibrations based on this audio.

[0034] <Mixing of low-frequency components> If the audio contains low-frequency components, it is possible to provide the user with a sense of attack and vibrations that follow the audio by transmitting vibrations of the low-frequency components of the audio signal to the listener. Therefore, the vibration signal generation device 100 may have an extraction unit 140 that extracts low-frequency components (for example, from 20Hz to 100Hz) from the audio signal, and a mixing unit 150 that mixes the vibration signal generated by the vibration signal generation unit 130 with the low-frequency component signal extracted by the extraction unit 140.

[0035] <Relaxation / Awakening Effect> The inventor discovered that when a user listens to music and is also subjected to vibrations in the lower frequency range of the human perceptible frequency range, the user's parasympathetic nervous system becomes more active, meaning the user becomes more relaxed. The inventor also discovered that when a user listens to music and is also subjected to vibrations in the higher frequency range of the human perceptible frequency range, the user's sympathetic nervous system becomes more active, meaning the user becomes more alert.

[0036] Therefore, the vibration signal generating device 100 may further include a mode determination unit 160 that determines one mode from a plurality of modes relating to changes in the state of mind and body. The vibration signal generating unit 130 of the vibration signal generating device 100 may perform frequency modulation based on the mode determined by the mode determination unit 160.

[0037] The multiple modes should ideally include a relaxation mode (first mode) aimed at relaxing the user. If the mode determined by the mode determination unit 160 is the relaxation mode, the vibration signal generation unit 130 should perform frequency modulation such that the frequency decreases as the envelope level increases, as shown in Figure 4(A). This way, lower frequency vibrations are emphasized more than higher frequency vibrations. Therefore, it becomes possible to make the user feel the lower frequency vibrations more strongly, thus relaxing the user.

[0038] Multiple modes may include an awakening mode (second mode) aimed at waking the user. If the mode determined by the mode determination unit 160 is the awakening mode, the vibration signal generation unit 130 should perform frequency modulation such that the frequency increases as the envelope level increases, as shown in Figure 3(A). In this way, vibrations with higher frequencies are emphasized more than vibrations with lower frequencies. Therefore, it becomes possible to make the user feel the higher frequency vibrations more strongly and to wake the user up.

[0039] Furthermore, the multiple modes may include a normal mode (a third mode) that is neither intended for relaxation nor alertness.

[0040] In normal mode, when frequency modulation is performed so that the frequency decreases as the envelope level increases, if the mode determined by the mode determination unit 160 is the arousal mode, the vibration signal generation unit 130 should perform frequency modulation so that the frequency decreases more rapidly as the envelope level increases than in normal mode, as shown in Figure 7(A). In this way, in relaxation mode, lower frequency vibrations are emphasized more than higher frequency vibrations compared to normal mode. Therefore, it becomes possible to make the user feel higher frequency vibrations more and relax the user more.

[0041] In normal mode, when frequency modulation is performed so that the frequency increases as the envelope level increases, if the mode determined by the mode determination unit 160 is the awakening mode, the vibration signal generation unit 130 should perform frequency modulation so that the frequency increases more rapidly as the envelope level increases than in normal mode, as shown in Figure 7(B). In this way, in awakening mode, higher frequency vibrations are emphasized more than lower frequency vibrations compared to normal mode. Therefore, it becomes possible to make the user feel higher frequency vibrations more strongly and awaken the user more effectively.

[0042] Furthermore, the inventor discovered that when a user listens to music and receives vibrations based on this music, but not synchronized with the music, but delayed, the user's parasympathetic nervous system becomes more active, meaning the user relaxes. The inventor also discovered that when a user listens to music and receives vibrations based on this music, but not synchronized with the music, but advanced, the user's sympathetic nervous system becomes more active, meaning the user becomes more alert.

[0043] Therefore, the envelope information derivation unit 120 of the vibration signal generation device 100 may derive the envelope based on the mode determined by the mode determination unit 160.

[0044] The envelope information derivation unit 120 may, for example, use a peak hold time based on the mode determined by the mode determination unit 160 to derive envelope information relating to the envelope of the audio signal through peak hold processing. In this case, it is preferable to make the peak hold time for the first mode longer than the peak hold time for the normal mode, and the peak hold time for the second mode shorter than the peak hold time for the normal mode. By doing so, in relaxation mode, the user can be made to feel that the vibration is delayed compared to the audio, compared to the normal mode, thereby making the user more relaxed. On the other hand, in awakening mode, the user can be made to feel that the vibration is ahead of the audio, compared to the normal mode, thereby making the user more awakened.

[0045] The envelope information derivation unit 120 may, for example, derive the envelope of the audio signal by absolute average using an average time based on the determined mode. In this case, it is preferable to make the average time for the first mode longer than the average time for the normal mode, and the average time for the second mode shorter than the average time for the normal mode. Doing so will make the user feel that the vibration is delayed compared to the audio in the relaxation mode compared to the normal mode, thereby making the user more relaxed, and will make the user feel that the vibration is ahead of the audio in the awakening mode compared to the normal mode, thereby making the user more awakened.

[0046] The mode determination unit 160 may, for example, determine the mode based on user input. In this case, for example, the vibration signal generator 100 may have means for receiving user input. Alternatively, the vibration signal generator 100 may have means for acquiring audio characteristics, and the mode determination unit 160 may determine the mode based on these audio characteristics. Furthermore, the vibration signal generator 100 may have means for learning through machine learning what effects the user seeks when playing audio, and the mode determination unit 160 may determine the mode based on these learning results.

[0047] Furthermore, the vibration signal generator 100 may also have means for acquiring the user's biological information, and the mode determination unit 160 may determine the mode based on the acquired biological information. In this case, it is preferable to acquire information about the user's heart rate as the user's biological information. The information about the user's heart rate may include the user's heart rate and information about the user's heart rate variability (for example, LF (Low Frequency), HF (High Frequency), and the ratio of LF to HF, which is LF / HF). It is possible to know the state of mind and body from the information about heart rate and heart rate variability. For example, when a person is relaxed, their heart rate is low, and when they are awake, their heart rate is high. Therefore, for example, when the biological information indicates that the user is relaxed, the mode determination unit 160 may determine the mode to be awakened mode with the aim of awakening the user, and when the biological information indicates that the user is awake, the mode may be determined to be relaxed mode with the aim of relaxing the user.

[0048] The present invention has been described above with reference to preferred embodiments. Although the present invention has been described with reference to specific examples, various modifications and changes can be made to these examples without departing from the spirit and scope of the invention as described in the claims. [Explanation of Symbols]

[0049] 100 Vibration signal generation device 110 Audio signal acquisition unit 120 Envelope information derivation unit 130 Vibration signal generation unit 140 Extraction part 150 Mixing section 160 Mode Determination Unit 200 Vibration Generator 300 Audio signal output device 400 speakers

Claims

[Claim 1] A method for generating vibration signals, An envelope information derivation process for deriving envelope information related to the envelope of an audio signal, A fundamental signal, which is a wave with constant frequency and amplitude, is subjected to frequency modulation based on the envelope information. A vibration signal generation process that includes a vibration signal generation process that performs amplitude modulation and generates a vibration signal. method.

Images