Vibration presentation device, vibration generation system, vibration presentation program, recording medium storing vibration presentation program, and vibration generation method

Abstract

For low frequency vibrations that are far from the resonance frequency band of an actuator and are difficult to be presented, the actuator is used to present the vibrations that humans can easily perceive (misperceive) as being at a low frequency.　A vibration presentation device (1) presents vibrations in a first frequency band using an actuator (31) having the resonance frequency at a second frequency band, which is greater than the first frequency band, the vibration presentation device (1) comprising at least: an acquisition unit (101) that acquires a signal including the vibrations in the first frequency band in which the resonance frequency is lower than the second frequency band; a calculation unit (102) that determines a maximum value of the vibrations in the first frequency band acquired by the acquisition unit (101), and determines, from the maximum value, a maximum value time, which is the time when said vibrations reach the maximum value; and a control unit (103) that generates a single wave by controlling, on the basis of the maximum value time calculated by the calculation unit (102), the actuator during a transition time, which is the time before and after the maximum value time and includes the maximum value time.

Description

Vibration presentation device, vibration generation system, vibration presentation program, recording medium storing the vibration presentation program, and vibration generation method

[0001] The technology described in this specification relates to a vibration presentation device, a vibration generating system, a vibration presentation program, a recording medium storing the vibration presentation program, and a vibration generating method.

[0002] In recent years, in the fields of smartphones, game consoles, virtual reality (VR) devices, robot operation assistance, and the like, vibration feedback has become increasingly sophisticated in order to enhance bodily sensations and realism (see, for example, Patent Document 1).Since human vibration perception sensitivity is greatest around 200 Hz, vibrators mounted on mobile terminals such as smartphones are often small, linear resonant actuator (LRA) vibrators with a resonant frequency of around 200 Hz that can express a frequency band of about 100 to 300 Hz (see, for example, Patent Document 2).

[0003] In recent years, LRAs have become increasingly broadband, and devices capable of presenting a resonant frequency of approximately 50 to 400 Hz have been developed. There are also devices and game console controllers that set the vibrator at a resonant frequency of 60 Hz to 100 Hz (see, for example, Patent Document 3).

[0004] Japanese Patent Application Laid-Open No. 2003-228453 International Publication No. 2021 / 085506 Special Publication No. 2021-527257 Japanese Patent Application Laid-Open No. 2004-181304

[0005] However, it is difficult to present vibrations in the low frequency range of several tens of hertz, which is far from the resonant frequency, in the LRA with the aforementioned resonant frequency of around 50 Hz to 400 Hz. In order to further enhance the sensation of the low frequency band, the resonant frequency must be set to 50 Hz or less, so the mass of the resonant system must be increased or a separate vibrator must be installed for that purpose, which results in a larger volume occupied by the vibrator.

[0006] Patent Document 4 discloses an invention in which a movable body that moves between biasing means such as springs is adjusted by applying controlled accumulation signals and damping signals to a coil around the movable body, thereby adjusting the amplitude of the movable body and creating a pseudo-vibration formed by an envelope connecting the peaks of the amplitude at a frequency that can be felt by humans, thereby reducing the size of the movable body. However, this depends on the structure of the vibrator, the input of multiple signals is complicated, and it is unclear whether it can be reproduced at frequencies below 50 Hz.

[0007] Above all, the movements that currently need to be reproduced by game consoles, VR devices, etc. are complex, and there is a demand for easy provision of low frequencies, but the technology disclosed in Patent Document 4 makes this extremely difficult, and it cannot be used in particular when online real-time signal reproduction is required.

[0008] For this reason, there is a problem in that it is not easy to reproduce the sensation of low-frequency vibrations of several tens of Hz with a small information terminal such as a smartphone that is equipped only with an LRA with a resonance frequency of around 100 Hz to 300 Hz.

[0009] The technology described in this specification aims to present low-frequency vibrations that are difficult to present because they are outside the resonant frequency band of the actuator installed, without using an additional actuator, by devising ways to control the actuator, etc., so that humans can easily perceive (or be fooled into thinking) that the low frequency vibration is present.

[0010] The vibration presentation device of the present invention is a vibration presentation device that presents vibrations in a first frequency band using an actuator having a resonant frequency in a second frequency band that is greater than the first frequency band, and includes: an acquisition unit that acquires a signal including vibrations in the first frequency band, the resonant frequency of which is lower than the second frequency band; a calculation unit that determines a maximum value of the vibrations in the first frequency band acquired by the acquisition unit and determines, from the maximum value, a maximum value time, which is the time at which the maximum value is reached; and a control unit that controls the actuator within a transition time, which is the time before and after the maximum value time, based on the maximum value time calculated by the calculation unit, to generate a single wave.

[0011] In the present invention, vibrations at frequencies lower than the resonant frequency band that can be presented by the actuator can be converted into vibrations that humans can easily perceive (have an illusion of) as being of that frequency and then presented.

[0012] 1 is a schematic diagram illustrating an example of the configuration of a vibration generating system according to an embodiment.

[0023] Graphs are provided to explain the principle of the process of presenting a substitute vibration for low-frequency vibration by the vibration generating system shown in FIG. 1, where (a) is a graph showing an example of a presented low-frequency signal and a controlled single waveform (short-time pulse waveform), and (b) is a graph showing an example of reverberation vibration generated in a housing (measured by a laser displacement meter).

[0024] Graphs are provided to show the results of a subject experiment on the process of presenting a substitute vibration for low-frequency vibration by the vibration generating system shown in FIG. 1, where (a) is a table showing answer options provided by the subjects, and (b) to (e) are graphs showing answer results when the original (target signal) low-frequency waveform is 10 Hz, 20 Hz, 30 Hz, and 40 Hz, respectively.

[0025] Graphs are provided to explain a first example of the process of calculating amplitude and timing by the vibration generating system shown in FIG. 1, and (b) is a graph showing a second example of the process of calculating amplitude and timing by the vibration generating system shown in FIG. 1. Graphs are provided to explain the process of online calculation of the amplitude and timing of low-frequency vibration. 2A and 2B are block diagrams for explaining a process for generating a first vibration waveform and a second vibration waveform by the vibration generating system shown in Fig. 1. (a) is a block diagram for explaining a process for generating vibration when a vibrator using an actuator is used, and (b) is a block diagram for explaining a process for generating vibration when a mobile terminal is used.

[0013] Hereinafter, embodiments will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications and applications of techniques not explicitly stated in the embodiments. In other words, the present embodiments can be implemented with various modifications within the scope of the spirit thereof.

[0014] Furthermore, each drawing does not necessarily include only the components shown in the drawing, but may include other components. In the drawings below, parts with the same reference numerals indicate the same or similar parts unless otherwise specified.

[0015] [A] Embodiments A-1. System Configuration FIG. 1 is a diagram schematically illustrating an example configuration of a vibration generating system 100 according to an embodiment.

[0016] The vibration generating system 100 includes a vibration presentation device 1 and a vibration sensation device 3 .

[0017] The vibration presentation device 1 includes a CPU 11 , a memory 12 , a storage device 13 , an input unit 41 , an output unit 42 , and a transmission unit 43 , which are all connected to each other via a bus 15 .

[0018] The vibration sensory device 3 includes an actuator 31, a driving unit 32, and a receiving unit 33, and is housed in a housing .

[0019] The vibration presentation device 1 and the vibration sensation device 3 may be provided separately or integrally. Furthermore, a part of the vibration presentation device 1 may be provided in the vibration sensation device 3. Furthermore, when the vibration presentation device 1 and the vibration sensation device 3 are integrally formed, the housing 34 may house the entire vibration presentation device 1.

[0020] The vibration generating system 100 in one example of this embodiment is primarily used in smartphones, game consoles, virtual reality (VR) devices, robots, etc., but may also be applied to chairs, suits, headsets, etc. that include vibration devices.

[0021] The vibration presentation device 1 comprises a central processing unit (CPU) 11 for control and drive, a memory 12 for storing programs etc., a storage device 13, an input unit 41 for acquiring information including signals to be reproduced such as original signals, an output unit 42 for outputting information other than tactile sensations such as vibrations to the user, a transmission unit 43 for communicating information with the vibration sensation device 3, and a bus 15 for connecting them.

[0022] The CPU 11 is a processing device that performs various controls and calculations, and realizes various functions by executing an operating system (OS) and a vibration presentation program stored in the memory 12. That is, the CPU 11 may function as an acquisition unit 101, a calculation unit 102, and a control unit 103, as shown in FIG.

[0023] The CPU 11 is an example of a computer, and exemplarily controls the overall operation of the vibration presentation device 1. The device for controlling the overall operation of the vibration presentation device 1 is not limited to the CPU 11, and may be, for example, any one of an MPU, a DSP, an ASIC, a PLD, an FPGA, or a dedicated processor. Furthermore, the device for controlling the overall operation of the vibration presentation device 1 may be a combination of two or more of a CPU, an MPU, a DSP, an ASIC, a PLD, an FPGA, and a dedicated processor. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. Furthermore, PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

[0024] The memory 12 is a recording medium that stores an operating system (OS) and a vibration presentation program, and is composed of a read only memory (ROM) and a random access memory (RAM).

[0025] The storage device 13 is a device that stores data in a readable and writable manner, and may be, for example, a hard disk drive (HDD), a solid state drive (SSD), or a storage class memory (SCM). The storage device 13 stores information acquired by the input unit 41, signals and tactile signals related to vibrations to be reproduced in advance, various data calculated by the calculation unit 102 using the vibration presentation program, and the like.

[0026] The input unit 41 acquires various information online in the vibration presentation device 1, including acoustic information such as the sounds of music, movies, games, etc., shocks, sensations during operation, vibrations generated when the robot comes into contact with an object, etc. Note that the input unit 41 does not need to be provided if information on how to reproduce tactile sensations is stored in advance in the storage device 13, etc.

[0027] The output unit 42 generates images and sound sources in the vibration presentation device 1 to present to the user information other than that of the vibration sensation device 3. The output unit 42 does not have to be provided if the vibration presentation device 1 does not present images, sound sources, etc. to the user. The output unit 42 may also be provided in the vibration sensation device 3.

[0028] The transmitter 43 is a part that transmits a control signal via the receiver 33 of the vibration sensory device 3 by wire or wirelessly.

[0029] When the vibration presentation device 1 and the vibration sensation device 3 are integrated, the transmission unit 43 may use a communication method such as SPI, I2C, or I2S, or an analog signal such as a voltage. The transmission unit 43 may be built in as a function of the CPU 11.

[0030] In the case of a wired connection, communication methods such as USB, Thunderbolt (registered trademark), Ethernet (registered trademark), and HDMI (registered trademark) may be used.

[0031] In the case of wireless communication, the communication unit may be, for example, a communication unit for Bluetooth (registered trademark), WiFi, or ZigBee (registered trademark), or a communication unit for a wireless LAN (Local Area Network).

[0032] The acquisition unit 101, calculation unit 102, and control unit 103 of the CPU 11 will be specifically described.

[0033] The acquisition unit 101 acquires acoustic information such as the sound of music, movies, games, etc. obtained from the input unit 41, as well as signals such as impact, the sensation when operating, and vibrations generated when the robot comes into contact with an object.

[0034] If a signal or tactile signal related to the vibration to be reproduced is stored in advance in the storage device 13 or the like, the signal or tactile signal is read from the storage device 13 and acquired by the acquisition unit 101 .

[0035] The calculation unit 102 analyzes the haptic signal acquired by the acquisition unit 101, and separates and extracts the first frequency band and the second frequency band, or calculates the maximum value, the time (timing) of the maximum value, and the magnitude of the amplitude of the vibration in the first frequency band. Specific calculation processes will be described later.

[0036] The control unit 103 generates a control signal and controls the actuator. The control unit 103 controls the actuator 31 to reproduce the second frequency band calculated by the calculation unit 102, while controlling the actuator 31 to generate a pulse wave, such as a sine wave or a rectangular wave, having at least a predetermined cycle (hereinafter also referred to as a "single wave cycle time") within a predetermined time period including a local maximum value in the first frequency band, thereby reproducing a simulated first frequency band. Note that, as long as the cycle is within the predetermined time period, the waveform may be a half cycle, 1.5 cycles, or 2 cycles, or a waveform with several cycles of different amplitudes may be output. Note that using a waveform with several cycles of different amplitudes may effectively generate reverberation in the housing 34.

[0037] These generated sine waves, pulse waves, etc. may be collectively referred to as a single-shot waveform. The single-shot waveform period time for generating the single-shot waveform is equal to or shorter than one period of the reverberation vibration generated by the resonant system of the actuator 31 or the resonant system of the actuator 31 and the housing 34, and is preferably, for example, 0.002 seconds or more and 0.02 seconds or less, so that the user can fully experience it, taking into account the frequency of vibration that can be generated by current actuators. The period may also be determined according to the ratio of the period of the vibration waveform acquired by the acquisition unit 101.

[0038] Next, the vibration sensation device 3 will be described. In this embodiment, the vibration sensation device 3 is a component that reproduces and provides to the user acoustic information such as sounds from music, movies, games, etc., as well as tactile signals such as impacts, the sensation of operation, and vibrations generated when a robot comes into contact with an object. The vibration sensation device 3 includes an actuator 31 that generates vibrations, a drive unit 32, and a receiver 33. To reproduce the tactile signals more realistically, the actuator 31 is equipped with an actuator 31 having a resonance system in a high-frequency band (second frequency band) of approximately 50 to 350 Hz, where human vibration perception sensitivity is high. In the present invention, for example, a high-frequency band is reproduced using an actuator 31 with a frequency of 200 Hz ± 150 Hz, and the actuator 31 is further used to convert and present simulated vibrations in a low-frequency band (first frequency band) of 100 Hz or less (10 Hz to 100 Hz, particularly 10 Hz to 50 Hz) so that they can be perceived by humans.

[0039] In this embodiment, the actuator 31 is a voice coil type actuator using a magnet and a coil, such as an LRA (linear resonance actuator). The actuator 31 may be provided inside a housing 34. In this case, the actuator 31 and the housing 34 form a resonance system. The resonance frequency of the resonance system of the actuator 31 may be, for example, 200 Hz±150 Hz. Furthermore, the upper limit of the resonance system of the actuator 31 may be, for example, 350 Hz, and the lower limit may be 50 Hz.

[0040] The actuator 31 is not limited to a voice coil type actuator, and may be configured as a weight in which the actuator 31 is supported and connected by an elastic body within the housing 34. In this case, a resonance system is configured by the actuator 31, the elastic body, the weight, and the housing 34. A battery or various parts within the housing 34 may be used as the weight.

[0041] The driving unit 32 is a component that drives the actuator 31 based on a digital signal or an analog signal that controls the actuator 31 and that is generated by the control unit 103 and received by the receiving unit 33. The driving unit 32 may include an amplifier (in other words, an amplifier) ​​and a feedback circuit (not shown) for driving the actuator 31.

[0042] The receiving unit 33 is a part that receives a control signal transmitted from the transmitting unit 43 of the vibration presentation device 1. The receiving unit 33 may be omitted if the signal from the transmitting unit 43 is sent directly to the driving unit 32.

[0043] A-2. Substitute Vibration Presentation Process A specific method of the present invention for reproducing a simulated first frequency band by using substitute vibrations will now be described.

[0044] Figure 2 is a graph explaining the principle of the process of presenting an alternative vibration to a low-frequency vibration by the vibration generating system 100 shown in Figure 1, where (a) in Figure 2 is a graph showing an example of a presented low-frequency signal and a short-time single waveform to be controlled, and (b) in Figure 2 is a graph showing an example of a reverberation vibration generated in the housing 34 (measured by a laser displacement meter).

[0045] In the graph shown in FIG. 2( a), the horizontal axis represents time (Time [s]) and the vertical axis represents amplitude (Amplitude). The dotted line graph represents the waveform of the original signal (a target signal to be reproduced by the alternative vibration (hereinafter also referred to as the "target signal")) of 10 Hz, and the solid line graph represents a pulse signal (Control pulse signal), which is a single waveform generated by controlling the actuator 31. A short-duration pulse signal is generated as an input waveform by controlling the actuator 31 in accordance with the timing of peaks of the low-frequency signal of the continuous target signal. In the example shown in FIG. 2( a), a sine wave with one cycle is used as the pulse signal input by the actuator 31. A pulse signal is generated in accordance with the peaks of the low-frequency signal, thereby expressing a characteristic low-frequency cycle.

[0046] The controlling pulse signal (in other words, the alternative waveform) is generated by the following equation: where A is the amplitude of the original low frequency, f is the frequency of the alternative stimulus, and t' is the time when the original low frequency takes its maximum value.

[0047] In the graph shown in Figure 2(b), the horizontal axis represents time (Time [s]), the left vertical axis represents the voltage of the original signal and control signal (Original / Control signal [V]), and the right vertical axis represents the displacement due to measured vibration (Measured vibration [μm]). The dotted line represents the original signal (target signal), the thick line represents the pulse signal (Control pulse signal) generated by controlling the actuator 31, and the thin line represents the measured displacement (Measured vibration) generated in the housing 34 of the vibration generation system 100. In response to a short-term pulse input from the actuator 31, reverberation vibration is generated due to the natural vibration of the housing 34 containing the actuator 31. The reverberation vibration takes time to decay. It is believed that this decaying waveform generates a low-frequency sensation that is difficult to reproduce with the actuator 31.

[0048] A-3. Evaluation of the sensation of substitute vibration We investigated the extent to which the aforementioned method of presenting substitute vibration can actually present a low-frequency sensation.

[0049] 3 shows the results of a subject experiment on the presentation process of an alternative vibration to low-frequency vibration by the vibration generating system 100. (a) is a table showing the answer options given by the subjects. (b) is a graph showing the average results of three answers given by the subjects when the low-frequency waveform of the target signal is 10 Hz. (c) is a graph showing the average results of three answers given by the subjects when the low-frequency waveform of the target signal is 20 Hz. (d) is a graph showing the average results of three answers given by the subjects when the low-frequency waveform of the target signal is 30 Hz. (e) is a graph showing the average results of three answers given by the subjects when the low-frequency waveform of the target signal is 40 Hz. (b) to (e) show the maximum and minimum values ​​of the answer options, as well as the interquartile range and median.

[0050] In the subject experiment shown in Figure 3, sine waves of 10 Hz, 20 Hz, 30 Hz, and 40 Hz were set as the low-frequency vibration target signal, and sine waves of 60 Hz, 80 Hz, and 100 Hz were used as the substitute vibration stimuli for each. The subject's subjective experience was evaluated to determine whether the low-frequency vibration target signal could be presented using the above-mentioned substitute vibration presentation processing method. In the subjective experience experiment, to prevent the subject from selecting answers based on information other than the vibration they felt, the subject held a voice coil-type vibrator capable of presenting approximately 10 Hz to 100 Hz in one hand and experienced the vibration. The subjects were seven men and women in their twenties.

[0051] First, to teach the subject the difference between low and high frequencies, a 10 Hz sine wave was presented as the low frequency and an 80 Hz sine wave as the high frequency, repeatedly until the subject could distinguish the difference.

[0052] The stimuli that the subjects were asked to evaluate physically were four types: the target signal, which was the low-frequency vibration itself (Raw Wave); alternative vibrations created using a 60 Hz sine wave; alternative vibrations created using an 80 Hz sine wave; and alternative vibrations created using a 100 Hz sine wave. These were used for the target signal, with low-frequency vibrations of 10 Hz, 20 Hz, 30 Hz, and 40 Hz, respectively.

[0053] In the subjective experience experiment, a stimulus was presented for 5 seconds using a voice coil vibrator, and subjects were asked to respond on a 7-point Likert scale, as shown in Figure 3(a), as to whether they judged the stimulus to be low frequency or high frequency. Note that a score closer to 1 indicates a higher frequency (a physical sensation closer to the 80 Hz instructed to the subject), and a score closer to 7 indicates a lower frequency (a physical sensation closer to the 10 Hz instructed to the subject).

[0054] The stimuli were presented repeatedly as many times as the subject could respond. Each type of stimulus was presented randomly to avoid order effects. The subjects were deafened by white noise and noise-reducing earmuffs during the experiment. The stimuli were output from a voice coil vibrator (VP4, Acouve Laboratory, Inc.) held by the subject via a USB audio interface (ASUS, XONAR U7 MK II) and an audio amplifier (SMSL SA-36A PRO).

[0055] As shown in Figure 3(b), when the target low-frequency vibration was 10 Hz, a very strong low-frequency sensation was presented, not only for the 10 Hz low-frequency vibration itself (raw wave), but also for single waveforms of 60 Hz, 80 Hz, and 100 Hz. In addition, in the case of 20 Hz, the low-frequency sensation was slightly weaker than that of 10 Hz, including the 20 Hz low-frequency vibration itself (raw wave).

[0056] For 30 Hz and 40 Hz, the low frequency vibrations (raw waves) themselves at 30 Hz and 40 Hz were often felt to be similar to high frequencies.

[0057] On the other hand, for all frequencies of 10 Hz, 20 Hz, 30 Hz, and 40 Hz, the median scale values ​​of the responses tended to be close to the values ​​given by the subjects for the 10 Hz, 20 Hz, 30 Hz, and 40 Hz low-frequency vibrations themselves and the values ​​given by the subjects for the substitute vibrations created at 60 Hz, 80 Hz, and 100 Hz, and it is thought that for all frequencies of 10 Hz, 20 Hz, 30 Hz, and 40 Hz, the subjects obtained a bodily sensation similar to the bodily sensation caused by the 10 Hz, 20 Hz, 30 Hz, and 40 Hz low-frequency vibrations themselves (Raw Waves) from the substitute vibrations created at 60 Hz, 80 Hz, and 100 Hz.

[0058] As described above, it can be seen that even with vibrations of 60 Hz, 80 Hz, or 100 Hz, by using the method of presenting a substitute vibration of the present invention, it is possible to present to the subject a sensation close to the target low-frequency vibration.

[0059] In the subject experiment in FIG. 3, a voice coil type vibrator capable of presenting vibrations of approximately 10 Hz to 100 Hz was used to strictly prevent subjects from selecting answers based on information other than vibration. However, it has been confirmed that even with a voice coil type vibrator that cannot present vibrations of 10 Hz to 50 Hz, by using the method of the present invention, it is possible to present to the subject a sensation similar to low-frequency vibrations that cannot be presented in the original manner.

[0060] A-4. Method for determining the amplitude and timing of substitute vibration from a target signal The method for creating substitute vibration from an ideal original signal (target signal) has been described above, but in reality, the original signal (target signal) reproduced in music, games, etc. is a complex vibration. Below, we will describe how to create the substitute vibration of the present invention from such complex vibration.

[0061] Figure 4(a) is a graph illustrating a first example of a process for calculating the amplitude in the calculation unit 102 and the timing for generating a single waveform by controlling the actuator 31 in the control unit 103, using an example of a signal such as vibration generated by music or the like, which has been acquired in advance by the acquisition unit 101 of the vibration generating system 100 shown in Figure 1 or which has been stored in advance in the storage device 13, etc. Figure 4(b) is a graph illustrating a second example.

[0062] In the first example shown in FIG. 4A, the vibration to be reproduced is shown as a low-frequency signal (original signal (target signal)) after separation into a high-frequency signal and a low-frequency signal, as described below. The low-frequency signal is a signal having a predetermined frequency or less, for example, 100 Hz or less, preferably 80 Hz or less, and more preferably 60 Hz or less. The frequency may be determined appropriately depending on the range of frequencies that are difficult to reproduce with the actuator 31 to be installed. The low-frequency signal may be processed as the entire low-frequency signal after separation, or may be further separated into predetermined low-frequency ranges (for example, when a low-frequency signal of 10 Hz or more but less than 80 Hz is acquired, it may be further separated into 10 Hz to 50 Hz and 50 Hz to 80 Hz, and the calculation unit 102 may process each low-frequency signal). In this case, various drive parameters of the actuator 31, such as the waveform, single-wave period, and amplitude of the single wave, may be controlled for each separated frequency band, so that a combination of multiple alternative vibrations that are closer to the human experience may be presented for each frequency band.

[0063] The calculation unit 102 calculates multiple maximum and minimum values ​​contained in the signal waveform of the low-frequency band. Experiments conducted by the inventors have shown that if a substitute vibration is generated at a very short time interval when the reverberation vibration has not yet sufficiently decayed, the subject will not perceive it as the target low-frequency vibration but will perceive it as the high-frequency vibration inherent to the actuator 31. Therefore, to generate a substitute vibration for the low-frequency vibration, adjacent maximum values ​​among the calculated maximum values ​​are selected that are separated by a predetermined time interval (hereinafter also referred to as the "interval time") (e.g., 0.1 seconds or more). If adjacent maximum values ​​are not separated by the predetermined time interval, only the most distinctive (largest) of the adjacent maximum values ​​is selected. In the illustrated example, the points marked with an "X" are not selected as maximum values ​​because they are not separated by the predetermined time interval. The position of the maximum value selected in this manner (hereinafter also referred to as the "maximum time") is calculated as the timing for presenting the substitute vibration.

[0064] On the other hand, with regard to the amplitude, in the first example, the distance from the maximum value to the intersection of a straight line connecting the multiple minimum values ​​and a perpendicular line is calculated as the amplitude of the alternative vibration (see the double arrow) (hereinafter, sometimes referred to as the "maximum value amplitude").

[0065] In this manner, the timing and amplitude for presenting the alternative vibration are determined.

[0066] 4B, the calculation unit 102 calculates only a plurality of maximum values ​​included in the waveform of the low frequency band. The selection of the maximum values ​​is the same as in the first example, and the position of the selected maximum value (maximum value time) is calculated as the timing time for presenting the alternative vibration.

[0067] On the other hand, in the second example, the maximum amplitude is calculated as the distance from a predetermined reference value to the maximum value (see the double arrow) as the amplitude of the alternative vibration.

[0068] The first example is effective when the background noise is large and the target signal waveform in the low frequency band is small compared to the noise, while the second example is effective when the target signal waveform in the low frequency band is sufficiently large compared to the noise.

[0069] The timing at which the single waveform is output does not necessarily have to be exactly the maximum value time, but may be generated by a time before or after the maximum value time, that is, by a predetermined time transition (hereinafter also referred to as "transition time") from the maximum value time. The predetermined time transition (transition time) may be set so that the timing at which the housing vibration (reverberation vibration) controlled by the single waveform reaches its maximum coincides with the maximum value time determined from the target signal, or is approximately the same as the maximum value time, so long as it does not cause discomfort to humans.

[0070] The transition time is determined based on the degree of deviation that humans can tolerate from the visual and auditory senses presented by the output unit 42. Existing research has shown that humans have a threshold at which they notice a deviation of about 0.04 seconds for repeated experiences and about 0.03 seconds for single experiences. Therefore, the transition time can be set within a range of about ±0.04 seconds, for example, but is preferably set within a range of about ±0.02 seconds to avoid causing discomfort to sensitive humans.

[0071] Furthermore, it is desirable that the timing of generating the single waveform maintains the interval between the calculated maximum values.

[0072] The reason why the timing does not need to be exactly the time of the maximum value is that the person experiencing the vibration will hardly notice if the timing of outputting the single waveform is off by a certain amount, and it is thought that reproducing the time between the main maximum values ​​of the target signal to be reproduced is important in presenting the user with a substitute vibration in the low-frequency range.

[0073] 4A and 4B, the maximum value and the time when the maximum value is reached are determined from the waveform, amplitude height, etc. However, the present invention is not limited to this. For example, the maximum value and the time when the maximum value is reached may be determined from the root mean square value, effective value, amount of change, etc. Any method may be used as long as it can determine the maximum value and the time when the maximum value is reached, which are characteristic points of the signal.

[0074] The amplitude of the substitute vibration does not need to be exactly the same as the maximum amplitude calculated in Figures 4(a) and (b), but may be determined by multiplying the calculated maximum amplitude by a predetermined ratio. The predetermined ratio may be adjusted according to the characteristics of the housing and actuator used so as to approximate the low-frequency sensation of the original signal (target signal). The calculated amplitude may be adjusted using an exponential function or the like so as to match the nonlinear subjective intensity perceived by humans. The predetermined ratio may be, for example, 80% or more and 120% or less of the calculated maximum amplitude.

[0075] The amplitude of the substitute vibration may be presented as a certain constant amplitude in advance, in cases where the maximum amplitude of the signal to be reproduced does not fluctuate much, etc. In this case, there is no need to calculate the maximum amplitude.

[0076] Next, a method for determining the amplitude and timing of low-frequency vibration in response to a target signal acquired online in real time by the input unit 41 will be described.

[0077] FIG. 5 is a graph illustrating the process of determining the amplitude and timing of a substitute vibration, in which information acquired in real time by the input unit 41 while online is acquired as a signal by the acquisition unit 101, the information is separated into high-frequency vibration and low-frequency vibration, and the target low-frequency vibration is extracted. The horizontal axis represents the passage of time.

[0078] The extracted original low-frequency vibration signal (target signal) is divided at fixed intervals, and the maximum value within each divided interval is calculated. The fixed divided intervals may be the same as the intervals required for processing the separated high-frequency vibration. The divided intervals may be 0.02 seconds or less, and may be 0.01 seconds, 0.005 seconds, or other intervals. Within the divided intervals, intervals where the maximum value changes from increasing to decreasing are detected, and the maximum value immediately before the change to decreasing is calculated as the maximum value time of the original signal. Time differentiation may be used to calculate the maximum value. After a maximum value (peak value) is detected, the next peak may be excluded according to a predetermined rule (e.g., not if it is 20% or more larger than the previous maximum value) for a predetermined time T (e.g., within 0.1 seconds) determined from the aforementioned interval time. Circles in FIG. 5 indicate intervals with maximum values ​​detected but not excluded within the predetermined time, and crosses indicate intervals with excluded maximum values.

[0079] Case #1 indicates a case where a next maximum is not detected within a predetermined time T. In this case #1, the detected maximum is determined as the output timing (maximum time) of the single wave, as usual. Case #2 indicates a case where a next maximum is detected within the predetermined time, but is not excluded because it is greater than the previous maximum by a predetermined value. The predetermined value is, for example, 20% or more greater than the previous maximum. In this case #2, both the first detected maximum and the next maximum that is 20% or more greater than the previous maximum are determined as the output timing (maximum time) of the single wave. However, the elapsed time of the predetermined time T may be given priority, and a next maximum that has not yet elapsed may be excluded from the output timing of the single wave. Case #3 indicates a case where a next maximum is detected within the predetermined time T, but is excluded because it is smaller than the previous maximum. In this case #3, only the first detected maximum value is determined as the output timing, and the next smaller maximum value is excluded from the output timing of the single wave. However, if the next maximum value (the first maximum value determined as the output timing in case #3) for which the predetermined time T has not yet elapsed in case #2 is excluded from the output timing of the single wave, prioritizing the elapsed time of the predetermined time T, the next smaller maximum value in case #3 may be determined as the output timing (maximum value time) of the single wave.

[0080] Whether to prioritize a point that is greater than the previous maximum value by a predetermined value or to prioritize the elapsed time of the predetermined time T is determined according to the situation of the signal to be reproduced and the situation to be presented.

[0081] The method for calculating the maximum value can be used to calculate the minimum value by finding the minimum value within the interval and then finding the interval where the signal changes from decreasing to increasing. The calculated maximum and minimum values ​​can be applied to calculate the amplitude of a short-term single waveform by applying the method shown in Figure 4(a). In the case of online processing, the output timing of the single waveform is as soon as possible after the amplitude of the single waveform is calculated using the method shown in Figures 4(a) and (b). Regarding the amplitude, the amplitude of the single waveform may be calculated in advance by buffering the signal for a predetermined time T or more.

[0082] When the original signal acquired online is reproduced after going online, the output timing and amplitude may be reset as a pre-acquired signal according to the techniques shown in Figures 4(a) and 4(b). On the other hand, in order to reproduce the online feel, the output timing and amplitude may be stored in the storage device 13 and reproduced using those output timing and amplitude.

[0083] 4(a), 4(b), and 5, the maximum value times and maximum value amplitudes calculated by the calculation unit 102 are linked to other visual and auditory information as time-series data for reproduction and stored in the storage device 13. By reading out the time-series data for reproduction stored in the storage device 13, the drive timing of the actuator 31 can be controlled by the control unit 103 at any time along with the visual and auditory information, providing the user with a highly realistic experience.

[0084] A-5. Processing of the Calculation Unit Next, the processing of the calculation unit 102 of the vibration generating system 100 will be described in detail.

[0085] FIG. 6 is a block diagram for explaining the process of generating a vibration waveform in a first frequency band, which is a low frequency region, and a vibration waveform in a second frequency band, which is a high frequency region, from an original signal such as a vibration generated by music, a movie, a game, etc. acquired by the acquisition unit 101 by the calculation unit 102, or an original signal pre-stored in the storage device 13, etc.

[0086] Step 201 is a step of separating signals below a predetermined frequency from a signal X(t) acquired by the acquisition unit 101 or previously stored in the storage device 13 or the like (in other words, the signal to be reproduced before conversion) into a high-frequency band signal H(t) and a low-frequency band signal L(t).

[0087] The high frequency band signal H(t) is filtered using a high frequency band pass filter such as a high pass filter to remove signals below a predetermined frequency. A well-known method is used for the separation and removal means. The separated high frequency band signal H(t) is then converted into a second vibration waveform S in a second vibration waveform generation step 202, which is a waveform in the high frequency band. ２ (t) is generated.

[0088] On the other hand, the low-frequency band signal is also filtered from the signal X(t) using a low-frequency band-pass filter such as a low-pass filter to extract a low-frequency band signal L(t) having a frequency equal to or lower than a predetermined frequency. Next, in calculation step 203 in the calculation unit 102, the amplitude (maximum amplitude) A of the single wave is calculated from the extracted low-frequency band signal L(t) using the method described with reference to FIGS. ｉ (i represents the number of the single wave to be generated) and the output time (maximum value time) t ｉ In the first vibration generation step 204, the amplitude A ｉ and output time t ｉ The first vibration waveform (alternative vibration) S, which is a waveform in the low frequency band, １ In this case, the first vibration generating step 204 determines the shape of the single-shot wave by referring to single-shot wave parameters for determining the single-shot waveform. The single-shot wave parameters may include values ​​such as the single-shot wave period time, transition time, interval time, or amplitude ratio of the single-shot wave.

[0089] A-6. Processing of the control unit Figure 7(a) is a block diagram explaining the vibration generation process when the actuator 31 is controlled by the control unit 103 of a general device, and Figure 7(b) is a block diagram explaining the vibration generation process when using a device that has a device-specific parameter conversion function, such as a mobile terminal such as a smartphone.

[0090] In the high frequency gain adjustment step 302 in FIG. 7A, the second vibration waveform S of the high frequency band signal generated in the second vibration waveform generation step 202 in FIG. ２ On the other hand, in the low-frequency gain adjustment step 303, the first vibration waveform (alternative vibration) S of the low-frequency band signal generated in the first vibration generation step 204 in FIG. １ (t) is subjected to low frequency gain adjustment using a low frequency band gain adjuster.

[0091] In the synthesis step 304, the first vibration waveform S １ (t) and the second vibration waveform S ２In the driving step 305, the control unit 103 of the vibration generating system 100 generates a signal based on the composite wave obtained in the combining step 304, and drives the actuator 31.

[0092] In FIG. 7B, when the vibration generating system 100 has a parameter conversion function specific to a device such as a mobile terminal, the high frequency band signal is converted into the second vibration waveform S of the high frequency band signal generated in the second vibration waveform generating step 202 of FIG. ２ The signal (t) is converted into a high-frequency parameter series based on the conversion defined by the operating system (OS) of the mobile terminal. The signal converted into a high-frequency parameter series in step 402 is converted into a time series and an amplitude series in step 403. Then, in step 404, the control unit 103 drives the actuator 31 to create and reproduce a haptic pattern.

[0093] On the other hand, for the low frequency band signal, the amplitude A of the single wave calculated by the calculation unit 102 in calculation step 203 of FIG. ｉ and output time (timing) t ｉ is converted into parameters specific to the mobile terminal in step 502. The signal converted into a parameter series is then converted into a time series and an amplitude series in step 503. Then, in step 504, the control unit 103 drives the actuator 31 to create and reproduce a haptic pattern.

[0094] [B] Modifications In the vibration presentation device 1 according to the above-described embodiment, low-band vibrations of about several tens of Hz, which are lower than the resonant frequency band that can be presented by the actuator, are converted into vibrations that are easily perceived by humans.

[0095] The present invention is carried out using an LRA with a resonance frequency of about 100 Hz to 300 Hz, which is generally installed in mobile terminals and the like, and can be used in combination with various technologies.

[0096] For example, the inventors of the present application have disclosed a technology in the cited Patent Document 2 that uses an energy control unit to maintain the energy of a high-frequency signal, thereby converting it to around 200 Hz while maintaining the high-frequency tactile sensation. This technology can be used in conjunction with the presentation method of the present application that uses a substitute vibration for low-frequency vibration, and even if an LRA with a narrow frequency band is used, by combining the present invention with the technology disclosed in Patent Document 2, it becomes possible to present a bodily sensation over a wide frequency band of around several tens of Hz to 400 Hz, thereby increasing the degree of freedom in device design and enabling miniaturization and cost reduction.

[0097] [C] Effects According to the vibration presentation device, the vibration presentation program, the computer-readable recording medium storing the vibration presentation program, and the vibration presentation method in the embodiments, for example, the following effects can be achieved.

[0098] The present invention can convert and present vibrations with frequencies lower than the resonant frequency band of the actuator 31 into vibrations that are easily perceived by humans as low frequencies by creating substitute vibrations using the actuator 31. In particular, in mobile devices such as smartphones that use an LRA as a vibrator, it has traditionally been difficult for the LRA to express low frequencies of about 10 Hz to 50 Hz, but by using the present invention, it is possible to present a low-frequency sensation.

[0099] This technology enables the expression of low-frequency vibrations in mobile devices and game controllers, enabling realistic vibration expression. It also enables the presentation of tactile vibrations from music, movies, videos, etc. on mobile devices.

[0100] The present invention acquires a signal including vibrations in a first frequency band, which are vibrations in the low frequency band, from the original signal that is the target to be reproduced, finds the maximum value of the acquired vibrations in the first frequency band, and calculates the maximum value time at calculation unit 102, which finds the time at which the maximum value occurs.Based on the calculated maximum value time, actuator 31 is driven to generate a single wave within a transition time that does not affect the human bodily sensation around the maximum value time.By presenting pseudo low frequency vibrations, even if actuator 31 is not capable of generating vibrations in the low frequency band, it is possible to present the user with a low frequency bodily sensation, and this does not require an additional actuator, leading to a smaller device.

[0101] Furthermore, by determining the maximum amplitude at the maximum time, it is possible to present low-frequency vibration with higher accuracy.

[0102] Furthermore, by determining the single wave cycle time and transition time for generating the single wave based on human bodily sensations, it is possible to reduce the sense of high frequency vibration.

[0103] When calculating the maximum value time from a signal having multiple maximum values, the calculation unit 102 sets the interval time, which is the time interval between adjacent maximum values, to 0.1 seconds or more, so that the physical sensation that the presented alternative waveform gives to a person does not give the sensation of high-frequency vibration.

[0104] When providing vibration online, if the intensity of a second maximum value immediately following a first maximum value among the multiple maximum values ​​is greater than the intensity of the first maximum value by a predetermined percentage or more, the calculation unit 102 may calculate the second maximum value even if the interval between the first maximum value and the second maximum value is equal to or less than the predetermined interval time. As a result, in real-time, such as online, by providing the user with an alternative waveform at a characteristic point in the original waveform, it is possible to reflect the bodily sensation of low frequencies in real time.

[0105] The control unit 103 generates a sine wave or pulse wave with an intensity between 80% and 120% of the intensity of the maximum amplitude, thereby appropriately controlling the intensity of the alternative waveform and enhancing the sense of realism.

[0106] The actuator of the present invention uses a housing that generates reverberant vibrations, and by utilizing reverberation, it is possible to present a more realistic low-frequency substitute vibration to the user.

[0107] [D] Others The disclosed technology is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the embodiments. The configurations and processes of the embodiments can be selected or combined as needed.

[0108] 1 includes one actuator 31, the number of actuators 31 is not limited to one, and the number of actuators 31 included in the vibration generating system 100 can be changed in various ways.

[0109] 1: Vibration presentation device 3: Vibration sensation device 11: CPU 12: Memory 13: Storage device 15: Bus 31: Actuator 32: Drive unit 33: Receiving unit 34: Housing 41: Input unit 42: Output unit 43: Transmitting unit 100: Vibration generating system 101: Acquisition unit 102: Calculation unit 103: Control unit

Claims

1. A vibration presentation device that presents vibration in a first frequency band by an actuator having a resonance frequency in a second frequency band larger than the first frequency band, an acquisition unit that acquires a signal including vibration of at least the first frequency band, the first frequency band being a resonant frequency lower than the second frequency band; a calculation unit that obtains a maximum value of the vibration of the first frequency band obtained by the obtaining unit, and obtains a maximum value time, which is a time at which the maximum value occurs, from the maximum value; a control unit that controls the actuator to generate a single wave within a transition time that is a time before and after the maximum value time based on the maximum value time calculated by the calculation unit; A vibration presentation device comprising:

2. The calculation unit calculates a maximum value amplitude, which is an amplitude of the maximum value at the maximum value time. The vibration presentation device according to claim 1 .

3. The single wave period time for generating the single wave is 0.002 seconds or more and 0.02 seconds or less. The vibration presentation device according to claim 1 .

4. The transition time is 0.04 seconds or less. The vibration presentation device according to claim 1 .

5. the calculation unit calculates a plurality of the maximum values, and calculates the maximum value times for the maximum values ​​whose time intervals between adjacent maximum values ​​are 0.1 seconds or more. The vibration presentation device according to claim 1 .

6. the calculation unit calculates a plurality of the maximum values, and when an intensity of a second maximum value immediately following a first maximum value among the plurality of maximum values ​​is greater than an intensity of the first maximum value by a predetermined ratio or more, calculates the maximum value time even if an interval time between the first maximum value and the second maximum value is equal to or less than the interval time. The vibration presentation device according to claim 5 .

7. The control unit generates the single wave having an amplitude of 80% or more and 120% or less of the maximum amplitude. The vibration presentation device according to claim 2 .

8. The first frequency band is equal to or greater than 10 Hz and equal to or less than 100 Hz. The vibration presentation device according to claim 1 .

9. The calculation unit calculates the maximum value of vibration based on a waveform of vibration or an amplitude of vibration. The vibration presentation device according to claim 1 .

10. The vibration presentation device according to claim 1 , a vibration sensation device including the actuator having a resonant frequency in the second frequency band, and a housing in which reverberation vibration is generated by controlling the actuator, Vibration generation system.

11. The transition time is set to a time including the maximum value time when the reverberation vibration of the housing is maximized. The vibration generating system according to claim 10.

12. On the computer, acquiring a signal including vibrations in a first frequency band, the first frequency band being a lower resonant frequency than a second frequency band; a calculation step of determining a maximum value of the vibration of the first frequency band obtained by the obtaining step, and determining a maximum value time, which is a time at which the maximum value occurs, from the maximum value; a driving step of controlling an actuator having a resonant frequency in the second frequency band within a transition time, which is the time before and after the maximum value time, based on the maximum value time calculated by the calculation step, to generate a single wave.

13. A recording medium storing the vibration presentation program according to claim 12.

14. A vibration presentation method for presenting a vibration in a first frequency band by an actuator having a resonance frequency in a second frequency band larger than the first frequency band, comprising: acquiring a signal including vibrations in at least the first frequency band, the first frequency band being a resonant frequency lower than the second frequency band; a calculation step of determining a maximum value of the vibration of the first frequency band obtained by the obtaining step, and determining a maximum value time, which is a time at which the maximum value occurs, from the maximum value; a driving step of controlling an actuator having a resonant frequency in the second frequency band within a transition time that is a time before and after the maximum value time based on the maximum value time calculated in the calculating step to generate a single wave; A vibration generating method comprising: