CONTROL DEVICE, SPEAKER DEVICE, AND AUDIO OUTPUT METHOD

The control device and speaker system address the issue of noise interference from tactile presentation by generating an opposite-phase audio signal to cancel out noise from the haptic unit, ensuring a clear musical experience while providing tactile sensations.

JP7593322B2Active Publication Date: 2024-12-03SONY GROUP CORP
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
JP2021543707
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-09-03
Filing Date
2020-08-25
Publication Date
2024-12-03
Estimated Expiration
2040-08-25

AI Technical Summary

Technical Problem

Headphones and wearable speakers that provide tactile presentation can cause vibrations or noise from the vibrator to reach the user's ears, interfering with music listening, especially when higher frequency components are used to express various tactile sensations.

Method used

A control device and speaker system that includes a haptic control section to generate tactile signals and an audio control section to produce an audio signal in opposite phase to cancel out noise from the tactile presentation unit, using a cancellation signal generation section to create a second audio signal that counteracts the noise generated by the haptic presentation.

Benefits of technology

Effectively reduces noise interference from tactile presentation devices by canceling out the noise generated by the haptic unit, allowing users to experience tactile sensations without disturbing their musical experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593322000001
    Figure 0007593322000001
  • Figure 0007593322000002
    Figure 0007593322000002
  • Figure 0007593322000003
    Figure 0007593322000003
Patent Text Reader

Abstract

A control device according to one embodiment of the present technology is equipped with a haptic control part and a sound control part. On the basis of a haptic signal for haptic presentation, the haptic control part generates a haptic control signal that drives a haptic presentation unit. On the basis of a first sound signal and a second sound signal, which is generated on the basis of the haptic signal and includes a sound component in anti-phase to a sound produced by the haptic presentation unit, the sound control part generates a sound control signal that drives a sound output unit.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present technology relates to a control device for controlling an audio output device having a haptic function, a speaker device, and an audio output method. [Background technology]

[0002] In recent years, applications that stimulate the sense of touch through human skin using tactile reproduction devices have been used in a variety of situations. Currently, tactile reproduction devices in wide use include eccentric rotating masses (ERMs) and linear resonant actuators (LRAs), and these devices have a resonant frequency that is sensitive to human tactile sensation (around a few hundred Hz) (see, for example, Patent Document 1).

[0003] Since the frequency band in which humans have the highest tactile sensitivity is several hundred Hz, the majority of vibration reproduction devices target this frequency band. Other tactile reproduction devices proposed include electrostatic displays and surface acoustic wave displays that aim to realize the desired tactile sensation by controlling the friction coefficient of the touching part (see, for example, Patent Document 2). Other proposed devices include an airborne ultrasound tactile display that uses the acoustic radiation pressure of focused ultrasound, and an electrotactile display that electrically stimulates nerves and muscles connected to tactile receptors.

[0004] One application of these devices, particularly in music listening, is to incorporate a vibration reproduction device into the headphone housing and reproduce vibrations at the same time as playing music, emphasizing deep bass. Wearable speakers have also been proposed that are not headphones but are worn around the neck. Some of these devices use the contact with the user's body to transmit vibrations to the user from the back along with the sound output from the speaker (see, for example, Patent Document 3), while others transmit vibrations to the user by utilizing the resonance of the back pressure of the speaker vibration (see, for example, Patent Document 4). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-202486 A [Patent Document 2] JP 2001-255993 A [Patent Document 3] Japanese Patent Application Publication No. 10-200977 [Patent Document 4] Patent Application No. 2017-43602 Summary of the Invention [Problem to be solved by the invention]

[0006] In headphones and wearable (neck) speakers that provide tactile sensations, the vibration of the vibrator provided for tactile sensations or the noise caused by the driving of other tactile sensation presentation devices can reach the ears of the user and interfere with listening to music. Although the structure of a wearable speaker can prevent noise from reaching the ears to some extent, if the vibration contains higher frequency components to express a variety of tactile sensations, noise of a frequency to which the human ear is more sensitive will be generated.

[0007] In view of the above circumstances, an object of the present invention is to provide a control device, a speaker device, and an audio output method that can reduce noise from a tactile presentation device. [Means for solving the problem]

[0008] A control device according to an embodiment of the present technology includes a haptic control unit and a sound control unit. The haptic control unit generates a haptic control signal for driving a haptic presentation unit based on a haptic signal for haptic presentation. The audio control unit generates an audio control signal for driving an audio output unit based on a first audio signal and a second audio signal generated based on the tactile signal and including an audio component that is in phase opposite to the audio generated from the tactile presentation unit.

[0009] The audio control unit may generate the audio control signal by adding the second audio signal to the first audio signal.

[0010] The device may further include a cancellation signal generating unit that generates the second audio signal based on the haptic signal.

[0011] The audio control unit may generate a first signal that drives a first audio output unit that reproduces the first audio signal, and the cancellation signal generation unit may generate a second signal that drives a second audio output unit that reproduces the second audio signal.

[0012] The cancellation signal generation unit may generate the second audio signal based on the frequency characteristics of the audio output unit, the frequency characteristics of the tactile presentation unit, and a conversion function from vibration generated by the tactile presentation unit to sound.

[0013] A speaker device according to an embodiment of the present technology includes a haptic presentation unit, an audio output unit, a haptic control unit, and an audio control unit. The haptic control unit generates a haptic control signal for driving the haptic presentation unit based on a haptic signal for haptic presentation. The audio control unit generates an audio control signal for driving the audio output unit based on a first audio signal and a second audio signal generated based on the tactile signal and including an audio component that is in phase opposite to the audio generated from the tactile presentation unit.

[0014] The device may further include a storage unit that stores the haptic signal and the first audio signal.

[0015] The device may further include a communication unit capable of communicating with a server that stores the haptic signal and the first audio signal.

[0016] The audio output unit may include a common speaker unit for reproducing the first audio signal and the second audio signal.

[0017] The audio output unit may comprise a first speaker unit for reproducing the first audio signal and a second speaker unit for reproducing the second audio signal.

[0018] The audio output unit includes a right speaker and a left speaker; The speaker device may further include a neck-hanging connector that connects the right speaker and the left speaker.

[0019] The haptic presentation unit may include a vibration device.

[0020] Alternatively, the tactile presentation unit may include an ultrasound generator, an electrical muscle stimulation device, or an air cannon generator.

[0021] An audio output method according to one embodiment of the present technology includes acquiring a tactile signal for tactile presentation and a first audio signal; generating a tactile control signal for driving a tactile presentation unit based on the tactile signal; generating a second audio signal including an audio component in antiphase with audio generated from the tactile presentation unit based on the tactile signal; and generating an audio control signal for driving an audio output unit based on the first audio signal and the second audio signal. [Brief description of the drawings]

[0022] [Figure 1] 1 is a perspective view of a speaker device according to a first embodiment of the present technology; [Diagram 2]FIG. 2 is a perspective view showing a state in which the speaker device is mounted on a user. [Diagram 3] 2 is a schematic cross-sectional view of a main part of the speaker device. FIG. [Figure 4] FIG. 2 is a block diagram showing a configuration example of the speaker device. [Diagram 5] It is the flow of audio and vibration signals that is transmitted as sound and vibration to the user's ears and skin. [Figure 6] 4 is an explanatory diagram of the operation of the speaker device. [Figure 7] FIG. 1 is an image illustrating the basic principle of noise cancellation. [Figure 8] 11 is a flow diagram showing a processing procedure for generating a cancellation signal from a vibration signal for providing a tactile sensation. [Figure 9] FIG. 11 is a block diagram showing a speaker device according to a second embodiment of the present technology. [Figure 10] 11A and 11B are diagrams illustrating a state in which noise is generated from a vibration signal for tactile presentation. [Figure 11] FIG. 13 is a block diagram showing a speaker device according to a third embodiment of the present technology. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0024] <First embodiment> (Basic configuration of speaker device) Fig. 1 is a perspective view showing a configuration example of a speaker device according to an embodiment of the present technology. This speaker device (audio output device) 100 has a function of actively presenting a tactile sensation (such as vibration) to a user U at the same time as sound. As shown in Fig. 2, the speaker device 100 is a wearable speaker placed on both shoulders of the user U, for example.

[0025] The speaker device 100 includes a right speaker 100R, a left speaker 100L, and a connector 100C that connects the right speaker 100R and the left speaker 100L. The connector 100C is formed in an arbitrary shape that allows it to be hung around the neck of a user U, and the right speaker 100R and the left speaker 100L are positioned on both shoulders or above the chest of the user U.

[0026] Fig. 3 is a schematic cross-sectional view of the main parts of the right speaker 100R and the left speaker 100L of the speaker device 100 in Fig. 1 and Fig. 2. The right speaker 100R and the left speaker 100L typically have a symmetrical structure on the left and right. Note that Fig. 3 is merely a schematic diagram and does not necessarily correspond to the shapes and dimensional ratios of the speakers shown in Fig. 1 and Fig. 2.

[0027] The right speaker 100R and the left speaker 100L each include, for example, an audio output unit 250, a tactile presentation unit 251, and a housing 254 that houses them. The right speaker 100R and the left speaker 100L typically reproduce audio signals in a stereo system. The reproduced sound is not particularly limited as long as it is a reproducible sound or audio, such as music, conversation, or sound effects.

[0028] The sound output unit 250 is an electroacoustic conversion type dynamic speaker. The sound output unit 250 includes a diaphragm 250a, a voice coil 250b wound around the center of the diaphragm 250a, a fixing ring 250c that holds the diaphragm 250a in a housing 254, and a magnet assembly 250d that is arranged facing the diaphragm 250a. The voice coil 250b is arranged perpendicular to the direction of the magnetic flux generated in the magnet assembly 250d. When an audio signal (AC current) is supplied to the voice coil 250b, the diaphragm 250a vibrates due to the electromagnetic force acting on the voice coil 250b. The diaphragm 250a vibrates in accordance with the signal waveform of the audio signal, generating a reproduced sound wave.

[0029] The tactile presentation unit 251 includes a vibration device (vibrator) capable of generating tactile vibrations, such as an eccentric motor (ERM), a linear actuator (LRA), or a piezoelectric element. The tactile presentation unit 251 is driven by inputting a tactile signal for tactile presentation prepared separately from a playback signal. The amplitude and frequency of the vibration are not particularly limited. The tactile presentation unit 251 is not limited to being composed of a single vibration device, and may be composed of multiple vibration devices. In this case, the multiple vibration devices may be driven simultaneously or separately.

[0030] Housing 254 has an opening (sound guide port) 254a for passing audio output (reproduced sound) to the outside on a surface facing diaphragm 250a of sound output unit 250. Opening 254a is formed linearly along the longitudinal direction of housing 254 as shown in Fig. 1, but is not limited thereto and may be formed of a plurality of through holes or the like.

[0031] The tactile presentation unit 251 is disposed, for example, on the inner surface opposite to the opening 254a of the housing 254. The tactile presentation unit 251 presents tactile vibration to the user via the housing 254. In order to improve the transferability of the tactile vibration, a part of the housing 254 may be made of a material with a relatively low rigidity. The shape of the housing 254 is not limited to the shape shown in the figure, and any appropriate shape such as a disk shape or a rectangular parallelepiped shape can be adopted.

[0032] Next, a description will be given of a control system of the speaker device 100. Fig. 4 is a block diagram showing an example of the configuration of a speaker device applied to this embodiment.

[0033] The speaker device 100 includes a control device 1 that controls driving of an audio output unit 250 and a tactile presentation unit 251 of the right speaker 100R and the left speaker 100L. The control device 1 and other elements described later are built into a housing 254 of the right speaker 100R or the left speaker 100L. Alternatively, the control device 1 may be configured as an external device connected to the right speaker 100R and the left speaker 100L by wire or wirelessly.

[0034] As shown in FIG. 3, the control device 1 includes a voice control unit 13, a haptic control unit 14, and a cancellation signal generation unit 17. The control device 1 can be realized by hardware elements used in a computer, such as a CPU (Central Processing Unit), RAM (Random Access Memory), and ROM (Read Only Memory), as well as necessary software. Instead of or in addition to the CPU, a PLD (Programmable Logic Device) such as an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or an ASIC (Application Specific Integrated Circuit), etc. may be used. The control device 1 executes a predetermined program to configure a voice control unit 13, a haptic control unit 14, and a cancellation signal generation unit 17 as functional blocks.

[0035] The speaker device 100 includes a storage (storage unit) 11, a decoding unit 12, an audio output unit 15, a haptic output unit 16, and a communication unit 18 as other hardware components.

[0036] Based on the haptic signal for haptic presentation, the haptic control unit 14 generates a haptic control signal for driving the haptic output unit 16. The haptic signal is data for haptic presentation (haptic data) stored in the storage 11 or an external server device 50 such as a cloud server, as will be described later. The haptic output unit 16 includes the haptic presentation unit 251 shown in FIG.

[0037] Based on the first audio signal and the second audio signal, the audio control unit 13 generates an audio control signal for driving the audio output unit 15. In this embodiment, the audio control unit 13 generates the audio control signal by adding the second audio signal to the first audio signal.

[0038] Here, the first audio signal is a music piece or other audio signal. The second audio signal is an audio signal generated based on a haptic signal and includes an audio component in the opposite phase to the audio generated from the haptic output unit 16. The first audio signal is data for audio playback (audio data) stored in the storage 11 or the server device 50. The second audio signal corresponds to a cancellation signal for canceling the audio generated when the haptic output unit 16 is driven, and is generated by the cancellation signal generation unit 17.

[0039] The cancellation signal generating unit 17 is configured to be capable of generating the second audio signal based on the haptic signal. The haptic signal is data (vibration data) for haptic presentation stored in the storage 11 or the server device 50. In this embodiment, the second audio signal is configured to be generated in the cancellation signal generating unit 17, but is not limited to this, and the second audio signal may be stored in the storage 11 or the server device 50 together with the first audio signal and the haptic signal.

[0040] The storage 11 is a storage device such as a non-volatile semiconductor memory capable of storing the first audio signal and the haptic signal. In this embodiment, the first audio signal and the haptic signal are stored in the storage 11 as appropriately encoded digital data.

[0041] The decoding unit 12 decodes the first audio signal and the haptic signal stored in the storage 11. The decoding unit 12 may be configured as a functional block that is part of the control device 1.

[0042] The communication unit 18 is configured with a communication module that can be connected to the network 10 in a wired or wireless manner. The communication unit 18 is configured to be able to communicate with the server device 50 via the network 10 and to be able to acquire the first audio signal and the haptic signal stored in the server device 50.

[0043] The audio output section 15 includes, for example, an audio output unit 250 for the right speaker 100R and the left speaker 100L shown in FIG. The haptic output section 16 includes, for example, the haptic presentation unit 251 shown in FIG.

[0044] (Speaker device operation) Next, a typical operation of the speaker device 100 configured as above will be described.

[0045] The control device 1 receives from the server device 50 or reads from the storage 11, and thereby obtains digital signals for outputting sound and a touch (a first sound signal and a touch signal).

[0046] Next, the decoding unit 12 performs an appropriate decoding process on the acquired data to extract audio data (first audio signal) and haptic data (haptic signal), and inputs them to the audio control unit 13 and the haptic control unit 14, respectively.

[0047] The audio control unit 13 and the haptic control unit 14 perform various processes on the input data. The output (audio control signal) of the audio control unit 13 is input to the audio output unit 15, and the output (haptic control signal) of the haptic control unit 14 is input to the haptic output unit 16. The audio output unit 15 and the haptic output unit 16 each include a D / A converter, a signal amplifier, and a playback device (corresponding to the audio output unit 250 and the haptic presentation unit 251). The D / A converter and the signal amplifier may be included in the audio control unit 13 and the haptic control unit 14. The signal amplifier may include a volume adjustment unit adjusted by the user U.

[0048] The haptic control unit 14 transmits the haptic data to the cancellation signal generation unit 17. The cancellation signal generation unit 17 generates a cancellation signal (second audio signal) that cancels the noise output from the haptic output unit 16, and transmits it to the audio control unit 13. The cancellation signal is generated based on the haptic data and contains an audio component that is in the opposite phase to the audio (noise) generated from the haptic output unit 16. A method for generating the cancellation signal will be described in detail later.

[0049] Based on the input haptic data, the haptic control unit 14 generates a haptic control signal for driving the haptic output unit 16. Based on the input audio data and the cancellation signal, the audio control unit 13 generates an audio control signal for driving the audio output unit 15. As a result, a reproduced sound including an audio component for canceling out the sound (noise) generated from the haptic output unit 16 is output from the audio output unit 15.

[0050] (Principle of noise cancellation) Next, a description will be given of the cancellation signal generated by the cancellation signal generating unit 17. Here, a case will be described in which content including an audio signal and a vibration signal for presenting a tactile sensation is reproduced by a vibration speaker equipped with an actively drivable vibrator for presenting a tactile sensation, and music with vibration is listened to.

[0051] The vibrations transmitted from the vibration speaker to the user's body (skin) are output from the vibrator driven by the vibration signal. On the other hand, the sound transmitted from the vibration speaker to the user's ear is the sum of not only the sound from the speaker driven by the audio signal, but also the vibration of the vibrator driven by the vibration signal that has been converted into sound. The components converted from vibration to sound that are added to the sound from the speaker often disrupt the user's music experience as noise.

[0052] 5 is an explanatory diagram of the operation of a speaker device without a noise cancellation function, showing the flow in which an audio signal and a vibration signal are converted into sound and vibration and transmitted to the user's ears and skin. As shown in FIG. 5, for example, the sound Y output from the speaker unit 20 is a signal obtained by multiplying the Fourier transform F1 of the audio signal f1 by the frequency characteristic H1 of the speaker unit 20 (hereinafter, unless otherwise specified, the signal will be described as having been Fourier transformed).

[0053] On the other hand, the vibration Z output from the vibrator 21 for the Fourier transform F2 of the vibration signal f2 is a signal multiplied by the frequency characteristic H2 of the vibrator 21, and this is transmitted to the skin as the vibration Z. Furthermore, the vibrator 21 generates a sound that has been converted from vibration to sound (conversion function H3) and is transmitted to the user's ear. Therefore, the sound Y that reaches the ears of user U is Y = H1F1+ H3H2F2 It becomes.

[0054] The frequency characteristic H1 of the speaker unit 20 is, for example, the frequency characteristic of the speaker unit 20 itself, or this frequency characteristic multiplied by the gain of a signal amplifier provided in the preceding stage of the speaker unit 20, or the like. The frequency characteristic H2 of the vibrator 21 is defined in the same manner as the frequency characteristic H1 of the speaker unit 20. A conversion function H3 from vibration to sound is defined as a function (characteristic) based on the mechanical structure and arrangement of the vibrator 21, for example.

[0055] On the other hand, Fig. 6 is an explanatory diagram of the operation of the speaker device 100 of this embodiment having a noise cancellation function. As shown in Fig. 6, the vibration signal F2 output from the haptic control unit 14 is transmitted to the cancellation signal generating unit 17, which generates a cancellation signal F3 and transmits it to the audio control unit 13. The cancellation signal F3 is generated using the above-mentioned characteristics H1, H2, and H3. Here, the characteristics H1, H2, and H3 may be known values ​​if they are known characteristics, or may be values ​​specified by prior measurement for each speaker device 100 as a pre-shipment test.

[0056] The voice control unit 13, which has received the cancellation signal F3, adds the cancellation signal F3 to the voice signal F1 and transmits it to the speaker unit 20 (voice output unit 15), which outputs the result as voice. The cancellation signal F3 is generated so as to cancel only the sound generated from the vibrator 21 among the sounds that reach the ears of the user U. As a result, the user U can sense the vibration Z output from the vibrator 21 (tactile output unit 16) as it is, and hear the voice from which the noise generated from the vibrator 21 has been cancelled.

[0057] The basic principle of noise cancellation using a cancellation signal will be described with reference to Fig. 7. For noise 40 that is undesirable for a user U to perceive, a cancellation signal 41 is generated so that a signal 42 obtained by adding the noise 40 and the cancellation signal 41 is minimized. A typical cancellation signal 41 of this type is a signal that generates a sound with the same amplitude as the waveform of the noise 40 but in the opposite phase. The cancellation signal 41 is set, for example, in dB units.

[0058] (Example of noise cancellation) The process for canceling audio noise will now be described in detail. It is desirable that the cancellation signal F3, when output from the speaker unit 20, be a sound that is in the opposite phase to the audio noise from the vibrator 21. Therefore, to generate the cancellation signal F3 from the vibration signal F2, the sound (H3H2F2) generated from the vibrator 21 is estimated taking into account the frequency characteristic H2 of the vibrator 21 itself and the conversion function H3 from the vibration of the vibrator 21 to sound, and the frequency characteristic H1 of the speaker unit 20 is further discounted.

[0059] The cancellation signal F3 thus generated can be expressed by a formula as G1H3H2F2, where G1 is a frequency characteristic having an inverse characteristic that can discount the frequency characteristic H1 of the speaker unit 20. The calculation of the cancellation signal F3 requires information on the characteristics H1, H2, and H3, and therefore these must be measured in advance. For this measurement, a known method such as a method for identifying characteristics of an acoustic device is used.

[0060] Moreover, assuming that the speaker device 100 is worn by a user U, the vibration-to-sound conversion function H3 is desirably measured in a state in which the speaker device 100 is worn by a mannequin, a test subject, or the like. The calculation process of the cancellation signal F3 can be realized, for example, by multiplying the frequency characteristics G1, H3, H2, and F2 that have been Fourier transformed. Alternatively, it can be realized by performing an inverse Fourier transform of the vibration signal F2 and the frequency characteristics G1, H3, and H2, that is, by convolution of the impulse responses g1, h3, and h2 of each system. It can also be realized by implementing a filter having a characteristic (-G1H3H2) in the cancellation signal generation unit 17 as hardware.

[0061] Although there is also known a technology (ANC: Active Noise Control) that uses a vibration speaker equipped with a microphone to sense the sound from a vibrator with the microphone and uses the sensing signal as a cancellation signal, in this embodiment, a highly accurate noise cancellation function can be realized without using such a microphone, by feedforward control that uses the vibration signal that is input to the haptic output unit 16. This is because the speaker device 100 can be designed structurally to contain the noise from the haptic output unit 16 so that it is difficult for the noise to reach the ears of the user U, and because the noise to be canceled is audio noise from the haptic output unit 16 that is measured in advance, unlike random environmental noise, and therefore can be predicted to a certain extent.

[0062] Therefore, according to the speaker device 100 of the present embodiment, since a microphone for noise sensing is not required, the device configuration can be simplified or made smaller. Also, by using the vibration signal before output that is buffered from the network 10 or the storage 11, it is possible to cancel noise without any delay as a feedforward control.

[0063] In the speaker device 100 of this embodiment, the haptic output unit 16 is configured as a separate unit from the audio output unit 15, which increases the degree of freedom in designing the haptic output unit 16 and allows a variety of haptic vibrations to be presented to the user. Moreover, the cancellation signal F3 is reproduced in synchronization with the driving of the haptic output unit 16, which effectively removes noise that accompanies the driving of the haptic output unit 16. Furthermore, it is also possible to reproduce content that presents a haptic sensation at a timing different from the output audio from the audio output unit 15 or in a wider band than the audio.

[0064] In the speaker device 100, a single limiter or multiple limiters such as weak, medium, and strong limiters for setting a maximum value of the cancellation signal may be provided in the audio output unit 15 in dB units for sensitivity adjustment. This allows the user U to select the noise canceling level himself / herself.

[0065] Fig. 8 shows a process flow of the cancellation signal generator 17 and the audio controller 13, which generate a cancellation signal from the vibration signal F2 using the known frequency characteristic H2 of the vibrator 21, the vibration-to-sound conversion function H3, and the frequency characteristic H1 of the speaker unit 20. Here, as an example, the cancellation signal is generated by digital signal processing using a filter having a frequency characteristic (-G1H3H2).

[0066] 8, first, in step S101, the cancellation signal generating unit 17 obtains the vibration signal f2 to be reproduced from the haptic control unit 14. Next, in step S102, the cancellation signal generating unit 17 applies a digital filter having a frequency characteristic of a coefficient (-G1H3H2) measured and recorded in advance to the vibration signal f2. As a result, a cancellation signal f3 (second audio signal) is generated.

[0067] The audio control unit 13 generates an audio control signal for driving the audio output unit 15 by adding this cancellation signal f3 to the audio signal f1 (first audio signal). At this time, the amplification factors applied to each of the audio signal f1 and the vibration signal f2 during playback may be taken into consideration. For example, when the audio signal f1 is amplified by a1 times and the vibration signal f2 is amplified by a2 times and output from the speaker unit 20 and the vibrator 21, respectively, the cancellation signal generation unit 17 receives information on these magnifications from the audio control unit 13 and the haptic control unit 14, and multiplies the cancellation signal f3 by these magnification ratios (a2 / a1) in step S103.

[0068] Then, in step S104, the audio control unit 13 generates the audio control signal by adding the cancellation signal (f3×(a2 / a1)) multiplied by the magnification ratio to the sound signal f1. As a result, the sound Y that is not perceived as noise by the user U is output from the audio output unit 15.

[0069] In the above explanation, the vibrator 21 has been used as an example of the tactile presentation device (tactile presentation unit 251) in the tactile output unit 16, but the present invention is equally applicable to other tactile presentation devices other than vibrators (e.g., an airborne ultrasonic generator, an electrical muscle stimulation device, an air cannon generator, etc.).

[0070] <Second embodiment> Next, a second embodiment of the present technology will be described.

[0071] In the above-described first embodiment, the audio output section 15 (audio output unit 250) is configured so that the audio signal (first audio signal) and the cancellation signal (second audio signal) are reproduced by a common speaker unit (speaker section 20). In contrast, in this embodiment, the audio signal (first audio signal) and the cancellation signal (second audio signal) are reproduced by separate speakers.

[0072] 9 is an explanatory diagram of the operation of a speaker device 200 in the second embodiment of the present technology. The speaker device 200 of this embodiment further includes a speaker section 22 (second audio output unit, second speaker) that reproduces a cancellation signal F3 as a second signal, in addition to a speaker section 20 (first audio output unit, first speaker) that reproduces an audio signal F1 as a first signal.

[0073] In this embodiment, the cancellation signal generating unit 27 outputs a control signal to the speaker unit 22 to reproduce the cancellation signal F3 from the speaker unit 22. The cancellation signal F3 is expressed by the formula (-G4H3H2F2) where G4 is a main wave number characteristic having an inverse characteristic that can discount the frequency characteristic H4 of the speaker unit 22. As a result, a composite sound of the reproduced sound of the audio signal F1 output from the speaker unit 20 and the reproduced sound of the cancellation signal F3 output from the speaker unit 22 is presented to the user U, and as a result, the sound from the vibrator 21 can be cancelled.

[0074] This embodiment can also provide the same advantageous effects as those of the above-described first embodiment. In particular, according to this embodiment, the speaker unit 22 that reproduces the cancellation signal F3 can be configured with specifications different from those of the speaker unit 20 that reproduces the audio signal F1, so that the degree of freedom in designing each of the speaker units 20 and 22 can be increased.

[0075] <Third embodiment> In each of the above-described embodiments, the case where the tactile output unit 16 is a vibration device (vibrator 21) has been described as an example, but in this embodiment, a case where the tactile output unit 16 is a tactile presentation device other than a vibration device will be described.

[0076] Examples of tactile presentation devices other than vibration devices include those that shoot an airflow toward the user U by driving a diaphragm (air cannon generator), those that apply acoustic radiation pressure with an ultrasonic array focused on the body of the user U (ultrasound generator), and those that directly stimulate the tactile receptors and muscles of the user U with electrical stimulation (electrical muscle stimulation device). In other words, the tactile presentation device may be something other than one that is intended to directly vibrate the skin with vibration.

[0077] In these tactile presentation devices, noise is often generated when they are driven, which may interfere with listening to music. Therefore, the noise is canceled by generating a cancellation signal using the characteristics of each device and outputting it from a speaker. As shown in FIG. 10, the conversion function from the drive of the tactile presentation device to sound may be, for example, a function H5 having a specific frequency characteristic like a vibration device, or a nonlinear function H6 that always outputs a signal with a constant amplitude and frequency during the driving section (period). When the conversion function is H5, audio noise 70 of the form shown in the upper part of FIG. 10 is generated from the vibration signal f2, and when the conversion coefficient is H6, audio noise 71 shown in the lower part of FIG. 10 is generated from the signal f2. Although the behavior of the audio noise may differ greatly, the flow and principle of the noise cancellation process are the same as those in FIG. 6 and FIG. 7.

[0078] 11 is an explanatory diagram of the operation of the speaker device 300 in this embodiment. In the figure, the same reference numerals are given to the parts corresponding to those in the first embodiment (FIG. 6), and detailed description thereof will be omitted.

[0079] 11, the tactile presentation device 80 includes a tactile presentation device other than a vibration device. Here, assuming that the conversion function from drive of the tactile presentation device 80 to sound is H6 and the frequency characteristic of the tactile presentation device is H7, the cancellation signal generation unit 17 generates a cancellation signal F3 having characteristics expressed by the formula (-G1H6H7F2). As a result, a sound Y that is not perceived as noise by the user U is output from the speaker unit 20 (audio output unit 15).

[0080] <Modification> For example, in each of the above embodiments, a wearable speaker device that is hung around the user's neck has been described as an example of a speaker device, but the present technology is not limited to this and can also be applied to headphones or earphones that are worn on the user's head. In this case, the haptic presentation unit may be configured as a device separate from the speaker device. In this case, the haptic presentation unit can be placed at any position, so that a haptic sensation can be presented to a desired position on the user's body.

[0081] The present technology can also be configured as follows. (1) a haptic control unit that generates a haptic control signal for driving a haptic presentation unit based on a haptic signal for haptic presentation; a sound control unit that generates a sound control signal for driving a sound output unit based on a first sound signal and a second sound signal that is generated based on the haptic signal and includes a sound component in antiphase with a sound generated from the haptic presentation unit; A control device comprising: (2) The control device according to (1), The audio control unit generates the audio control signal by adding the second audio signal to the first audio signal. Control device. (3) The control device according to (1) or (2) above, The apparatus further includes a cancellation signal generating unit that generates the second audio signal based on the haptic signal. Control device. (4) The control device according to (3) above, the audio control unit generates a first signal to drive a first audio output unit to reproduce the first audio signal; The cancellation signal generator generates a second signal for driving a second audio output unit that reproduces the second audio signal. Control device. (5) The control device according to (3) or (4) above, The cancellation signal generation unit generates the second audio signal based on a frequency characteristic of the audio output unit, a frequency characteristic of the haptic presentation unit, and a conversion function from vibration generated by the haptic presentation unit to sound. Control device. (6) a tactile presentation unit; An audio output unit; a haptic control unit that generates a haptic control signal for driving the haptic presentation unit based on a haptic signal for haptic presentation; a sound control unit that generates a sound control signal for driving the sound output unit based on a first sound signal and a second sound signal that is generated based on the haptic signal and includes a sound component in antiphase with a sound generated from the haptic presentation unit; A speaker device comprising: (7) The speaker device according to (6), a storage unit for storing the haptic signal and the first audio signal; Further comprising Speaker device. (8) The speaker device according to (6) or (7) above, The apparatus further includes a communication unit capable of communicating with a server that stores the haptic signal and the first audio signal. Speaker device. (9) The speaker device according to any one of (6) to (8), The audio output unit includes a common speaker unit for reproducing the first audio signal and the second audio signal. Speaker device. (10) The speaker device according to any one of (6) to (8), The audio output unit includes a first speaker unit for reproducing the first audio signal and a second speaker unit for reproducing the second audio signal. Speaker device. (11) The speaker device according to any one of (6) to (10), The audio output unit includes a right speaker and a left speaker; The speaker device further includes a neck-hanging connector that connects the right speaker and the left speaker. Speaker device. (12) The speaker device according to any one of (6) to (11), The tactile presentation unit includes a vibration device. Speaker device. (13) The speaker device according to any one of (6) to (11), The tactile presentation unit includes airborne ultrasound, electromyography, or an air cannon. Speaker device. (14) acquiring a haptic signal for haptic presentation and a first audio signal; generating a haptic control signal for driving a haptic presentation unit based on the haptic signal; generating a second audio signal including an audio component in antiphase with the audio generated from the haptic presentation unit based on the haptic signal; generating an audio control signal for driving an audio output unit based on the first audio signal and the second audio signal; Audio output method. [Explanation of symbols]

[0082] 1...Control device 10…External network 11. Storage 12...Decoding section 13...Audio control unit 14…Haptic control unit 15...Audio output section 16…Tactile output unit 17, 27…Cancellation signal generator 20, 22…Speaker section 21... Transducer 80...Tactile presentation device 100, 200, 300...Speaker device 100C…Concatenated body 100L…Left speaker 100R…Right speaker

Claims

1. a haptic control unit that generates a haptic control signal for driving the haptic presentation unit based on the haptic signal for haptic presentation; a sound control unit that generates a sound control signal for driving a sound output unit based on a first sound signal and a second sound signal that is generated based on the haptic signal and includes a sound component in antiphase with a sound generated from the haptic presentation unit; a cancellation signal generation unit that generates the second audio signal based on the haptic signal; Equipped with The cancellation signal generation unit generates the second audio signal based on a frequency characteristic of the audio output unit, a frequency characteristic of the haptic presentation unit, and a conversion function from vibration generated by the haptic presentation unit to sound. Control device.

2. The control device according to claim 1 , The audio control unit generates the audio control signal by adding the second audio signal to the first audio signal. Control device.

3. The control device according to claim 1 , the audio control unit generates a first signal for driving a first audio output unit to reproduce the first audio signal; The cancellation signal generator generates a second signal for driving a second audio output unit that reproduces the second audio signal. Control device.

4. A tactile presentation unit; An audio output unit; a haptic control unit that generates a haptic control signal for driving the haptic presentation unit based on a haptic signal for haptic presentation; a sound control unit that generates a sound control signal for driving the sound output unit based on a first sound signal and a second sound signal that is generated based on the haptic signal and includes a sound component in antiphase with a sound generated from the haptic presentation unit; a cancellation signal generation unit that generates the second audio signal based on the haptic signal; Equipped with The cancellation signal generation unit generates the second audio signal based on a frequency characteristic of the audio output unit, a frequency characteristic of the haptic presentation unit, and a conversion function from vibration generated by the haptic presentation unit to sound. Speaker device.

5. The speaker device according to claim 4, A storage unit for storing the haptic signal and the first audio signal. Further comprising Speaker device.

6. The speaker device according to claim 4, The device further includes a communication unit capable of communicating with a server that stores the haptic signal and the first audio signal. Speaker device.

7. The speaker device according to claim 4, The audio output unit includes a common speaker unit for reproducing the first audio signal and the second audio signal. Speaker device.

8. The speaker device according to claim 4, The audio output unit includes a first speaker unit for reproducing the first audio signal and a second speaker unit for reproducing the second audio signal. Speaker device.

9. The speaker device according to claim 4, The audio output unit includes a right speaker and a left speaker; The speaker device further includes a neck-hanging connector that connects the right speaker and the left speaker. Speaker device.

10. The speaker device according to claim 4, The tactile presentation unit includes a vibration device. Speaker device.

11. The speaker device according to claim 4, The tactile presentation unit includes an ultrasonic generator, an electrical muscle stimulation device, or an air cannon generator. Speaker device.

12. obtaining a haptic signal for haptic presentation and a first audio signal; generating a haptic control signal for driving a haptic presentation unit based on the haptic signal; generating a second audio signal including an audio component in antiphase with the audio generated from the haptic presentation unit based on the haptic signal; generating an audio control signal for driving an audio output unit based on the first audio signal and the second audio signal; 1. A sound output method, comprising: The second audio signal is generated based on a frequency characteristic of the audio output unit, a frequency characteristic of the tactile presentation unit, and a conversion function from vibration generated by the tactile presentation unit to sound. Audio output method.

Citation Information

Patent Citations

  • Speaker equipment

    JP1998200977A

  • Computer input and output device using elastic wave

    JP2001255993A

  • Synchronization of haptic effect data in media transport streams

    JP2009513077A

  • Portable electronic apparatus, method for operating portable electronic apparatus, and operation program for portable electronic apparatus

    JP2010258807A

  • Method and system for coding and streaming tactile sense data

    JP2014239430A