CONTROL DEVICE, SIGNAL PROCESSING METHOD, AND SPEAKER DEVICE

Abstract

A control device according to one embodiment of the present invention is provided with a sound control unit and a vibration control unit.　The sound control unit uses sound signals of a plurality of channels as input signals to generate a sound control signal for each of the plurality of channels, each of said sound signals having a first sound component and a second sound component that is different from the first sound component. The vibration control unit takes the difference between sound signals of two channels from among the plurality of channels to generate a vibration control signal for vibration presentation.

Description

[Technical field]

[0001] The present technology relates to a control device, a signal processing method, and a speaker device. [Background technology]

[0002] In recent years, applications that stimulate the sense of touch through human skin using tactile reproduction devices have been used in a variety of situations. Currently, tactile reproduction devices in wide use include eccentric rotating masses (ERMs) and linear resonant actuators (LRAs), and these devices have a resonant frequency that is sensitive to human tactile sensation (around a few hundred Hz) (see, for example, Patent Document 1).

[0003] Since the frequency band in which humans have the highest tactile sensitivity is several hundred Hz, the majority of vibration reproduction devices target this frequency band. Other tactile reproduction devices proposed include electrostatic displays and surface acoustic wave displays that aim to realize the desired tactile sensation by controlling the friction coefficient of the touching part (see, for example, Patent Document 2). Other proposed devices include an airborne ultrasound tactile display that uses the acoustic radiation pressure of focused ultrasound, and an electrotactile display that electrically stimulates nerves and muscles connected to tactile receptors.

[0004] One application of these devices, particularly in music listening, is to incorporate a vibration reproduction device into the headphone housing and reproduce vibrations at the same time as playing music, emphasizing deep bass. Wearable (neck) speakers have also been proposed that are not headphones but are worn around the neck. These include those that utilize the contact with the user's body to transmit vibrations to the user from the back along with the sound output from the speaker (see, for example, Patent Document 3), and those that transmit vibrations to the user by utilizing the resonance of the back pressure of the speaker vibration (see, for example, Patent Document 4). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2016-202486 A [Patent Document 2] JP 2001-255993 A [Patent Document 3] Japanese Patent Application Publication No. 10-200977 [Patent Document 4] Patent Application No. 2017-43602 Summary of the Invention [Problem to be solved by the invention]

[0006] In headphones or wearable speakers that provide tactile sensations, when a vibration signal is generated from an audio signal and presented, generating a vibration signal from an audio signal that contains a large amount of human voice can generally result in vibrations that are unnatural or uncomfortable and that are not desired.

[0007] In view of the above circumstances, an object of the present invention is to provide a control device, a signal processing method, and a speaker device that can eliminate or reduce vibrations that are generally perceived as unnatural or uncomfortable. [Means for solving the problem]

[0008] A control device according to an embodiment of the present technology includes a sound control unit and a vibration control unit. The audio control unit receives audio signals of a plurality of channels, each of which has a first audio component and a second audio component different from the first audio component, as input signals, and generates an audio control signal for each of the plurality of channels. The vibration control unit takes a difference between audio signals of two channels out of the plurality of channels to generate a vibration control signal for vibration presentation.

[0009] The vibration control unit may be configured to band-limit the audio signals of the multiple channels or a differential signal of the audio signals of the multiple channels to a first frequency or lower.

[0010] The vibration control unit may be configured to output a mono signal obtained by mixing the audio signals of each channel as the vibration control signal for audio signals of the multiple channels that have a second frequency lower than the first frequency, and to output the differential signal as the vibration control signal for audio signals that exceed the second frequency and are lower than the first frequency.

[0011] The first frequency may be less than or equal to 500 Hz.

[0012] The second cutoff frequency may be less than or equal to 150 Hz.

[0013] The first sound component may be a voice sound.

[0014] The second audio components may be sound effects and background sounds.

[0015] The two channel audio signals may be left and right channel audio signals.

[0016] The vibration control unit may include an adjustment unit that adjusts a gain of the vibration control signal based on an external signal.

[0017] The adjustment unit may be configured to be able to switch between enabling and disabling generation of the vibration control signal.

[0018] The vibration control unit may include an adder unit that generates a monaural signal by mixing the audio signals of the two channels.

[0019] The vibration control unit may include a subtraction unit that calculates a difference between the audio signals. In this case, the subtraction unit is configured to be able to adjust a degree of subtraction of the difference.

[0020] A signal processing method according to one embodiment of the present technology includes receiving audio signals of a plurality of channels as input signals, each of the plurality of channels having a first audio component and a second audio component different from the first audio component, and generating an audio control signal for each of the plurality of channels. A difference between audio signals of two of the multiple channels is taken to generate a vibration control signal for vibration presentation.

[0021] A speaker device according to an embodiment of the present technology includes a sound output unit, a vibration output unit, a sound control unit, and a vibration control unit. The audio control unit receives audio signals of multiple channels as input signals, each of which has a first audio component and a second audio component different from the first audio component, generates audio control signals for each of the multiple channels, and drives the audio output unit. The vibration control unit takes a difference between audio signals of two of the multiple channels to generate a vibration control signal for vibration presentation, and drives the vibration output unit. [Brief description of the drawings]

[0022] [Figure 1] 1A and 1B are a perspective view and a bottom view of a speaker device according to a first embodiment of the present technology. [Diagram 2] FIG. 2 is a perspective view showing a state in which the speaker device is mounted on a user. [Diagram 3] 2 is a schematic cross-sectional view of a main part of the speaker device. FIG. [Figure 4] FIG. 2 is a block diagram showing a configuration example of the speaker device. [Diagram 5] 1 is a graph showing vibration detection threshold as a human tactile mechanism. [Figure 6] 1 is a graph showing a signal obtained by subjecting the spectrum of an audio signal to low-pass filter processing. [Figure 7] 4 is a flowchart for generating a vibration signal from an audio signal in the first embodiment of the present technology. [Figure 8] 1 is a graph showing a spectrum before differential processing, a spectrum after differential processing, and a spectrum after differential processing in which the low frequencies are retained. [Figure 9] 2 is a block diagram showing an internal configuration of a vibration control unit of the speaker device according to the embodiment. FIG. [Figure 10] 4 is a flowchart for generating a vibration signal from an audio signal in the first embodiment of the present technology. [Figure 11] FIG. 2 is a top view showing speaker arrangements in 5.1-channel and 7.1-channel audio signal formats. [Figure 12] 1 is a schematic diagram showing stream data relating to sound and vibration for a predetermined period of time. FIG. [Figure 13] FIG. 1 is a schematic diagram showing user interface software for controlling the gain of audio and vibration signals. [Figure 14] 11 is a graph showing an example of a signal of a sound effect and a background sound. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, embodiments of the present technology will be described with reference to the drawings.

[0024] <First embodiment> (Basic configuration of speaker device) 1A and 1B are a perspective view and a bottom view showing a configuration example of a speaker device according to an embodiment of the present technology. The speaker device (audio output device) 100 has a function of actively presenting vibration (tactile sensation) to a user U at the same time as audio. As shown in FIG. 2, the speaker device 100 is a wearable speaker placed on both shoulders of the user U, for example.

[0025] The speaker device 100 includes a right speaker 100R, a left speaker 100L, and a connector 100C that connects the right speaker 100R and the left speaker 100L. The connector 100C is formed in an arbitrary shape that allows it to be hung around the neck of a user U, and the right speaker 100R and the left speaker 100L are positioned on both shoulders or above the chest of the user U.

[0026] Fig. 3 is a schematic cross-sectional view of the main parts of the right speaker 100R and the left speaker 100L of the speaker device 100 in Fig. 1 and Fig. 2. The right speaker 100R and the left speaker 100L typically have a symmetrical structure on the left and right. Note that Fig. 3 is merely a schematic diagram and does not necessarily correspond to the shapes and dimensional ratios of the speakers shown in Fig. 1 and Fig. 2.

[0027] The right speaker 100R and the left speaker 100L each include, for example, an audio output unit 250, a vibration presentation unit 251, and a housing 254 that accommodates them. The right speaker 100R and the left speaker 100L typically reproduce audio signals in a stereo system. The reproduced sound is not particularly limited as long as it is a reproducible sound or audio, such as music, conversation, or sound effects.

[0028] The sound output unit 250 is an electroacoustic conversion type dynamic speaker. The sound output unit 250 includes a diaphragm 250a, a voice coil 250b wound around the center of the diaphragm 250a, a fixing ring 250c that holds the diaphragm 250a in a housing 254, and a magnet assembly 250d that is arranged facing the diaphragm 250a. The voice coil 250b is arranged perpendicular to the direction of the magnetic flux generated in the magnet assembly 250d. When an audio signal (AC current) is supplied to the voice coil 250b, the diaphragm 250a vibrates due to the electromagnetic force acting on the voice coil 250b. The diaphragm 250a vibrates in accordance with the signal waveform of the audio signal, generating a reproduced sound wave.

[0029] The vibration presentation unit 251 includes a vibration device (vibrator) capable of generating haptic vibration, such as an eccentric motor (ERM), a linear actuator (LRA), or a piezoelectric element. The vibration presentation unit 251 is driven by inputting a vibration signal for haptic presentation prepared separately from a playback signal. The amplitude and frequency of the vibration are not particularly limited. The vibration presentation unit 251 is not limited to being composed of a single vibration device, and may be composed of multiple vibration devices. In this case, the multiple vibration devices may be driven simultaneously or separately.

[0030] Housing 254 has an opening (sound guide port) 254a for passing audio output (reproduced sound) to the outside on a surface facing diaphragm 250a of sound output unit 250. Opening 254a is formed linearly along the longitudinal direction of housing 254 as shown in Fig. 1, but is not limited thereto and may be formed of a plurality of through holes or the like.

[0031] The vibration presentation unit 251 is disposed, for example, on the inner surface opposite to the opening 254a of the housing 254. The vibration presentation unit 251 presents haptic vibration to the user via the housing 254. In order to improve the transmissibility of the haptic vibration, a part of the housing 254 may be made of a material with a relatively low rigidity. The shape of the housing 254 is not limited to the shape shown in the figure, and any appropriate shape such as a disk shape or a rectangular parallelepiped shape can be adopted.

[0032] Next, a description will be given of a control system of the speaker device 100. Fig. 4 is a block diagram showing an example of the configuration of a speaker device applied to this embodiment.

[0033] The speaker device 100 includes a control device 1 that controls driving of an audio output unit 250 of the right speaker 100R and the left speaker 100L and a vibration presentation unit 251. The control device 1 and other elements described later are built into a housing 254 of the right speaker 100R or the left speaker 100L. The external device 60, which will be described later in detail, is an external device such as a smartphone or a remote controller, and operation information of a switch, button, etc. operated by a user is wirelessly transmitted and input to the control device 1 (described later).

[0034] As shown in FIG. 3, the control device 1 includes a voice control unit 13 and a vibration control unit . The control device 1 can be realized by hardware elements used in a computer, such as a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM), and necessary software. Instead of or in addition to the CPU, a programmable logic device (PLD) such as a field programmable gate array (FPGA), a digital signal processor (DSP), or an application specific integrated circuit (ASIC), etc. may be used. The control device 1 executes a predetermined program to configure a voice control unit 13 and a vibration control unit 14 as functional blocks.

[0035] The speaker device 100 includes a storage (storage unit) 11, a decoding unit 12, an audio output unit 15, a vibration output unit 16, and a communication unit 18 as other hardware components.

[0036] Based on an input signal such as a musical piece or other audio signal, the audio control unit 13 generates an audio control signal for driving the audio output unit 15. The audio signal is data for audio reproduction (audio data) stored in the storage 11 or the server device 50.

[0037] Based on the vibration signal, the vibration control section 14 generates a vibration control signal for driving the vibration output section 16. The vibration signal is generated by utilizing an audio signal, as will be described later.

[0038] The storage 11 is a storage device such as a non-volatile semiconductor memory capable of storing an audio signal. In this embodiment, the audio signal is stored in the storage 11 as appropriately encoded digital data.

[0039] The decoding unit 12 decodes the audio signal stored in the storage 11. The decoding unit 12 may be omitted as necessary, or may be configured as a functional block of the control device 1.

[0040] The communication unit 18 is configured as a communication module that can be connected to the network 10 by wire (e.g., a USB cable) or wirelessly by Wi-Fi, Bluetooth (registered trademark), etc. The communication unit 18 is configured as a receiving unit that can communicate with the server device 50 via the network 10 and acquire an audio signal stored in the server device 50.

[0041] The audio output section 15 includes, for example, an audio output unit 250 for the right speaker 100R and the left speaker 100L shown in FIG. The vibration output unit 16 includes, for example, the vibration presentation unit 251 shown in FIG.

[0042] (Typical operation of a speaker system) Next, a typical operation of the speaker device 100 configured as above will be described.

[0043] The control device 1 receives a signal from the server device 50 or reads a signal from the storage 11, and generates signals for driving the audio output unit 15 and the vibration output unit 16 (an audio control signal and a vibration control signal).

[0044] Next, the decoding unit 12 performs an appropriate decoding process on the acquired data to extract voice data (voice signal), and inputs the voice data (voice signal) to the voice control unit 13 and the vibration control unit 14, respectively. The audio data format may be a linear PCM format of raw data, or a data format that is highly efficiently encoded using an audio codec such as MP3 or AAC.

[0045] The voice control unit 13 and the vibration control unit 14 perform various processes on the input data. The output (voice control signal) of the voice control unit 13 is input to the voice output unit 15, and the output (vibration control signal) of the vibration control unit 14 is input to the vibration output unit 16. The voice output unit 15 and the vibration output unit 16 each include a D / A converter, a signal amplifier, and a playback device (corresponding to the voice output unit 250 and the vibration presentation unit 251). The D / A converter and the signal amplifier may be included in the audio control unit 13 and the vibration control unit 14. The signal amplifier may include a volume adjustment unit, an equalizing adjustment unit, a vibration amount adjustment unit by gain adjustment, etc., which are adjusted by the user U.

[0046] Based on the input audio data, the audio control unit 13 generates an audio control signal for driving the audio output unit 15. Based on the input tactile data, the vibration control unit 14 generates a vibration control signal for driving the vibration output unit 16.

[0047] When using a wearable speaker, it is common to use sound, which has a high correlation with vibration, since vibration signals are rarely provided separately from audio signals in broadcast content, packaged content, online content, game content, etc. In other words, processing is performed based on the audio signal, and the generated vibration signal is output. When such vibrations are presented, they may be perceived as unpleasant by the user. For example, when dialogue, narration, and live commentary in sports videos are presented as vibrations, users feel as if their body is being shaken by someone else's voice, which is often unpleasant for users.

[0048] In addition, these sound components have a relatively high volume and their central frequency band is within the vibration presentation frequency range (several hundred Hz), so they vibrate more than other vibration components, masking components such as impact, rhythm, and texture that should actually be vibrated. On the other hand, when playing content for which audio signals and vibration signals are prepared separately, vibrations that make the user feel uncomfortable or uncomfortable should not be presented because the vibration signals are created in advance with the intention of the content creator. However, since people have different sensory preferences, there is a possibility that vibrations that make the user feel uncomfortable or uncomfortable may be presented in some cases.

[0049] The control device 1 of this embodiment is configured as follows in order to eliminate or reduce vibrations that cause discomfort or discomfort to the user in an active vibration wearable speaker.

[0050] (Control device) As described above, the control device 1 has the voice control unit 13 and the vibration control unit 14. The voice control unit 13 and the vibration control unit 14 are configured to have the following functions in addition to the functions described above.

[0051] The audio control unit 13 receives audio signals of a plurality of channels, each of which has a first audio component and a second audio component different from the first audio component, as input signals, and generates an audio control signal for each of the plurality of channels. The audio control signal is a control signal for driving the audio output unit 15.

[0052] The first sound component is typically a voice sound. The second sound component is a sound component other than the voice sound, for example, a sound effect or a background sound. The second sound component may be both a sound effect and a background sound, or may be either one of them. In this embodiment, the number of channels is two, a left channel and a right channel. The number of channels is not limited to two, left and right, and may be three or more channels including a center channel, a rear channel, a subwoofer, etc.

[0053] The vibration control unit 14 calculates the difference between the audio signals of two of the multiple channels to generate a vibration control signal for vibration presentation. The vibration control signal is a control signal for driving the vibration output unit 16.

[0054] As described later, the same signal is usually used for the left and right channels for the voice sound, and the above-mentioned differential processing is used to obtain a vibration control signal in which the voice sound is cancelled out. This makes it possible to generate a vibration control signal based on an audio signal other than the voice sound, such as a sound effect or background sound.

[0055] On the other hand, the vibration detection threshold shown in Figure 5 is known as a human tactile mechanism (quoted from "Four cahnnels mediate the mechanical aspects of touch" SJ Bolanowski 1988). Humans are most sensitive to vibrations at frequencies between 200 and 300 Hz, with sensitivity decreasing as the frequency moves away from this band. Typically, the vibration presentation range is considered to be several Hz to 1 kHz, but in reality, frequencies above 500 Hz affect hearing as noise, so the upper limit is set to around 500 Hz.

[0056] In this embodiment, the vibration control unit 14 has a low-pass filter function that band-limits the audio signal to a predetermined frequency (first frequency) or lower. Fig. 6(A) shows a spectrum (logarithmic spectrum) 61 of the audio signal, and Fig. 6(B) shows a spectrum 62 obtained by subjecting the spectrum 61 to processing with a low-pass filter (for example, a cutoff frequency of 500 Hz). The vibration control unit 14 generates a vibration signal using the audio signal (spectrum 62) after the low-pass filter. The first frequency is not limited to 500 Hz, and may be a frequency lower than this.

[0057] Regarding the number of channels of the vibration signal, the left and right audio signals may be band-limited and output as two-channel vibration signals. However, the user may feel uncomfortable if different vibrations are presented on the left and right sides. In this embodiment, a mono signal that mixes the left and right channels is output as the same vibration signal on both sides. This mixed mono signal is calculated as the average value of the audio signals on the left and right channels, for example, as shown in the following (Equation 1).

[0058] VM(t)=(AL(t)+AR(t))×0.5 (Formula 1)

[0059] Here, VM(t) is the value of the vibration signal at time t, AL(t) is the value of the left channel of the band-limited audio signal at time t, and AR(t) is the value of the right channel of the band-limited audio signal at time t.

[0060] The above-mentioned configuration of the speaker device 100 enables reproduction of sound and vibration for existing content. In this embodiment, the vibration control unit 14 in Fig. 4 performs signal processing using (Equation 1) on two-channel digital sound signals of existing content, thereby removing or reducing noise generated from dialogue, narration, live commentary, etc.

[0061] Incidentally, the elements that make up a two-channel stereo audio signal in typical content are thought to include three major elements: voice sounds such as dialogue and narration, sound effects for production purposes, and background sounds such as music and environmental sounds. (Content audio = voice sound + sound effects + background sound)

[0062] Content creators adjust the sound quality and volume of each component, then mix them together to generate the final content. When doing so, they take into consideration the sense of sound localization (direction from which the sound comes), and typically assign voice as the same signal to both the left and right channels so that it is always heard from a stable foreground position (front). Sound effects and background sounds are usually assigned as different signals to the left and right channels to enhance the sense of realism.

[0063] 14 is a graph showing an example of a signal of a sound effect 141 (for example, a chime sound) and a background sound 142 (for example, a piece of music). Each signal has left channel data (upper row) and right channel data (lower row). It can be seen that both the sound effect 141 and the background sound 142 are similar in general form in the left and right channels, but are different signals.

[0064] This two-channel audio mixing is shown in (Equation 2) and (Equation 3), where AL(t) is the value of the audio signal in the left channel at time t, AR(t) is the value of the audio signal in the right channel at time t, S(t) is the value of the voice signal at time t, EL(t) is the value of the sound effect signal in the left channel at time t, ER(t) is the value of the sound effect signal in the right channel at time t, ML(t) is the value of the background sound signal in the left channel at time t, and MR(t) is the value of the background sound signal in the right channel at time t.

[0065] AL(t)= S(t)+EL(t)+ML(t)...(Formula 2) AR(t)= S(t)+ER(t)+MR(t)...(Formula 3)

[0066] Here, S(t) is cancelled out by using the signal obtained by performing differential processing of the left and right channels of the audio signal as the vibration signal VM(t) as shown in the following (Equation 4). This eliminates vibrations in response to audio signals such as dialogue, narration, and live commentary, and eliminates unpleasant vibrations.

[0067] VM(t) = AL(t) - AR(t) =EL(t)-ER(t)+ML(t)-MR(t)...(Formula 4) Incidentally, (Equation 4) may be AR(t)-AL(t).

[0068] As described above, the vibration control unit 14 is not limited to the case where the left and right channel audio signals are band-limited, the band-limited left and right channel audio signals are differentially processed, and the differentially processed audio signals are output as the vibration control signal. For example, as shown in Fig. 7, the vibration control unit 14 may perform differential processing on the left and right channels of the audio signals, perform band-limiting processing on the differentially processed audio signals (differential signals), and output the band-limited differential signals as the vibration control signal.

[0069] FIG. 7 is a flowchart showing another example of the procedure executed in the vibration control unit 14 for generating a vibration signal from a sound signal.

[0070] In step S71, the audio signal output from the decoding unit 12 in FIG. 4 is input, and a difference signal between the left and right channels of the audio signal is obtained according to the above-mentioned (Equation 4). Then, in step S72, the difference signal obtained in step S71 is subjected to low-pass filtering with a cutoff frequency equal to or lower than a predetermined frequency (eg, 500 Hz) in the same manner as in FIG. 6, thereby obtaining a band-limited audio signal.

[0071] After that, in step S73, the band-limited signal obtained in step S72 is multiplied by a gain coefficient corresponding to the vibration volume designated by the user via an external UI or the like. After that, in step S74, the signal obtained in step S73 is output to the vibration output section 16 as a vibration control signal.

[0072] Depending on the mixing method used by the content creator, the voice may be emphasized by applying effects such as reverb and compression. In this case, different signals are assigned to the left and right channels, but even in this case, the main components of the voice are assigned as the same signal to the left and right, so the difference signal (Equation 4) reduces the discomfort and unpleasant vibrations caused by the voice compared to normal signals.

[0073] On the other hand, according to the above-mentioned (Equation 4), VM(t) is a signal from which the same magnitude of signals (centrally located components) at the same time have been removed from both the left and right channels, but the EL(t), ER(t), ML(t) and MR(t) terms in (Equation 2) and (Equation 3) also contain signals of the same magnitude at the same time. In other words, by performing the processing of (Equation 4), the signal that is originally desired to vibrate may be damaged and may not vibrate. Also, since VM(t) in (Equation 4) is a difference result, if the correlation between the original signals is high, the magnitude of the signal may become smaller than the original signal.

[0074] For example, Fig. 8(A) shows a mixed monaural signal ((L+R) x 0.5) (corresponding to spectrum 62 in Fig. 6) of the audio signals of the left and right channels before differential processing, and Fig. 8(B) shows the spectrum (LR) 81 of the audio signal after differential processing. The spectrum 81 after differential processing has an overall level that is lower than the maximum value L1 (e.g., -24 dB) of spectrum 62, and furthermore, signals below 150 Hz are lost. Therefore, the frequency band below the lower limit frequency of the voice (human voice) (for example, 150 Hz) is excluded from the differential processing and the left and right signal addition processing (Equation 1) is performed, and the frequency band above the lower limit frequency is removed by differential processing. This makes it possible to maintain the low-frequency signal components that are to be vibrated, as shown in Figure 8(C).

[0075] That is, for audio signals of multiple channels that have a second frequency (150 Hz in this example) lower than the first frequency (500 Hz in this example) or lower, the vibration control unit 14 outputs a mono signal that mixes the audio signals of each channel as a vibration control signal, and for audio signals that exceed the second frequency and are equal to or lower than the first frequency, the vibration control unit 14 outputs the differential signal of these audio signals as a vibration control signal. The values ​​of the first frequency and the second frequency are not limited to the above example, and can be set arbitrarily.

[0076] FIG. 9 is a block diagram showing an example of the internal configuration of the vibration control unit 14 of the speaker device 100 in this embodiment. The vibration control unit 14 includes an adder 91, an LPF unit 92, a subtractor 93, a BPF unit 94, a synthesizer 95, and an adjuster 96.

[0077] The adder 91 down-mixes the two-channel audio signals received via the communication unit 18 to a monaural signal in accordance with (Equation 1). The LPF unit 92 performs low-pass filtering with a cutoff frequency of 150 Hz to convert the main components of the audio signal into signals with a frequency band of 150 Hz or less. The subtraction unit 93 performs differential processing on the two-channel audio signals received via the communication unit 18 in accordance with (Equation 4). The BPF unit 94 performs band-pass filtering with a passband of 150 Hz to 500 Hz to convert the main component of the audio signal into a signal of 150 Hz to 500 Hz. The combiner 95 combines the signal input from the LPF unit 92 and the signal input from the BPF unit 94 . The adjustment unit 96 is for adjusting the gain of the entire vibration control signal when adjusting the volume of the vibration by an input operation from the external device 60. The adjustment unit 96 outputs the gain-adjusted vibration control signal to the vibration output unit 16.

[0078] The adjustment unit 96 may further be configured to be capable of switching between enabling and disabling the generation of the vibration control signal by the addition process by the adder 91, the band limiting process by the LPF unit 92 and the BPF unit 94, and the subtraction process by the subtractor 93. In the case of the process that does not generate the vibration control signal (hereinafter also referred to as the generation disable process), the audio signal of each channel is directly input to the adjustment unit 96, and the vibration control signal is generated. The user can arbitrarily set whether or not to employ the generation invalidation process, and typically, a control command for the generation invalidation process is input to the adjustment unit 96 via the external device 60.

[0079] As described later, the subtraction unit 93 may also be configured to adjust the degree of subtraction when calculating the difference between the audio signals of the left and right channels via the external device 60. In other words, the generation of vibration control signals derived from voice sounds is not limited to being completely eliminated, and the magnitude of vibration derived from voice sounds may be arbitrarily set according to the user's preference. As a method for adjusting the degree of subtraction, for example, a difference signal between the left channel of a two-channel audio signal and the right channel multiplied by a coefficient is used as the vibration control signal. The coefficient can be set arbitrarily, and the audio signal multiplied by the coefficient may be the left channel instead of the right channel.

[0080] FIG. 10 is a flowchart showing a series of processes for generating a vibration signal from an audio signal in this embodiment. In step S101, the left and right signal addition process of (Equation 1) is performed in the adder 91. After that, in step S102, the LPF unit 92 performs low-pass filtering with a cutoff frequency of 150 Hz on the signal after the addition process.

[0081] After that, in step S103, the left and right signal difference process of (Equation 4) is performed in the subtraction unit 93. At this time, a voice reduction coefficient (described later) adjusted by the user and input from the external device 60 may be taken into consideration. Then, in step S104, the signal after the difference processing is subjected to bandpass filtering with a lower cutoff limit frequency of 150 Hz and an upper cutoff limit frequency of 500 Hz in the BPF unit 94. The upper cutoff limit frequency is appropriately selected in the same manner as the lower limit frequency. After that, in step S105, the signal after the process of step S102 and the signal after the process of step S104 are synthesized in the synthesis unit 95.

[0082] Then, in step S106, a signal obtained by multiplying the signal after the process of step S105 by a vibration gain coefficient set by the user via an external UI (User Interface) or the like is obtained in adjustment unit 96. Then, in step S107, the signal after the process of step S106 is output to vibration output units 16, 251 as a vibration control signal.

[0083] As described above, according to this embodiment, when a vibration signal is generated from a received audio signal, it is possible to remove or reduce vibration components that cause a user a sense of incongruity or discomfort.

[0084] <Second embodiment> For example, in disc standards such as DVD and Blue-Ray, digital broadcasting systems, game content, and the like, 5.1-channel or 7.1-channel audio signals are used as multi-channel audio formats. In these formats, the recommended speaker arrangement is as shown in Fig. 11, and content creators assign audio signals for each channel assuming this speaker arrangement. In particular, human voices such as dialogue and narration are generally assigned to the front center channel (FC in Fig. 11) so that they are heard from directly in front of the listener.

[0085] When the multi-channel audio format described above is input, the remaining signals except for the front center channel signal are downmixed and converted to mono or stereo signals. After that, the signal is low-pass filtered (e.g., cutoff frequency 500 Hz) and output as a vibration control signal. This prevents the vibration output section from vibrating in accordance with a person's voice, and the user does not feel uncomfortable vibrations.

[0086] When downmixing from 5.1 channels and 7.1 channels, for example, the following (Equation 5) and (Equation 6) are used, respectively.

[0087] VM(t)=αFL(t)+βFR(t)+γSL(t)+δSR(t)+εSW(t) (Formula 5) VM(t)=αFL(t)+βFR(t)+γSL(t)+δSR(t)+εSW(t)+θLB(t)+μRB(t) (Equation 6)

[0088] Here, VM(t) is the value of the vibration signal at time t, FL(t), FR(t), SL(t), SR(t), SW(t), LB(t) and RB(t) are the values ​​of the audio signal at time t corresponding to each speaker arrangement FL, FR, SL, SR, SW, LB and RB, and α, β, γ, δ, ε, θ and μ are downmix coefficients for each signal. The downmix coefficients may be any numerical values, or all channels may be equally divided and each coefficient may be set to, for example, 0.2 for (Equation 5) or 0.143 for (Equation 6).

[0089] As described above, in this embodiment, the front center channel signal of the multi-channel audio signal is removed or reduced, and then the other channels are downmixed to produce a vibration signal. This makes it possible to reduce or remove unpleasant vibrations in response to human voices when presenting vibrations using a multi-channel audio signal as input.

[0090] <Third embodiment> The first and second embodiments of the present technology remove or reduce voices in the content and maintain as many necessary vibration components as possible, but may not be suitable for music content in which it is desirable to express a sense of rhythm as vibration or for some user subjective preferences. Therefore, a mechanism is provided that allows a user to voluntarily select whether to implement the present technology. In this case, the enable / disable control may be performed by software in a content transmitter (for example, an external device 60 such as a smartphone, a television, or a game machine), or the control may be performed by providing an operation unit (not shown) such as a hardware switch or button on the housing 254 of the speaker device 100.

[0091] In addition to the enable / disable control, a function for adjusting the degree of voice reduction may be provided. The following formula (7) shows the formula (4) with voice reduction degree adjustment, and the formulas for multi-channel are shown in (8) (5.1 channel) and (9) (7.1 channel). VM(t)=AL(t)-AR(t)×Coeff (Formula 7) VM(t)=αFL(t)+βFR(t)+γSL(t)+δSR(t)+εSW(t)+FC(t)×Coeff (Formula 8) VM(t)=αFL(t)+βFR(t)+γSL(t)+δSR(t)+εSW(t)+θLB(t)+μRB(t)+FC(t)×Coeff (Formula 9)

[0092] Here, Coeff is a voice reduction coefficient and takes a positive real number equal to or less than 1.0. The closer Coeff is to 1.0, the more effective the voice reduction is, and the closer it is to 0, the less effective the voice reduction is. In this embodiment, by providing such an adjustment function, the user can freely adjust the degree of voice reduction (that is, the degree of vibration) to suit his or her own preferences.

[0093] The coefficients Coeff in (Equation 7), (Equation 8), and (Equation 9) are adjusted by the user in the external device 60. The adjusted coefficients Coeff are input from the external device 60 to the subtraction unit 93 (see FIG. 9). In the subtraction unit 93, differential processing of the audio signal is performed according to (Equation 7), (Equation 8), or (Equation 9) in accordance with the number of input channels.

[0094] <Fourth embodiment> The above describes an embodiment in which a vibration signal is generated from an audio signal and vibrations are presented to the user. However, in this embodiment, a case is described in which a vibration signal independent of an audio signal is included as a future content configuration. FIG. 12 is a schematic diagram showing stream data relating to sound and vibration for a predetermined period of time (eg, several ms).

[0095] The stream data 121 includes a header 122, audio data 123, and vibration data 124. The stream data 121 may also include video data. The header 122 stores information about the entire frame, such as a sync word for identifying the beginning of the stream, the entire data size, and information indicating the data type. This is followed by audio data 123 and vibration data 124. The audio data 123 and vibration data 124 are transmitted to the speaker device 100 over time.

[0096] As an example, it is assumed here that the audio data is a two-channel audio signal (left and right), and the vibration data is a four-channel vibration signal. For example, voice sounds, sound effects, background sounds, and rhythms may be set in these four channels. Each part of a musical band, such as vocals, bass, guitar, and drums, may also be set in these four channels.

[0097] The external device 60 is provided with user interface software (UI or GUI (external operation input unit)) 131 for controlling the gain of audio and vibration signals (see FIG. 13). The signal gain of each channel of audio and signals is controlled by the user operating a control tool (e.g., a slider) displayed on the screen. This allows the user to reduce or eliminate unpleasant vibrations according to their own preferences by reducing the gain of the channel corresponding to the vibration signal that is output and that the user finds unpleasant.

[0098] As described above, in this embodiment, when an audio signal and a vibration signal are received independently, the vibration of a channel that the user does not want to vibrate among the vibration signal channels used for vibration presentation is muted or reduced by controlling the channel on the user interface. This allows the user to reduce or eliminate unpleasant vibrations according to his or her preference.

[0099] <Other technologies> In the above first embodiment, the description has been given on the two-channel stereo sound that is most commonly used in existing content, but in some cases, it may be possible to process one-channel monaural sound content. In this case, since differential processing between the left and right channels is impossible, it is possible to estimate and remove the human voice components. One possible method is, for example, the use of mono channel sound source separation technology. Specifically, NMF (non-negative matrix factorization) and RPCA (robust principal component analysis) are examples of methods that can be used to estimate the signal components of the human voice, and the vibration caused by the voice can be reduced by subtracting the estimated signal components from VM(t) in Equation 1.

[0100] The present technology can also be configured as follows. (1) an audio control unit that receives audio signals of a plurality of channels, each of which has a first audio component and a second audio component different from the first audio component, as an input signal, and generates an audio control signal for each of the plurality of channels; a vibration control unit that calculates a difference between audio signals of two channels among the plurality of channels and generates a vibration control signal for vibration presentation; A control device comprising: (2) The control device according to (1), The vibration control unit band-limits the audio signals of the multiple channels or a differential signal of the audio signals of the multiple channels to a first frequency or lower. Control device. (3) The control device according to (2), The vibration control unit is configured to: For an audio signal having a second frequency lower than the first frequency, a mono signal obtained by mixing the audio signals of each channel is output as the vibration control signal; For an audio signal that exceeds the second frequency and is equal to or lower than the first frequency, the differential signal is output as the vibration control signal. Control device. (4) The control device according to (2) or (3), The first frequency is less than or equal to 500 Hz. Control device. (5) The control device according to (3) above, The second cutoff frequency is 150 Hz or less. Control device. (6) The control device according to any one of (1) to (5) above, The first sound component is a voice sound. Control device. (7) The control device according to any one of (1) to (6) above, The second audio components are sound effects and background sounds. Control device. (8) The control device according to any one of (1) to (7) above, The two channel audio signals are left and right channel audio signals Control device. (9) The control device according to any one of (1) to (8), The vibration control unit has an adjustment unit that adjusts a gain of the vibration control signal based on an external signal. Control device. (10) The control device according to (9), The adjustment unit is configured to be able to switch between enabling and disabling generation of the vibration control signal. Control device. (11) The control device according to any one of (1) to (9), The vibration control unit has an adder unit that generates a monaural signal by mixing the audio signals of the two channels. Control device. (12) The control device according to any one of (1) to (11), The vibration control unit has a subtraction unit that calculates a difference between the audio signals, The subtraction unit is configured to be able to adjust the degree of subtraction of the difference. Control device. (13) A method for generating an audio control signal for each of a plurality of channels using, as an input signal, audio signals of a plurality of channels each having a first audio component and a second audio component different from the first audio component; A difference between audio signals of two of the multiple channels is taken to generate a vibration control signal for vibration presentation. Signal processing methods. (14) an audio output unit; A vibration output unit; an audio control unit that receives audio signals of a plurality of channels as input signals, each of which has a first audio component and a second audio component different from the first audio component, generates an audio control signal for each of the plurality of channels, and drives the audio output unit; a vibration control unit that calculates a difference between audio signals of two channels among the plurality of channels to generate a vibration control signal for vibration presentation and drives the vibration output unit; A speaker device comprising: [Explanation of symbols]

[0101] 1...Control device 10…External Network 11. Storage 12...Decoding section 13...Audio control unit 14...Tactile (vibration) control unit 15...Audio output section 16...Tactile (vibration) output unit 20, 22…Speaker section 21... Transducer 60...External device 80…Tactile presentation device 100, 200, 300...Speaker device 100C…Concatenated body 100L…Left speaker 100R…Right speaker 250…Audio output unit 251...Tactile (vibration) display unit

Claims

1. an audio control unit that receives audio signals of a plurality of channels, each of which has a first audio component and a second audio component different from the first audio component, as input signals, and generates an audio control signal for each of the plurality of channels; a vibration control unit that calculates a difference between audio signals of two channels among the plurality of channels and generates a vibration control signal for vibration presentation; A control device comprising:

2. The control device according to claim 1 , The vibration control unit band-limits the audio signals of the multiple channels or a differential signal of the audio signals of the multiple channels to a first frequency or lower. Control device.

3. The control device according to claim 2, The vibration control unit is configured to: For an audio signal having a second frequency lower than the first frequency, a mono signal obtained by mixing the audio signals of the respective channels is output as the vibration control signal; For an audio signal that exceeds the second frequency and is equal to or lower than the first frequency, the difference signal is output as the vibration control signal. Control device.

4. The control device according to claim 2, The first frequency is less than or equal to 500 Hz. Control device.

5. The control device according to claim 3, The second frequency is less than or equal to 150 Hz. Control device.

6. The control device according to claim 1 , The first sound component is a voice sound. Control device.

7. The control device according to claim 1 , The second audio components are sound effects and background sounds. Control device.

8. The control device according to claim 1 , The two channel audio signals are left and right channel audio signals. Control device.

9. The control device according to claim 1 , The vibration control unit has an adjustment unit that adjusts a gain of the vibration control signal based on an external signal. Control device.

10. The control device according to claim 9, The adjustment unit is configured to be able to switch between enabling and disabling generation of the vibration control signal. Control device.

11. The control device according to claim 1 , The vibration control unit has an adder unit that generates a monaural signal by mixing the audio signals of the two channels. Control device.

12. The control device according to claim 1 , The vibration control unit has a subtraction unit that calculates a difference between the audio signals, The subtraction unit is configured to be able to adjust the degree of subtraction of the difference. Control device.

13. generating an audio control signal for each of a plurality of channels using, as an input signal, audio signals of a plurality of channels each having a first audio component and a second audio component different from the first audio component; A difference between audio signals of two channels among the plurality of channels is taken to generate a vibration control signal for vibration presentation. Signal processing methods.

14. An audio output unit; A vibration output unit; an audio control unit that receives audio signals of a plurality of channels as input signals, each of the audio signals having a first audio component and a second audio component different from the first audio component, generates an audio control signal for each of the plurality of channels, and drives the audio output unit; a vibration control unit that calculates a difference between audio signals of two channels among the plurality of channels to generate a vibration control signal for vibration presentation and drives the vibration output unit; A speaker device comprising:

Images