Audio equipment and portable devices

Abstract

We provide "audio equipment and portable devices" that generate a good sense of localization and realism. [Solution] A left speaker L_SPK is provided on the left side of the front of the game controller 100, a right speaker R_SPK on the right side of the front, a center speaker C_SPK in the center of the left-right direction on the front, and a rear center speaker CU_SPK in the center of the left-right direction on the back. The left and right 2ch stereo signals SigIN output by the sound source device 1 are upmixed to generate a 3ch surround audio signal consisting of Lmid, Rmid, and Cmid audio signals. Lmid is Lin, Rmid is Rin, and Cmid is the sum of the components correlated with the other of Lin and Rin. Lmid is output from the left speaker L_SPK, Rmid is output from the left speaker R_SPK, and a sound with a sliver effect applied to Cmid is output from the rear center speaker CU_SPK.

Description

Technical Field

[0001] The present invention relates to an audio device and a portable device.

Background Art

[0002] As a technology related to the present invention, there is known a technology for generating left and right channel audio signals by respectively convolving acoustic transfer characteristics (head transfer functions) from an audio localization target position to each of the listener's left and right ears with an audio signal and outputting them to left and right speakers, thereby providing a sense of audio localization to the audio localization target position (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When applying the technology of outputting the left and right channel audio signals obtained by convolving the head transfer functions to each of the left and right ears described above to the left and right speakers provided in a relatively small portable device such as a game controller, it has been difficult to generate an audio with a good sense of localization and presence due to factors such as size.

[0005] Therefore, an object of the present invention is to provide an audio device that can generate an audio with a good sense of localization and presence even in a portable device.

Means for Solving the Problems

[0006] To achieve the above objectives, the present invention provides a portable device used in a handheld form, comprising: a left speaker positioned to the left of the user's perspective and outputting sound toward the user; a right speaker positioned to the right of the user's perspective and outputting sound toward the user; a rear center speaker positioned in the center of the portable device and outputting sound in the opposite direction to the user; a pseudo-surround signal generation means that generates a left channel audio signal, a right channel audio signal, and a center channel audio signal which is an audio signal representing the mutually correlated components in the left and right two-channel audio signals from a target audio signal which is a left and right two-channel audio signal; and a reverb effect application means that applies a reverb effect to the center channel audio signal and generates the audio signal with the reverb effect applied as the rear center channel audio signal. Here, the left channel audio signal is output from the left speaker, the right channel audio signal is output from the right speaker, and the rear center channel audio signal is output from the rear center speaker.

[0007] Furthermore, the audio device may be provided with a center speaker positioned in the center of the portable device from the user's perspective, which outputs sound towards the user, and the audio signal of the center channel may be output from the center speaker. Furthermore, the audio device may be provided with a localization processing unit that generates, as the left and right symmetric audio signals, an audio signal obtained by convolving a head-related transfer function from the localization position of the target sound image to the user's left ear into a monaural 1-channel audio signal, and an audio signal obtained by convolving a head-related transfer function from the localization position of the target sound image to the user's right ear into the 1-channel audio signal.

[0008] Furthermore, the audio device may be equipped with a sensor that detects values ​​related to the reflection of sound output from the rear center speaker, and the reverb effect application means may change the characteristics of the reverb effect according to the reflection conditions estimated from the values ​​detected by the sensor.

[0009] Furthermore, the audio device may be equipped with a sensor that detects the distance to a reflector that reflects sound output from the rear center speaker, which is located in the opposite direction from the user as viewed from the portable device, and the reverb effect application means may change the characteristics of the reverb effect according to the distance to the reflector detected by the sensor.

[0010] With the audio device described above, a center channel audio signal is generated from the left channel audio signal and the right channel audio signal. The left channel audio signal is output from the left speaker, the right channel audio signal is output from the right speaker, and the center channel audio signal is output from the rear center speaker as a rear center channel audio signal with a reverb effect applied.

[0011] Therefore, the user can be given a sense of localization through three audio signals: left / right channel audio signals output from the left / right speakers, and a rear-center channel audio signal output from the rear-center speaker that contains only components correlated with both the left and right channels. Furthermore, the rear-center channel audio signal output from the rear-center speaker is not simply the same as the center channel audio signal, but rather has a reverb effect added to it, creating a distinctive audio signal for the user, thereby generating a good sense of localization and a realistic sound image.

[0012] Furthermore, by incorporating a center speaker and outputting the audio signal from the center speaker, it is possible to generate a sound image with even greater localization. Now, the present invention provides a portable device equipped with the above-described audio device. Here, this portable device may be a game controller. [Effects of the Invention]

[0013] As described above, the present invention provides an audio device that can generate a sound image with good localization and realism, even in portable devices. [Brief explanation of the drawing]

[0014] [Figure 1] This figure shows a game controller according to an embodiment of the present invention. [Figure 2] This diagram shows the output direction of each speaker in a game controller according to an embodiment of the present invention. [Figure 3] This figure shows the configuration of an audio signal processing device according to an embodiment of the present invention. [Figure 4] This figure shows the configuration of a component separation unit that can be used in a surround audio signal generation unit according to an embodiment of the present invention. [Figure 5] This figure shows an example of how to use a game controller according to an embodiment of the present invention. [Figure 6] This figure shows another example configuration of the audio signal processing device according to an embodiment of the present invention. [Modes for carrying out the invention]

[0015] The embodiments of the present invention will be described below, using the application to a game controller as an example. Figure 1 shows the game controller 100 according to this embodiment. Figure 1a shows the front side of the game controller 100, and Figure 1b shows the back side of the game controller 100. In each figure, the bottom of the page is the user's side (the front), and the top of the page, opposite the front, is the back. As shown in the figure, the game controller 100 has various operation keys and a display for game operations arranged on its surface, and the lower left and right parts of the operation panel with two operation keys for game operations arranged on the back side are extended forward so that they can be held by the left and right hands respectively. A left speaker L_SPK is provided at the left position on the front side of the game controller 100, a right speaker R_SPK is provided at the right position on the front side of the game controller 100, and a center speaker C_SPK is provided at the center position in the left - right direction on the front side of the game controller 100. Also, a rear center speaker CU_SPK and a sensor S which is a distance sensor and a microphone are provided at the center position in the left - right direction on the back side of the game controller 100. Here, as shown in FIG. 2, the left speaker L_SPK, the right speaker R_SPK, and the center speaker C_SPK output sound from the front side of the game controller 100 toward the user (in the opposite direction to the back side), and the rear center speaker CU_SPK outputs sound from the back side of the game controller 100 in the opposite direction to the user (in the opposite direction to the front side). Note that M in the figure is a monitor for displaying the game screen.

[0016] Next, a sound source device and an acoustic signal processing device are housed in the game controller 100, and the game controller - built - in audio device is constituted by these sound source device, acoustic signal processing device, left speaker L_SPK, right speaker R_SPK, center speaker C_SPK, rear center speaker CU_SPK, and sensor S.

[0017] FIG. 3 shows the configuration of this audio device. As shown in the figure, the audio device includes a sound source device 1 that outputs 2 - channel stereo signals SigIN of Lin and Rin, and an acoustic signal processing device 2. Then, the audio signal processing device 2 includes a pseudo surround audio signal generation unit 21 that upmixes the 2-channel stereo signals SigIN of Lin and Rin input from the sound source device 1 to generate a 3-channel surround audio signal composed of the audio signals of Lmid, Rmid, and Cmid. Here, Lmid is Lin itself, or the sum of a component uncorrelated with Rin of Lin and a component correlated with Rin, Rmid is Rin itself, or the sum of a component uncorrelated with Lin of Rin and a component correlated with Lin, and Cmid is the sum of the components correlated with the other of Lin and Rin. However, Lmid may be the sum by weighted addition of a component uncorrelated with Rin of Lin and a component correlated with Rin, and Rmid may be the sum by weighted addition of a component uncorrelated with Lin of Rin and a component correlated with Lin. In this case, the weights used for each weighted addition are those pre-tuned so as to obtain a desired sense of localization.

[0018] Here, such separation of the correlated component and the uncorrelated component can be performed using a component separation unit having the configuration shown in FIG. 4. The component separation unit shown in FIG. 4 separates, from signal A and signal B, a component CA of signal B correlated with signal A and a component SB of signal B uncorrelated with signal A. As shown in the figure, the component separation unit includes a variable filter 41, an update unit 42 that updates the transfer function (filter coefficient) W of the variable filter 41 by an adaptive algorithm such as the LMS algorithm, and an adder 43. The variable filter 41, the update unit 42, and the adder 43 constitute an adaptive filter.

[0019] The variable filter 41 takes signal A as an input, and the adder 43 subtracts and outputs the output of the variable filter 41 from signal B. The update unit 42 executes an adaptive algorithm using the output of the adder 43 as an error, and updates the transfer function W of the variable filter 41 so that the power of the error is minimized.

[0020] The power of the adder 43's output is minimized when the output of the variable filter 41 matches component CA of signal B, which is correlated with signal A. At this point, the output of the adder 43, obtained by subtracting the output of the variable filter 41 from signal B, represents component SB of signal B, which is uncorrelated with signal A. Therefore, the component CA of signal B that is correlated with signal A can be separated as the output of the variable filter 41, and the component SB of signal B that is uncorrelated with signal A can be separated as the output of the adder 43. Furthermore, by swapping signal A in signal B, the component CB of signal A that is correlated with signal B and the component SA of signal A that is uncorrelated with signal B can be separated in the same way.

[0021] Returning to Figure 3, the acoustic signal processing device 2 includes a reverb effect application unit 22 that applies a reverb effect (reverberation effect) to the Cmid output by the pseudo-surround audio signal generation unit 21 and outputs it as CUmid, and an effect characteristic control unit 23 that can control the characteristics of the reverb effect applied by the reverb effect application unit 22 according to the detected value of the sensor S.

[0022] However, the characteristics of the reverb effect applied by the reverb effect application unit 22 are fixed, and the effect characteristic control unit 23 does not need to perform any control over them.

[0023] Next, the sound signal processing device 2 includes a left amplifier L_AMP that drives the left speaker L_SPK with a signal Lout which is an amplified signal Lmid output by the pseudo-surround audio signal generation unit 21 and outputs left channel sound from the left speaker L_SPK; a right amplifier R_AMP that drives the right speaker R_SPK with a signal Rout which is an amplified signal Rmid output by the pseudo-surround audio signal generation unit 21 and outputs right channel sound from the right speaker R_SPK; a center amplifier C_AMP that drives the center speaker C_SPK with a signal Cout which is an amplified signal Cmid output by the pseudo-surround audio signal generation unit 21 and outputs center channel sound from the center speaker C_SPK; and a CU center amplifier L_AMP that drives the rear center speaker CU_SPK with a signal CUout which is an amplified signal CUmid output by the reverb effect application unit 22 and outputs rear center channel sound from the rear center speaker CU_SPK.

[0024] Next, we will explain the case in which the effect characteristic control unit 23 controls the characteristics of the reverb effect applied by the reverb effect application unit 22 according to the detected value of the sensor S. As shown in Figures 5a and 5b, the direct sound of the rear center channel output from the rear center speaker CU_SPK (solid arrow in the figure) does not reach the user; instead, the user hears the rear center channel sound (dotted arrow in the figure) that has been reflected off the desk or floor in front of the user. As shown in the diagram, the reflection points of the rear center channel sound change depending on how the user uses the game controller 100, and consequently, the distance to the reflection points, the intensity of the reflections at those points, and the sound quality also change. Therefore, if the characteristics of the reverb effect applied by the reverb effect application unit 22 are fixed, the rear center channel sound heard by the user will differ depending on how the user uses the game controller 100.

[0025] Therefore, the effect characteristic control unit 23 controls the characteristics of the reverb effect applied by the reverb effect application unit 22 according to the detected value of the sensor S, so that the rear center channel sound is heard by the user as similarly as possible, regardless of how the user uses the game controller 100.

[0026] Here, the reverb effect applied by the reverb effect application unit 22 is basically done by using the Cmid output by the pseudo-surround audio signal generation unit 21 as the direct sound, and mixing the direct sound with the pseudo-reverberation sound to generate CUmid. The simulated reverberation consists of early reflections and late reverberations. Here, the characteristics of the reverb effect to be controlled can be the overall gain of CUmid, the gain of the direct sound, the gain of the simulated reverberation, the delay time, the decay time, the sound quality, and the gain, delay time, decay time, and sound quality of the early reflections and late reverberations respectively.

[0027] More specifically, for example, if a distance sensor is provided as the sensor S, the effect characteristic control unit 23 controls the characteristics of the reverb effect applied by the reverb effect application unit 22 so that the greater the distance to the reflection point of the rear center channel sound detected by the sensor S, the greater the overall gain of CUmid.

[0028] Furthermore, if a microphone is provided as the sensor S, the overall gain of CUmid is controlled by the reverb effect application unit 22 to control the characteristics of the reverb effect applied so that the volume of the rear center channel sound detected by the sensor S reaches a predetermined level. Embodiments of the present invention have been described above. With the audio system described above, the user can be given a sense of localization through audio signals consisting of left / right channel sounds output from the left speaker L_SPK and right speaker R_SPK, and a rear center channel sound output from the rear center speaker CU_SPK that contains only components correlated with both the left and right channels. Furthermore, since the rear center channel sound output from the rear center speaker CU_SPK is an audio signal that gives the user a sense of uniqueness by adding a reverb effect to the center channel sound, it is possible to generate a sound image with good localization and a sense of presence.

[0029] Furthermore, in this embodiment, the center channel sound is also output from the center speaker C_SPK to further enhance the sense of localization. Furthermore, the center speaker C_SPK and the configuration for outputting the center channel sound from the center speaker C_SPK in the above embodiments may be omitted if a sufficiently good sense of localization and a realistic sound image can be generated without providing these components. The above describes the case where the sound source device 1 of the audio system outputs a stereo signal SigIN, but this embodiment can also be applied when the sound source device 1 outputs a monaural signal. In other words, in this case, as shown in Figure 6, a localization processing device 6 is provided that generates a two-channel stereo signal SigIN (Lin and Rin) from the monaural signal SIN output by the sound source device 1 and outputs it to the sound signal processing device 2. Here, the positioning processing device 6 includes a head transfer function convolution unit 61 and a positioning control unit 62. Furthermore, the head-to-head transfer function convolution section 61 includes a left orbital head-to-head transfer function convolution section 611 and a right orbital head-to-head transfer function convolution section 612. The localization control unit 62 sets the head-related transfer function from the localization position to the listener's left ear in the left ear head-related transfer function convolution unit 611, and sets the head-related transfer function from the localization position to the listener's right ear in the right ear head-related transfer function convolution unit 612, according to the localization position of the sound image of the set target audio signal SIN.

[0030] Here, the localization position of the sound image of the target audio signal S may be determined by setting the distance from the localization position to the listener as a predetermined distance, depending on the direction of the localization position relative to the listener. The left orbital head-level transfer function convolution unit 611 convolves the head-level transfer function set by the localization control unit 62 onto the audio signal SIN output by the sound source device 1 and outputs it as Lin to the acoustic signal processing device 2. The right orbital head-level transfer function convolution unit 612 convolves the head-level transfer function set by the localization control unit 62 onto the audio signal SIN output by the sound source device 1 and outputs it as Rin to the acoustic signal processing device 2.

[0031] Although the above explanation has used the game controller 100 as an example, this embodiment is not limited to the game controller 100, but can be similarly applied to any portable device used in a handheld form by the user, such as other controllers or tablet devices. Furthermore, it can also be similarly applied to portable devices that only display information without accepting user input, such as portable displays. [Explanation of Symbols]

[0032] 1...Sound source device, 2...Acoustic signal processing device, 6...Sound localization processing device, 21...Pseudo-surround audio signal generation unit, 22...Reverb effect application unit, 23...Effect characteristic control unit, 41...Variable filter, 42...Update unit, 43...Adder, 61...Head-related transfer function convolution unit, 62...Sound localization control unit, 100...Game controller, 611...Left ear-head-related transfer function convolution unit, 612...Right ear-head-related transfer function convolution unit.

Claims

1. A portable device used in a handheld form, with a left speaker positioned on the left side from the user's perspective, which outputs sound towards the user, The portable device has a right speaker positioned to the right of the user, which outputs sound towards the user, The aforementioned portable device includes a rear center speaker positioned in the center from the user's perspective, which emits sound in the direction opposite to the user, A pseudo-surround signal generation means generates a left channel audio signal, a right channel audio signal, and a center channel audio signal which is an audio signal representing the mutually correlated components in the left and right two-channel audio signals, from a target audio signal which is a left and right two-channel audio signal. The system includes a reverb effect application means that applies a reverb effect to the audio signal of the center channel and generates the audio signal with the reverb effect applied as the audio signal of the rear center channel. An audio device characterized in that the left channel audio signal is output from the left speaker, the right channel audio signal is output from the right speaker, and the rear center channel audio signal is output from the rear center speaker.

2. The audio device according to claim 1, The aforementioned portable device is equipped with a center speaker positioned in the center from the user's perspective, which outputs sound towards the user. An audio device characterized in that the audio signal of the center channel is output from the center speaker.

3. The audio device according to claim 1, An audio device characterized by having a localization processing unit that generates, as the left and right symmetric audio signals, an audio signal obtained by convolving a head-related transfer function from the localization position of the target sound image to the user's left ear into a monaural 1-channel audio signal, and an audio signal obtained by convolving a head-related transfer function from the localization position of the target sound image to the user's right ear into the 1-channel audio signal.

4. The audio device according to claim 1, The system includes a sensor that detects values ​​related to the reflection of sound output from the rear center speaker, The reverb effect application means is characterized by changing the characteristics of the reverb effect according to the reflection conditions estimated from the values ​​detected by the sensor.

5. The audio device according to claim 1, The portable device is equipped with a sensor that detects the distance to a reflector that reflects sound output from the rear center speaker, which is located in the opposite direction from the user as viewed from the portable device. The reverb effect application means is characterized by changing the characteristics of the reverb effect according to the distance to the reflector detected by the sensor.

6. A portable device comprising the audio device according to claim 1, 2, 3, 4, or 6.

7. A portable device according to claim 6. The portable device in question is characterized by being a game controller.

Images