Audio device and audio control method

Abstract

To provide an acoustic device and an acoustic control method that can perform proper surround reproduction according to a sound source signal.SOLUTION: An acoustic device comprises a separation part and an output control part. The separation part separates a predetermined sound source signal which requires no additional processing for pseudo reverberant sound from the sound source signal, and then removes the separated predetermined sound source signal from the sound source signal. The output control part applies a filter for generating a pseudo reverberant sound to the sound signal having the predetermined sound source signal removed by the separation part, and outputs the resulting signal.SELECTED DRAWING: Figure 1A

Description

[Technical field]

[0001] The present invention relates to an audio device and an audio control method. [Background technology]

[0002] Conventionally, there is known an acoustic device that outputs various sound source signals, such as music and voice, from multiple channels (see, for example, Patent Document 1). In the conventional technology, a pseudo reverberation sound generated from the sound source signal is actively added to a direct sound, and surround sound is reproduced on multiple channels. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5372142 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has room for improvement in terms of performing appropriate surround reproduction according to the sound source signal.

[0005] The present invention has been made in view of the above, and has an object to provide an audio device and an audio control method capable of performing appropriate surround reproduction according to a sound source signal. [Means for solving the problem]

[0006] In order to solve the above problems and achieve the object, the present invention provides an acoustic device comprising a separation unit and an output control unit. The separation unit separates a predetermined sound source signal that does not require processing to add artificial reverberation from a sound source signal, and removes the separated predetermined sound source signal from the sound source signal. The output control unit applies a filter for generating the artificial reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation unit, and outputs the result. Effect of the Invention

[0007] According to the present invention, it is possible to perform appropriate surround reproduction according to a sound source signal. [Brief description of the drawings]

[0008] [Figure 1A] FIG. 1A is a diagram for explaining an overview of an acoustic control method according to a first embodiment. [Figure 1B] FIG. 1B is a diagram for explaining an overview of the acoustic control method according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the configuration of an audio system including the audio device according to the first embodiment. [Diagram 3] FIG. 3 is a diagram illustrating the separation and extraction process performed by the separation unit. [Figure 4A] FIG. 4A is a diagram for explaining the sound image of the reproduced sound, etc. FIG. [Figure 4B] FIG. 4B is a diagram for explaining the sound image of the reproduced sound, etc. [Figure 4C] FIG. 4C is a diagram for explaining the sound image of the reproduced sound, etc. [Figure 4D] FIG. 4D is a diagram for explaining the sound image of the reproduced sound, etc. [Figure 4E] FIG. 4E is a diagram for explaining the sound image of the reproduced sound, etc. [Figure 4F] FIG. 4F is a diagram for explaining the sound image of the reproduced sound. [Figure 4G] FIG. 4G is a diagram for explaining the sound image of the reproduced sound, etc. [Diagram 5] FIG. 5 is a flowchart showing a processing procedure executed by the audio device according to the first embodiment. [Figure 6] FIG. 6 is a block diagram showing an example of the configuration of an audio system including an audio device according to the second embodiment. [Figure 7] FIG. 7 is a diagram illustrating a gain determination process performed by the determination unit. [Figure 8]FIG. 8 is a flowchart showing a processing procedure executed by the audio device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of an audio device and an audio control method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment.

[0010] (First embodiment) <Overview of Acoustic Control Method Using Acoustic Device According to First Embodiment> First, an overview of an acoustic control method using an acoustic device according to a first embodiment will be described below with reference to Fig. 1A and Fig. 1B. Fig. 1A and Fig. 1B are diagrams for explaining the overview of the acoustic control method according to the first embodiment.

[0011] The acoustic control method according to the first embodiment is executed, for example, by an acoustic device 1 shown in Fig. 1A. Fig. 1A shows a case in which, in a space inside a vehicle such as the cabin of a vehicle, a direct sound, which is a sound source signal, and an actual reverberation sound are output from two speakers FR, FL arranged on the left and right at the front, and a pseudo reverberation sound is output from two speakers RL, RR arranged on the left and right at the rear and added to the direct sound, thereby performing surround reproduction.

[0012] Here, the sound source signal is a signal of a sound source having a spatial spread (sound image width) by outputting different sounds from two speakers FR and FL, for example. In other words, the sound source signal is a stereo signal that is reproduced in stereo on two channels (speakers FR and FL).

[0013] The sound source signal is, for example, a signal of a mixture of sounds from multiple musical instruments and voices (vocals) such as classical music and opera, that is, a signal of a mixture of sounds from multiple sound sources, but is not limited to this. That is, the sound source signal may be a signal of a sound only from a voice, or may be a signal of a sound from a single instrument (sound source) such as only from a piano or only from a violin.

[0014] Here, as shown in FIG. 1A, it is known that a listener L who listens to sound from a sound source S, which is a position where a sound source such as a musical instrument or voice (vocals) is located in space, can perceive two types of spatial impressions. One spatial impression is the sound image width ASW, which is defined as the "width of the apparent sound source" perceived as blended with the direct sound in both time and space, and the other spatial impression is the sense of envelopment LEV, which is defined as the "feeling that the listener is surrounded by sound sources other than the apparent sound source." The sound image width ASW is the width of the sound image A derived from the direct sound and early reflected sound components, which are the early components of the sound source signal. The sense of envelopment LEV is the sound image B derived from the reverberation sound components, which are the later components of the sound source signal.

[0015] In designing and evaluating the sound image width ASW and the sense of envelopment LEV, an index using the so-called "law of the first wave front" is sometimes used. In this index, two regions R1 and R2 are defined by two thresholds TH1 and TH2, as shown in Figure 1B.

[0016] Region R1 is a region that contains components (initial components) that mainly contain direct sound among the components contained in the sound source signal. For example, if the initial components of region R1 are large, the sound image width ASW becomes large, and the listener L feels that the sound is diffused, and depending on the sound source, the sound quality is evaluated as poor (unclear). Note that direct sound is, for example, a sound recorded directly from a voice (vocal) or an instrument, and does not include sound reflected by a wall, etc.

[0017] Moreover, region R2 is a region that mainly contains components (late components) including reverberation sounds among the components included in the sound source signal. For example, if the reverberation sound components in region R2 are large, the envelopment feeling LEV becomes large, and the listener L feels that the sound is diffused, and the envelopment feeling is evaluated as being full. Note that reverberation sounds are, for example, recorded sounds of voices, musical instruments, etc., reflected by walls, etc., and are sounds that are delayed in time from the direct sound.

[0018] In a relatively small space such as a vehicle interior, it is difficult to separate direct sound from reverberant sound, and reverberant sound is likely to be mixed into the direct sound. In such a case, the reverberant sound components in area R2 become smaller, and the envelopment LEV becomes smaller. Therefore, in a small space such as a vehicle interior where direct sound and reverberant sound are mixed, the acoustic device 1 actively adds pseudo-reverberant sound to the direct sound, thereby increasing the reverberant sound components in area R2 and ensuring the envelopment LEV.

[0019] In the above description, it has been described that the direct sound and the like are output from the speakers FR and FL, and the pseudo reverberation sound is output from the speakers RL and RR, but this is not limiting. That is, for example, the acoustic device 1 may output the direct sound and the like from all or some of the four speakers FR, FL, RL, and RR, and adjust the volume and the like. This allows, for example, the acoustic device 1 to move the spatial position of the sound image A of the direct sound and the like. Similarly, the acoustic device 1 may output the pseudo reverberation sound from all or some of the four speakers FR, FL, RL, and RR, and adjust the volume and the like. This allows, for example, the acoustic device 1 to move the spatial position of the sound image B of the pseudo reverberation sound. In this way, the acoustic device 1 can localize the sound image A of the direct sound and the sound image B of the pseudo reverberation sound at any position.

[0020] In the prior art, a process for adding pseudo reverberation may be performed by extracting surround components including early reflection sound components from a sound source signal based on the correlation between the left and right channels of the sound source, and delaying and outputting the extracted surround components. However, in such a case, it is not always possible to extract the intended surround components from various types of sound source signals. In addition, when the surround components are delayed and output as described above, the early reflection sound that was originally present may not be obtained, and as a result, appropriate surround sound reproduction may not be possible.

[0021] Therefore, the acoustic device 1 according to this embodiment is configured to be able to perform appropriate surround reproduction according to the sound source signal.

[0022] The processing of the acoustic device 1 will be specifically described below with reference to Fig. 1A. First, the acoustic device 1 separates a predetermined sound source signal that does not require processing to add artificial reverberation from the sound source signal of the sound source device 50, and removes the separated predetermined sound source signal from the sound source signal (step S1). A detailed method for separating the predetermined sound source signal will be described later. The predetermined sound source signal here includes, for example, a voice (vocal) component.

[0023] Next, the acoustic device 1 applies a filter for generating an artificial reverberation sound to the sound source signal from which the predetermined sound source signal has been removed (hereinafter, may be referred to as the "removed sound source signal") to generate a reverberation signal indicating the artificial reverberation sound and output it from the speakers RL and RR (step S2). Note that, as the filter, for example, an impulse response filter such as a FIR (Finite Impulse Response) filter or an IIR (Infinite Impulse Response) filter can be used, but is not limited thereto.

[0024] In this manner, in this embodiment, a filter is applied only to a sound source signal from which a predetermined sound source signal (e.g., a voice component) that does not require the addition of a pseudo reverberation sound is removed, and a reverberation signal is generated. In other words, a filter is not applied to a predetermined sound source signal among the sound source signals, and a reverberation signal is not generated.

[0025] As a result, even if the sound source signal includes a sound source signal that requires processing to add artificial reverberation and a sound source signal that does not (here, a specified sound source signal), a reverberation signal is generated only for the sound source signal that requires the processing, so that appropriate surround playback can be performed according to the sound source signal (more specifically, according to the content (type) of the sound source signal).

[0026] The above-mentioned filter is not applied to the audio components contained in the predetermined sound source signal, and the audio components are reproduced as direct sound. Therefore, the listener L does not feel the envelopment sense LEV in the reproduced predetermined sound source signal (sound in this case). This is because the listener L can hear the reproduced sound more clearly if there is no envelopment sense LEV for the sound (vocals) and the like.

[0027] In the above, an example in which a filter is applied only to the sound source signal from which the predetermined sound source signal has been removed has been shown, but the present invention is not limited to this. That is, for example, the acoustic device 1 may apply a filter to both the sound source signal from which the predetermined sound source signal has been removed and the predetermined sound source signal. In this case, for the sound source signal from which the predetermined sound source signal has been removed, a reverberation signal may be generated such that the reverberation level of the corresponding artificial reverberation sound is relatively large, while for the predetermined sound source signal, a reverberation signal may be generated such that the reverberation level of the corresponding artificial reverberation sound is relatively small. The above-mentioned "reverberation level" is an index value indicating the degree of reverberation of the artificial reverberation sound in a room (here, a vehicle interior) when the reverberation signal is reproduced from the speakers RL and RR, for example.

[0028] In the above description, the predetermined sound source signal includes a voice component, but is not limited to this. That is, the predetermined sound source signal may include a component of a sound that reverberates too much and sounds unnatural when the surround feeling LEV is increased during surround playback, specifically, a transient sound component such as a sound of a percussion instrument such as a drum, in which there is silence between striking points.

[0029] <Configuration of an audio system including an audio device according to the first embodiment> Next, the configuration of an audio system including the audio device 1 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of an audio system including the audio device 1 according to the first embodiment. In Fig. 2, only components necessary for explaining the features of this embodiment are shown as functional blocks, and descriptions of general components are omitted.

[0030] In other words, each component shown in Fig. 2 is a functional concept, and does not necessarily have to be physically configured as shown in the figure. For example, the specific form of distribution and integration of each functional block is not limited to that shown in the figure, and all or part of it can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0031] 2, the acoustic system 100 includes an acoustic device 1, a sound source device 50, various sensors 60, and a plurality of speakers FL, FR, RL, and RR. The acoustic system 100 according to this embodiment is mounted on a vehicle, but is not limited to this.

[0032] The sound source device 50 outputs a sound source signal to the acoustic device 1. The sound source signal is, for example, a stereo signal. The sound source signal becomes a spatially expanding sound image by outputting different signals from two speakers FL and FR, which are two channels, via the acoustic device 1.

[0033] The various sensors 60 include various sensors that detect the state of the vehicle. The various sensors 60 include sensors that can detect, for example, the running state of the vehicle, such as the vehicle speed, the open / closed state of the windows, the operating state of the air conditioner, the seating state of the occupants, fader adjustment instructions for the speakers FL, FR, RL, and RR (front / rear balance adjustment instructions by the occupants), whether or not the vehicle is being driven automatically, and output information indicating the detected state of the vehicle to the audio device 1. Note that, although the running state and the open / closed state of the windows are specifically shown as the state of the vehicle in the above description, these are merely examples and are not limiting.

[0034] A plurality of speakers FL, FR, RL, and RR are connected to an acoustic device 1. These speakers FL, FR, RL, and RR output, as sounds, signals output from the acoustic device 1. For example, the speakers FL and FR output direct sounds that are sound source signals, and the speakers RL and RR output pseudo reverberation sounds generated from the sound source signals, but are not limited to this.

[0035] The audio device 1 includes a control unit 2 and a storage unit 3. The control unit 2 includes an acquisition unit 21, a separation unit 22, and an output control unit 23. The audio device 1 includes a computer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a flash memory, an input / output port, and various other circuits.

[0036] The CPU of the computer functions as the acquisition unit 21, separation unit 22, and output control unit 23 of the control unit 2, for example, by reading and executing a program stored in the ROM.

[0037] At least one or all of the acquisition unit 21, separation unit 22 and output control unit 23 of the control unit 2 can be configured with hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0038] The storage unit 3 corresponds to a RAM or a flash memory. The RAM or the flash memory can store information on various programs, etc. The audio device 1 may obtain the above-mentioned programs and various information via another computer or a portable recording medium connected to a wired or wireless network.

[0039] The acquisition unit 21 acquires various information and signals. For example, the acquisition unit 21 acquires a sound source signal from the sound source device 50. For example, the acquisition unit 21 acquires a sound source signal that is a stereo signal. Specifically, the acquisition unit 21 acquires sound source signals output from two speakers FL and FR, which are two channels. The acquisition unit 21 outputs the acquired sound source signals to the separation unit 22 and the output control unit 23.

[0040] In the following, among the sound source signals that are stereo signals, the sound source signal that is output (played) from the speaker FL, which is the left channel, may be referred to as the "Lch sound source signal," and the sound source signal that is output from the speaker FR, which is the right channel, may be referred to as the "Rch sound source signal."

[0041] The acquisition unit 21 acquires information indicating the state of the vehicle (for example, the running state such as the vehicle speed, and the open / closed state of the windows) output from the various sensors 60, and outputs the acquired information to the output control unit .

[0042] The separation unit 22 performs a process of separating and extracting various sound source signals, including a predetermined sound source signal that does not require processing to add artificial reverberation, from the sound source signal. For example, the separation unit 22 can separate and extract an L component sound source signal, an R component sound source signal, a reverberation sound component sound source signal, and the like, in addition to the above-mentioned predetermined sound source signal.

[0043] The L component sound source signal is a sound source signal including a component (L component) of a sound reproduced on one channel (speaker FL) of the two channels, speakers FL and FR. The R component sound source signal is a sound source signal including a component (R component) of a sound reproduced on the other channel (speaker FR). The L component sound source signal is an example of a first sound source signal, and the R component sound source signal is an example of a second sound source signal. The reverberation sound component sound source signal is a sound source signal including actual reverberation sound components such as early reflection sounds.

[0044] In the above description, the separation unit 22 separates and extracts a predetermined sound source signal, an L component sound source signal, an R component sound source signal, and a reverberation component sound source signal from the sound source signal, but this is not limiting and may be configured to separate and extract, for example, a portion of these various sound source signals. In addition, since the predetermined sound source signal includes a voice component as described above, hereinafter, the predetermined sound source signal may be referred to as a "voice sound source signal".

[0045] Here, the separation / extraction process by the separation unit 22 will be described with reference to Fig. 3. Fig. 3 is a diagram illustrating the separation / extraction process by the separation unit 22. As shown in Fig. 3, first, an Lch sound source signal and an Rch sound source signal are input from the acquisition unit 21 to the separation unit 22.

[0046] Next, the separation unit 22 performs time-frequency analysis on the Lch sound source signal and the Rch sound source signal, respectively, to calculate Lch sound source data and Rch sound source data. For example, the separation unit 22 performs a short-time Fourier transform on the Lch sound source signal to convert it from the time domain to the time-frequency domain, and calculates the Lch sound source data. Similarly, the separation unit 22 performs a short-time Fourier transform on the Rch sound source signal to calculate the Rch sound source data.

[0047] The Lch sound source data and the Rch sound source data indicate frequency characteristics of the corresponding sound signal with respect to time. In detail, the Lch sound source data and the Rch sound source data are data in which the sound signal is divided into predetermined frequency bands and elapsed times, and a relative decibel value (not shown) is set for each divided region.

[0048] In this way, the separation unit 22 can accurately calculate the Lch sound source data and the Rch sound source data by performing time-frequency analysis on the sound source signal. In this embodiment, by using the accurately calculated sound source data, it is possible to perform appropriate surround reproduction according to the sound source signal.

[0049] Next, the separator 22 calculates difference information between the Lch sound source data and the Rch sound source data. The difference information is, for example, information indicating an LR channel difference obtained by subtracting the Lch sound source data from the Rch sound source data, and is specifically level difference information and phase difference information. In more detail, the level (amplitude) difference information is an inter-channel level difference (ICLD (Inter-channel Level Difference)), and the phase difference information is an inter-channel phase difference (ICPD (Inter-channel Phase Difference)).

[0050] In this way, the separation unit 22 calculates differential information indicating the difference between the sound source signals reproduced by each of the two channels (speakers FL and FR), and by using this differential information, it becomes possible to accurately separate audio sound source signals, etc., as described below.

[0051] Next, the separation unit 22 generates masks for separating and extracting various sound source signals based on the difference information. In detail, the separation unit 22 generates masks for separating and extracting a voice sound source signal, an L component sound source signal, an R component sound source signal, a reverberation sound component sound source signal, etc. based on the difference information. Note that, as the mask, for example, a binary mask taking only two values, 0 or 1, can be used, but is not limited to this.

[0052] In the following, a mask for separating and extracting a speech sound source signal may be referred to as a "speech mask." Also, masks for separating and extracting an L component sound source signal, an R component sound source signal, and a reverberation sound source signal may be referred to as an "L component mask," an "R component mask," and a "reverberation sound mask," respectively.

[0053] First, the generation of the “audio mask” will be described. Note that, since audio (vocals) is usually recorded in monaural, it is assumed here that the Lch audio source signal and the Rch audio source signal contain the same audio audio source signal.

[0054] The separator 22 sets the value of the audio mask (more precisely, the value in each region) based on ICLD, which is level difference information, and ICPD, which is phase difference information, among the difference information, and the following setting condition 1, to generate the audio mask.

[0055] [Setting condition 1] Mask value 1: |ICLD|< threshold a and |ICPD|< threshold b Mask value 0: Area other than the above Here, the threshold a and the threshold b are both set to relatively small values. More specifically, the threshold a and the threshold b are set to values ​​that allow the estimation that both ICLD and ICPD are 0 or close to 0, but are not limited thereto and can be set to any value.

[0056] As described above, since the Lch sound source signal and the Rch sound source signal contain the same audio sound source signal, the audio mask is set to 1 in the area containing the audio sound source signal (in other words, the area where ICLD and ICPD are both 0 or close to 0) and to 0 in the other areas according to setting condition 1.

[0057] Next, the separation unit 22 separates and extracts the speech source signal by using the generated speech mask. Specifically, the separation unit 22 separates and extracts the speech source signal from the time-frequency domain in which the value of the speech mask is set to 1.

[0058] For example, the separation unit 22 applies a voice mask to the Lch sound source data and the Rch sound source data and filters them to separate the voice sound source signals from the Lch sound source data and the Rch sound source data. Then, the separation unit 22 averages the two separated voice sound source signals and extracts the averaged signal as the voice sound source signal.

[0059] In the above, an example has been shown in which the audio mask is applied to both the Lch audio source data and the Rch audio source data, but this is not limited to this. For example, the audio mask may be applied to one of the Lch audio source data and the Rch audio source data, and the audio audio source signal may be separated and extracted from one of the data.

[0060] In the above description, the values ​​of the voice mask in the regions other than the region where ICLD and ICPD are 0 or near 0 are set to 0. However, in addition to or instead of this, for example, values ​​of a frequency region other than the voice band (for example, 300 to 3400 Hz) may be set to 0 in advance. That is, here, a process of separating and extracting a voice source signal is performed. For this reason, as described above, by setting values ​​of a frequency region other than the voice band, in other words a frequency region where a voice source signal cannot exist, to 0 in advance, it is possible to prevent erroneous detection that a voice source signal exists in a frequency region where a voice source signal cannot exist, and therefore to separate and extract a voice source signal with high accuracy.

[0061] Also, in the voice mask, for example, the value of the frequency range below the voice band may be set in advance to 1. This makes it possible to separate and extract the sound source signal of an instrument (such as a bass drum) that includes a monaural component and has a frequency below the voice band.

[0062] Next, we will explain how to generate an "L component mask." For example, there may be cases where the components of a certain musical instrument (for example, a piano) are contained more in the Lch sound source signal than in the Rch sound source signal. The L component mask is a mask for separating and extracting such musical instrument components as the L component sound source signal.

[0063] Specifically, the separation unit 22 sets the value of the L component mask (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the following setting condition 2, and generates the L component mask.

[0064] [Setting condition 2] Mask value 1: Area where ICLD<threshold c and ICPD<threshold d Mask value 0: Area other than the above Here, the threshold c and the threshold d are both set to values ​​equal to or less than 0. In detail, the threshold c and the threshold d are set to values ​​that allow for estimation that both ICLD and ICPD are negative values, but are not limited thereto and can be set to any values.

[0065] That is, since the Lch sound source signal contains many components of a certain instrument sound (piano in this case) as described above, in the difference information obtained by subtracting the Lch sound source data from the Rch sound source data, ICLD and ICPD in the area containing the components of the certain instrument sound are negative values. Therefore, according to the setting condition 2, the L component mask is set to 1 in the area containing the components of a certain instrument sound (piano in this case) (in other words, the area where ICLD and ICPD are both negative values) and 0 in the other areas.

[0066] Next, the separation unit 22 separates and extracts the L component excitation signal by using the generated L component mask. Specifically, the separation unit 22 separates and extracts the L component excitation signal from the time-frequency domain in which the value of the L component mask is set to 1. For example, the separation unit 22 applies the L component mask to the Lch excitation data and filters it, thereby separating the L component excitation signal from the Lch excitation data.

[0067] Next, we will explain how to generate the "R component mask". For example, there may be cases where the Rch sound source signal contains more components of a certain instrument sound (e.g., a guitar) than the Lch sound source signal. The R component mask is a mask for separating and extracting such instrument sound components as the R component sound source signal.

[0068] Specifically, the separation unit 22 sets the value of the R component mask (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the following setting condition 3, and generates the R component mask.

[0069] [Setting condition 3] Mask value 1: Area where ICLD>threshold e and ICPD>threshold f Mask value 0: Area other than the above Here, the threshold value e and the threshold value f are both set to values ​​equal to or greater than 0. In detail, the threshold value e and the threshold value f are set to values ​​that allow for estimation that both ICLD and ICPD are positive values, but are not limited thereto and can be set to any values.

[0070] That is, as described above, the Rch sound source signal contains many components of a certain instrument sound (here, guitar), so in the difference information obtained by subtracting the Lch sound source data from the Rch sound source data, ICLD and ICPD in the area containing the components of the certain instrument sound are positive values. Therefore, according to setting condition 3, the R component mask is set to 1 in the area containing the components of a certain instrument sound (here, guitar) (in other words, the area where ICLD and ICPD are both positive values), and to 0 in the other areas.

[0071] Next, the separation unit 22 separates and extracts the R component sound source signal by using the generated R component mask. Specifically, the separation unit 22 separates and extracts the R component sound source signal from the time-frequency domain in which the value of the R component mask is set to 1. For example, the separation unit 22 applies the R component mask to the Rch sound source data and filters it, thereby separating the R component sound source signal from the Rch sound source data.

[0072] Next, we will explain the "reverberation mask". For example, the Lch sound source signal and the Rch sound source signal may contain actual reverberation components such as early reflection sounds in addition to the monaural components such as the above-mentioned voice, L components, and R components. Some of these reverberation components tend to make the sound image unclear when reproduced, and the reverberation mask is a mask for separating and extracting such reverberation components as reverberation component sound source signals.

[0073] Specifically, the separation unit 22 sets the value of the reverberation mask (more precisely, the value in each region) based on the level difference information ICLD, the phase difference information ICPD, and the following setting condition 4, and generates the reverberation mask.

[0074] [Setting condition 4] Mask value 0: |ICLD|< threshold a and |ICPD|< threshold b Mask value 0: Area where ICLD<threshold c and ICPD<threshold d Mask value 0: Area where ICLD>threshold e and ICPD>threshold f Mask value 1: Area other than the above As can be seen from the contents of setting condition 4, the areas where the mask value is set to 1 in the above-mentioned setting conditions 1 to 3 are set to 0 in setting condition 4, and the values ​​of other areas are set to 1. That is, for example, areas where ICLD and ICPD have inverted positive and negative values ​​or areas where ICPD is approximately 180° are areas that contain reverberation that tends to make the sound image unclear when played back, so the value of the relevant area is set to 1.

[0075] Next, the separation unit 22 separates and extracts the reverberation sound component sound source signal by using the generated reverberation sound mask. Specifically, the separation unit 22 separates and extracts the reverberation sound component sound source signal from the time-frequency domain where the value of the reverberation sound mask is set to 1.

[0076] For example, the separation unit 22 applies a reverberation mask to the Lch sound source data and the Rch sound source data and filters them to separate the reverberation component sound source signals from the Lch sound source data and the Rch sound source data. Then, the separation unit 22 averages the two separated reverberation component sound source signals and extracts the averaged signal as the reverberation component sound source signal.

[0077] In the above, an example has been shown in which the reverberation mask is applied to both the Lch sound source data and the Rch sound source data, but the present invention is not limited to this. For example, the reverberation mask may be applied to one of the Lch sound source data and the Rch sound source data, and the reverberation sound component sound source signal may be separated and extracted from one of the data.

[0078] In the above, since the value of each mask is set to 1 or 0, separation distortion may occur in the separation process of each sound source signal, but in such a case, a low-pass filter may be combined with each mask to reduce the separation distortion. Also, a relaxed value such as 0.1 may be set for the area where the value of each mask is set to 0 to reduce the separation distortion.

[0079] 2, the separation unit 22 outputs the separated and extracted voice sound source signal, L component sound source signal, R component sound source signal and reverberation component sound source signal to the output control unit 23. In addition, the separation unit 22 removes the voice sound source signal (predetermined sound source signal) from the sound source signal to generate a removed sound source signal, and outputs the generated removed sound source signal to the output control unit 23.

[0080] The output control unit 23 performs predetermined processing on the sound source signal input from the acquisition unit 21, and the voice sound source signal, the L component sound source signal, the R component sound source signal, the reverberation component sound source signal and the removed sound source signal input from the separation unit 22, and outputs the results from the speakers FL, FR, RL and RR.

[0081] For example, the output control unit 23 performs D / A conversion on a sound source signal including a direct sound, etc., amplifies the D / A converted sound source signal, and outputs (plays) it from the speakers FL, FR. Here, the sound image of the played sound, etc. will be described with reference to Figs. 4A to 4G.

[0082] 4A to 4G are diagrams for explaining the sound image of the sound reproduced by the output control unit 23. FIG. 4A shows a state where only the sound source signal including the direct sound etc. is output and reproduced, in other words, a state where the pseudo reverberation sound etc. is not reproduced (added). Also, in FIGS. 4A to 4F, the sound image of the voice (vocals) is indicated by the symbol A1. Also, the sound source signal is a stereo signal as described above, and includes an Lch sound source signal and an Rch sound source signal. In FIGS. 4A to 4F, the sound image of the instrument sound etc. contained more in the Lch sound source signal than in the Rch sound source signal is indicated by the symbol AL, and the sound image of the instrument sound etc. contained more in the Rch sound source signal than in the Lch sound source signal is indicated by the symbol AR.

[0083] As shown in FIG. 4A, when the output control unit 23 outputs a sound source signal including a direct sound etc. from the speakers FL and FR, a sound image A1 of a voice and sound images AL and AR of a musical instrument sound are localized at a relatively narrow interval between the speakers FL and FR.

[0084] 2, the output control unit 23 has a filter 23a, and generates and outputs a pseudo reverberation sound using the filter 23a. The filter 23a is a filter for generating the pseudo reverberation sound. Note that, as the filter, for example, an FIR filter or an IIR filter can be used, but is not limited thereto.

[0085] For example, the output control unit 23 applies a filter 23a to the sound source signal from which a predetermined sound source signal has been removed, i.e., the removed sound source signal, to generate a reverberation signal that represents a pseudo-reverberation sound. The output control unit 23 then performs D / A conversion on the reverberation signal, amplifies the D / A converted reverberation signal, and outputs it from the speakers RL and RR. In this way, the pseudo-reverberation sound is added to the direct sound, etc.

[0086] Therefore, as shown in FIG. 4B, for example, the output control unit 23 adjusts the volume of the artificial reverberation sound output from the speakers RL and RR, and positions the sound image B of the artificial reverberation sound around the listener L, thereby allowing the listener L to feel a sufficient sense of being enveloped LEV.

[0087] In this way, the output control unit 23 according to this embodiment applies the filter 23a only to the sound source signal (removed sound source signal) from which a voice sound source signal that does not require processing to add artificial reverberation has been removed, and generates and outputs a reverberation signal.

[0088] As a result, even if a sound source signal includes a sound source signal that requires processing to add artificial reverberation and a sound source signal that does not (here, an audio sound source signal), a reverberation signal is generated and output only for the sound source signal that requires the processing, so that appropriate surround playback can be performed according to the sound source signal (more specifically, according to the content (type) of the sound source signal).

[0089] In the above, the output control unit 23 applies the filter 23a only to the sound source signal from which the voice sound source signal has been removed (removed sound source signal), but the present invention is not limited to this. That is, for example, the output control unit 23 may apply the filter 23a to both the sound source signal from which the voice sound source signal (predetermined sound source signal) has been removed and the voice sound source signal. At this time, the output control unit 23 outputs the sound source signal from which the voice sound source signal has been removed (removed sound source signal) so that the reverberation level of the artificial reverberation sound corresponding to the sound source signal from which the voice sound source signal has been removed is higher than the reverberation level of the artificial reverberation sound corresponding to the voice sound source signal. In other words, the output control unit 23 outputs the artificial reverberation sound corresponding to the voice sound source signal so that the reverberation level of the artificial reverberation sound corresponding to the removed sound source signal is lower than the reverberation level of the artificial reverberation sound corresponding to the removed sound source signal. In other words, the filter 23a is set to generate a reverberation signal having a reverberation level of the above-mentioned artificial reverberation sound.

[0090] This makes it possible to prevent the listener L from getting an unnatural impression that the sound image and tone of the voice (vocals) are extremely clear and appear to be in front. In detail, for example, when a large reverberation is applied only to the sound source signal of the instrumental sound (i.e., the removed sound source signal) for a music sound source signal containing a mixture of vocals and instrumental sounds, and no reverberation is applied to the voice sound source signal of the vocals, the listener L may get an unnatural impression that the sound image and tone of only the vocals are extremely clear and appear to be in front. This is because there is a psychological characteristic in which a sound image (here, the sound image of the instrument) feels far away when there is a large reverberation and a large amount of reflected sound, and accordingly, a sound image (here, the sound image of the vocals) without any reverberation appears to be closer to the listener. Therefore, by applying the filter 23a to both the sound source signal from which the voice sound source signal has been removed and the voice sound source signal, and by making the reverberation levels different as described above, it is possible to prevent the listener L from getting the unnatural impression described above.

[0091] The output control unit 23 may select the speaker that outputs the removed sound source signal to be played back in surround from among the speakers FL, FR, RL, and RR according to the content (type) of the removed sound source signal. As an example, the output control unit 23 may improve the sense of envelopment LEV by selecting the speaker that outputs the sound source signal and the reverberation signal so that the direct sound and the artificial reverberation sound are played back on either side of the listener L.

[0092] More specifically, the reverberation signal (in other words, the removed sound source signal to which the filter 23a is applied and which is reproduced in surround) includes the above-mentioned L component sound source signal and R component sound source signal, and the L component sound source signal and the R component sound source signal are separated and extracted by the separation unit 22. Therefore, the output control unit 23 reproduces (outputs) the L component sound source signal from the reverberation signal (the removed sound source signal to which the filter 23a is applied) in surround from the speaker RR, and reproduces the R component sound source signal from the speaker RL.

[0093] As a result, the listener L is positioned between the speaker FL, which reproduces the L component of the direct sound, and the speaker RR, which reproduces the L component of the artificial reverberation sound in surround sound, and between the speaker FR, which reproduces the R component of the direct sound, and the speaker RL, which reproduces the R component of the artificial reverberation sound in surround sound. Therefore, the sound image B of the artificial reverberation sound is easily localized around the listener L, and as a result, the sense of being surrounded by the listener LEV can be improved.

[0094] Furthermore, the output control unit 23 applies a filter 23a to the separated reverberation component sound source signal to generate a reverberation signal. The output control unit 23 may then output the reverberation signal generated from the reverberation component sound source signal from the speakers RL and RR. This causes a pseudo-reverberation sound due to the reverberation component sound source signal to be added to the direct sound, etc. In detail, for example, a pseudo-reverberation sound of a reverberation component that is likely to have an unclear sound image during playback among the actual reverberation components is added to the direct sound, thereby clarifying the sound image of the reverberation component and further improving the sense of envelopment LEV.

[0095] Incidentally, for example, in a vehicle cabin, there may be a case where another listener Lx is seated at the rear of the cabin (for example, in the back seat) as shown by an imaginary line in Fig. 4B. In this case, if the above-mentioned artificial reverberation sound adding process is performed, there is a risk that the reverberation sound including the artificial reverberation sound may become excessive for the listener Lx.

[0096] Therefore, the output control unit 23 according to the present embodiment may suppress the reverberation sound from becoming excessive for the listener Lx by performing a process of adding a pseudo reverberation sound according to the state of the vehicle. In detail, the output control unit 23 may suppress the reverberation sound from becoming excessive by applying a filter 23a to the removed sound source signal according to the state of the vehicle and outputting the removed sound source signal (i.e., the reverberation signal) to which the filter 23a has been applied from a channel (e.g., speakers RL, RR, etc.).

[0097] For example, when the vehicle is in a state in which the speaker Rx is located behind the listener Lx in the rear seat as shown by the imaginary line in Fig. 4B, the output control unit 23 outputs the removed sound source signal (reverberation signal) to which the filter 23a has been applied from the speaker Rx. This makes it possible to localize the sound image of the pseudo reverberation sound around the listener Lx, although this is not shown in the figure. Therefore, the listener Lx can also ensure the sense of envelopment LEV as with the listener L, and the reverberation sound can be prevented from becoming excessive.

[0098] Also, for example, the output control unit 23 may perform a process of adding a pseudo reverberation sound in response to a fader adjustment instruction (one example of a vehicle state) for the speakers FL, FR, RL, and RR. For example, when a fader adjustment instruction indicating that playback should be prioritized for the rear seats is detected from the various sensors 60 as a vehicle state, the output control unit 23 may weaken the reverberation signal output from the speakers RL and RR or prohibit the output of the reverberation signal. This reduces or eliminates the pseudo reverberation sound, making it possible to prevent the reverberation sound from becoming excessive for the listener Lx.

[0099] Also, for example, the output control unit 23 may perform a process of adding a pseudo reverberation sound depending on the seating state of the occupants (one example of the state of the vehicle). For example, when the various sensors 60 detect a seating state of the occupants indicating that no occupants are seated in the rear seats, that is, that no listener Lx is present, as the state of the vehicle, the output control unit 23 may output a reverberation signal from the speakers RL and RR. This prevents the above-mentioned excessive reverberation sound from occurring.

[0100] Also, for example, the output control unit 23 may perform a process of adding a pseudo reverberation sound in response to at least one of the vehicle conditions, such as the open / closed state of the vehicle window, the operating state of the air conditioner, and the vehicle speed, which are examples of the vehicle conditions. For example, when the rear seat window is open, the air conditioner is operating at a relatively strong wind, or the vehicle is running at a relatively high speed, a relatively loud noise is generated in the rear seat. Therefore, it is estimated that the listener Lx in the rear seat is in an environment where the above-mentioned reverberation sound is difficult to be felt. Therefore, the output control unit 23 may output a reverberation signal from the speakers RL and RR when the above-mentioned open window, strong wind operation of the air conditioner, high speed driving, etc. are detected as the vehicle conditions by the various sensors 60. As a result, since the listener Lx is in an environment where the listener Lx is difficult to be felt, it is possible to prevent the reverberation sound from becoming excessive for the listener Lx.

[0101] In the above, the reverberation signals are output from the speakers RL and RR arranged behind the listener L, but this is not limited to this and the reverberation signals may be output from other speakers, such as a speaker (not shown) arranged above the listener L.

[0102] In addition, the output control unit 23 can perform localization control of the sound image. Specifically, the output control unit 23 can output the audio source signal so that the sound image A1 of the sound generated by the separated audio source signal is localized at a predetermined position. For example, as shown in FIG. 4C, the output control unit 23 can localize the sound image A1 of the sound (vocals) generated by the audio source signal at a position displaced toward the listener L by emphasizing and outputting the separated audio source signal from the speakers RL and RR. This makes it possible to make it difficult for the sound image A1 of the sound and the sound images AL and AR of the musical instrument sound to overlap, and thus makes it possible to clarify the sounds corresponding to the respective sound images A1, AL, and AR. Note that, in this specification, "emphasis" means, for example, adding and reproducing new sounds corresponding to various signals, or increasing the volume of the reproduced sound, but is not limited thereto.

[0103] In the above description, the predetermined position is a position displaced toward the listener L, but is not limited to this and can be set to any position. For example, although not shown, a speaker may be disposed above the listener L, and an audio source signal may be emphasized and output from the speaker, so that the sound image A1 of the audio is localized at a position displaced upward. As a result, the sound image A1 and the sound images AL and AR are shifted in the height direction, so that, for example, the three-dimensional effect of the sound is increased, improving the sense of realism.

[0104] Furthermore, the output control unit 23 may emphasize and output the separated audio source signal from, for example, the speakers FL and FR. This adds the sound of the emphasized audio source signal to the sound (vocals) included in the direct sound, thereby making it possible to make the sound image A1 of the sound clearer.

[0105] Furthermore, the output control unit 23 can control the sound image width of the sound image. Specifically, as shown in Fig. 4D, the output control unit 23 may output the separated L component sound source signal from one speaker (channel) FL and control the sound image width of the sound due to the L component sound source signal. For example, the output control unit 23 emphasizes and outputs the separated L component sound source signal from the speaker FL. This adds the instrument sound of the emphasized L component sound source signal to an instrument sound (e.g., piano) which is an example of the L component included in the direct sound, and thus makes it possible to expand the sound image width of the sound image AL of the instrument sound to the left.

[0106] Similarly, the output control unit 23 may output the separated R component sound source signal from the other speaker (channel) FR and control the sound image width of the sound by the R component sound source signal. For example, the output control unit 23 emphasizes and outputs the separated R component sound source signal from the speaker FR. This adds the instrument sound of the emphasized R component sound source signal to an instrument sound (e.g., a guitar) that is an example of the R component included in the direct sound, thereby making it possible to expand the sound image width of the sound image AR of the instrument sound to the right.

[0107] 4D, the output control unit 23 may output the separated L component sound source signal from the speaker RL and control the sound image width of the sound image ALx of the sound (here, the instrument sound) due to the L component sound source signal to be expanded backward. Similarly, the output control unit 23 may output the separated R component sound source signal from the speaker RR and control the sound image width of the sound image ARx of the sound (here, the instrument sound) due to the R component sound source signal to be expanded backward. This makes it possible to localize the sound images ALx and ARx at a position that envelops the listener L.

[0108] Here, for example, noises such as the sound of the car running or the sound of an air conditioner (driving noise) are generated inside the car. In the example of FIG. 4E, the sound image of the noise is indicated by the symbol C. The sound image C of the noise may be localized near the listener L. In this case, the sound image C of the noise may be close to or overlap with the sound image A1 of the voice or the sound images AL and AR of the musical instrument sound, so that the noise may be mixed with the voice or music, and the sound images A1, AL, and AR may become unclear.

[0109] Therefore, for example, the output control unit 23 outputs the separated audio source signal from the speakers FL and FR with emphasis, or attenuates or prohibits the output of the audio source signal that was emphasized and output from the speakers RL and RR, thereby localizing the audio image A1 of the audio source signal at a position displaced in a direction (for example, forward) away from the noise image C (see arrow D1). This makes it possible, for example, to make it difficult for the audio image A1 and the noise image C to overlap, in other words, to make it difficult for the noise to mix with the audio, and to clarify the audio image A1.

[0110] Also, for example, the output control unit 23 emphasizes the separated L component sound source signal and outputs it from the speaker FL, or weakens or prohibits the output of the L component sound source signal that was emphasized and output from the speaker RL. As a result, the output control unit 23 localizes the sound image AL of the instrument sound of the L component sound source signal at a position displaced in a direction away from the sound image C of the noise (for example, forward) (see arrow D2a), or expands the sound image width of the sound image AL to the left (see arrow D2b). As a result, for example, it becomes possible to make it difficult for the sound image AL of the instrument sound and the sound image C of the noise to overlap, in other words, it becomes possible to make it difficult for the instrument sound to be mixed with the noise, and it becomes possible to clarify the sound image AL of the instrument sound.

[0111] Similarly, for example, the output control unit 23 emphasizes the separated R component sound source signal and outputs it from the speaker FR, or weakens or prohibits the output of the R component sound source signal that was emphasized and output from the speaker RR. As a result, the output control unit 23 localizes the sound image AR of the instrument sound of the R component sound source signal at a position displaced in a direction away from the sound image C of the noise (see arrow D3a), or expands the sound image width of the sound image AR to the right (see arrow D3b). As a result, for example, it becomes possible to make it difficult for the sound image AR of the instrument sound and the sound image C of the noise to overlap, in other words, it becomes possible to make it difficult for the instrument sound to be mixed with the noise, and the sound image AR of the instrument sound can be made clear.

[0112] The output control unit 23 may control the localization and width of the sound images A1, AL, and AR according to the position of the sound image C of the noise. That is, for example, the position of the sound image C of the noise changes according to the driving state such as the vehicle speed, the open / closed state of the windows, the operating state of the air conditioner, and the like. Therefore, for example, the correlation between the driving state, the open / closed state of the windows, the operating state of the air conditioner, and the position of the sound image C of the noise may be calculated in advance through an experiment or the like, and the output control unit 23 may control the localization and width of the sound images A1, AL, and AR based on information indicating the calculated correlation. The position of the sound image C of the noise may be detected by analyzing a sound collected by a microphone (not shown) or the like.

[0113] Here, for example, a vehicle may be configured to be able to run by automatic driving control that does not require the occupant (driver) to operate the vehicle. When such automatic driving control is being executed, the driving load on the occupant, the listener L, is reduced, so that an image 31 may be displayed on the window or outside the vehicle, as shown in FIG. 4F. Note that the image 31 can be set to any type of image, but here it is assumed to include an image of an instrument being played that corresponds to the instrument sound included in the sound source signal, an image of a singer emitting a voice (vocals), or the like.

[0114] In this way, when the vehicle is in a state in which the automatic driving control is executed and the image 31 is displayed, the output control unit 23 may perform localization control and sound image width control of the sound images A1, AL, and AR in accordance with the image 31. For example, the output control unit 23 localizes the sound image A1 of the sound generated by the sound source signal at a position where it overlaps with the image 31 by emphasizing and outputting the separated audio sound source signal from the speakers FL and FR. Also, for example, the output control unit 23 localizes the sound image AL of the instrument sound of the L component sound source signal at a position where it overlaps with the image 31 or controls the sound image width of the sound image AL by emphasizing and outputting the separated L component sound source signal from the speakers FL and RL. Similarly, the output control unit 23 localizes the sound image AR of the instrument sound of the R component sound source signal at a position where it overlaps with the image 31 or controls the sound image width of the sound image AR by emphasizing and outputting the separated R component sound source signal from the speakers FR and RR. As a result, the sound images A1, AL, and AR are superimposed on the image 31 of the singer or instrument, making it possible for the listener L to sense the spread of the sound images A1, AL, and AR in accordance with the image 31.

[0115] Also, as shown in FIG. 4G, the seat orientation of the listener L, who is an occupant, may be changed, for example, when automatic driving control is performed or when parking. In such a case, the output control unit 23 may perform localization control and sound image width control of the sound images A1, AL, and AR according to the seat orientation (or the face orientation of the listener L). For example, the output control unit 23 localizes the sound image A1 of the sound by the sound source signal at a position in front of the listener L by emphasizing and outputting the separated voice sound source signal from a speaker selected from the speakers FL, FR, RL, and RR according to the seat orientation. Also, for example, the output control unit 23 localizes the sound image AL of the instrument sound of the L component sound source signal at a position to the left of the listener L or controls the sound image width of the sound image AL by emphasizing and outputting the separated L component sound source signal from a speaker selected according to the seat orientation. Similarly, the output control unit 23 localizes the sound image AR of the instrument sound of the R component sound source signal to the right of the listener L and controls the sound image width of the sound image AR by emphasizing and outputting the separated R component sound source signal from a speaker selected according to the seat orientation, etc. This makes it possible to localize (place) the sound images A1, AL, and AR in positions that conform to the state of the vehicle, including the orientation of the seat (in other words, the orientation of the listener L). Note that the above-mentioned orientation of the listener L's face is detected by an in-car camera (not shown), but is not limited to this.

[0116] In the above, the localization control of the sound images A1, AL, and AR is performed according to the direction of the seat and the direction of the face of the listener L, but the present invention is not limited to this, and the localization control may be performed according to an instruction from the listener L (passenger), for example. In the localization control, the sound images A1, AL, and AR are prevented from overlapping with the sound image C of noise, as shown in Fig. 4G, so that the sound images A1, AL, and AR can be made clear.

[0117] 2, for example, the sound source signal may include vocals and voice guidance in navigation, etc. In such a case, the voice guidance may overlap with the vocals, making it difficult for the listener L to hear the voice guidance.

[0118] Therefore, for example, the output control unit 23 outputs the sound source signal (removed sound source signal) from which the voice sound source signal including vocals has been removed, and the sound source signal including the voice guidance, as direct sounds from the speakers FL and FR. As a result, although not shown in the figure, the sound images AL and AR of the musical instrument sounds and the sound image of the voice guidance are localized in the vehicle cabin. Therefore, the voice guidance does not overlap with the vocals, and the listener L can easily hear the voice guidance. In addition, the output control unit 23 at this time can also provide the voice guidance without lowering the audio volume.

[0119] <Control process of the audio device according to the first embodiment> Next, a specific processing procedure in the audio device 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure executed by the audio device 1 according to the first embodiment.

[0120] 5, the control unit 2 of the acoustic device 1 acquires a sound source signal from the sound source device 50 (step S10). Next, the control unit 2 separates a voice sound source signal, an L component sound source signal, an R component sound source signal, and a reverberation component sound source signal from the sound source signal (step S11).

[0121] Next, the control unit 2 removes the voice sound source signal from the sound source signal, and applies the filter 23a to the sound source signal from which the voice sound source signal has been removed and to the reverberation sound component sound source signal, respectively, to generate a reverberation sound signal (step S12).

[0122] Next, the control unit 2 controls the output of each of the sound source signal, the separated voice sound source signal, the L component sound source signal, the R component sound source signal, and the generated reverberation sound signal (step S13).

[0123] As described above, the acoustic device 1 according to the first embodiment includes the separation unit 22 and the output control unit 23. The separation unit 22 separates a predetermined sound source signal (voice sound source signal) that does not require processing to add artificial reverberation sound from the sound source signal, and removes the separated predetermined sound source signal from the sound source signal. The output control unit 23 applies a filter 23a for generating artificial reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation unit 22, and outputs the result. This makes it possible to perform appropriate surround reproduction according to the sound source signal.

[0124] Furthermore, since the separation unit 22 uses time-frequency analysis, it is possible to separate various sound source signals such as audio sound source signals with a relatively low amount of calculation. Furthermore, in this embodiment, sound image localization control, sound image width control, optimization of the envelopment feeling LEV, etc. are performed for each of the various sound source signals such as the separated audio sound source signals, so that it is possible to improve the overall spatial impression of the sounds reproduced from the speakers FL, FR, RL, and RR.

[0125] In this embodiment, the filter 23a is used in combination with a process for separating various sound source signals, such as a voice sound source signal, from a sound source signal. This allows the filter length of the filter 23a to be shorter than when only the filter 23a is used, and therefore the amount of calculation in the control unit 2 can be reduced.

[0126] Also, for example, if the sound source signal contains a short-duration instrument sound such as a percussion instrument, the frequency characteristics may contain prominent components whose levels stand out near a certain frequency. If such prominent components exist, reverberation sounds, including pseudo-reverberation sounds, tend to become excessive during surround playback. Also, the higher the frequency, the easier it is for the reverberation sounds to be separated, and the more likely the reverberation sounds become excessive. Therefore, for example, the control unit 2 can reduce the influence of the prominent components by performing a process that creates frequency characteristics that attenuate linearly or smoothly as the frequency increases, and therefore it becomes possible to prevent the reverberation sounds from becoming excessive even for a sound source signal that contains a short-duration instrument sound such as a percussion instrument.

[0127] Furthermore, the control unit 2 may, for example, when outputting reverberation signals to the speakers FR, FL, RL, and RR (surround channels), reduce the cross-correlation by introducing random delays or randomizing the time characteristics of the rise time of the filter 23a, etc. This allows the listener L to perceive the artificial reverberation sound as a more natural reverberation sound.

[0128] Second embodiment <Configuration of the audio device according to the second embodiment> Next, the configuration of the audio device 1 according to the second embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing an example of the configuration of an audio system 100 including the audio device 1 according to the second embodiment. Note that, in the following, the same reference numerals are used for the configuration common to the first embodiment, and the description thereof will be omitted.

[0129] The control unit 2 of the acoustic device 1 according to the second embodiment includes a filter setting unit 24 that sets the gain of a filter 23a that generates a pseudo reverberation sound. In the second embodiment, the filter setting unit 24 changes the gain of the filter 23a for each sound source (musical instrument or voice), thereby making it possible to output a more natural pseudo reverberation sound from a sound source signal of one song. Specifically, in the second embodiment, the filter setting unit 24 sets a filter with an optimized gain for each sound source based on the characteristics of each sound source included in the sound source signal, thereby making it possible to realize high-quality surround reproduction.

[0130] More specifically, the acquisition unit 21 of the control unit 2 acquires sound source information related to a sound source signal. The sound source information is, for example, information on the type (genre) of the sound source signal and the recording environment, but is not limited to this. The type of the sound source signal is, for example, voice, musical instruments (percussion instruments, wind instruments, etc.), classical music, etc. The recording environment is, for example, information on the place where the sound was recorded, such as a recording studio or concert hall.

[0131] The acquisition unit 21 acquires the sound source information by, for example, an input by a user such as the listener L, or acquires the sound source information from a server or the like via the Internet. The acquisition unit 21 may also acquire the sound source information by analyzing the acquired sound source signal. The acquisition unit 21 outputs the acquired sound source information to the filter setting unit 24.

[0132] The filter setting unit 24 includes a detection unit 24a and a determination unit 24b. The detection unit 24a detects, based on a sound source signal, feature information that is a feature of an acoustic component included in the sound source signal.

[0133] For example, the detection unit 24a detects channel relation information relating to the relationship between the sound source signals reproduced in the two channels as feature information relating to the features of the acoustic components. Specifically, the channel relation information includes information on the LR channel difference and the LR channel correlation.

[0134] The LR channel difference is the above-mentioned inter-channel level difference (ICLD) or inter-channel time difference (Inter-channel Time Difference), but is not limited to these. The LR channel correlation is information on the correlation components of two sound source signals reproduced in stereo on two channels. Specifically, the LR channel correlation is the inter-channel cross correlation (ICC), but is not limited to this.

[0135] In this way, the detection unit 24a can detect the acoustic components with high accuracy by detecting the channel relationship information as the feature information, and therefore, an optimal pseudo reverberation sound can be generated by the filtering process in the output control unit 23.

[0136] The detection unit 24a continuously detects the channel relationship information at regular intervals, arranges the information in a time series, calculates a moving average of the detected time series channel relationship information, and detects the moving average as feature information. This makes it possible to smooth out steep changes in the channel relationship information, and therefore to smooth out steep changes in the gain determined by the determination unit 24b at the subsequent stage. As a result, it is possible to smooth out changes in the pseudo reverberation sound generated by the output control unit 23, thereby realizing more natural surround sound reproduction.

[0137] The detection unit 24a maximizes the channel relation information for a section in which the sound pressure level (amplitude) of the sound source signal is less than a predetermined value among the time-series channel relation information, and then calculates a moving average. In other words, the detection unit 24a calculates a moving average so as to control in a direction in which surround sound is not output in a section in which the sound pressure level of the sound source signal is less than a predetermined value among the time-series channel relation information, and not output surround sound around the section less than the predetermined value.

[0138] This lowers the upper limit at which the law of the first wave front applies when there is silence between strikes, such as in a solo percussion performance, so the filter gain for the percussion instrument is lowered to control the surround level to an appropriate level.

[0139] The detection unit 24a also detects information related to the duration and frequency of the acoustic components as feature information. The duration is the time that the sound source signal continues. Specifically, the detection unit 24a calculates the envelope of the sound source signal, differentiates the calculated envelope, and then calculates the envelope again to detect the duration from the slope of the obtained envelope. The frequency is information related to the frequency components of the sound source signal. Specifically, the detection unit 24a detects the center of gravity of the frequency of the sound source signal in sound source data obtained by performing a short-time Fourier transform on the sound source signal.

[0140] In this way, the detection unit 24a can detect acoustic components with high accuracy by detecting information such as duration and frequency as feature information, and therefore can generate optimal artificial reverberation sound by filter processing in the output control unit 23 at a downstream stage.

[0141] The determination unit 24b determines the gain of the filter 23a used by the output control unit 23. Specifically, the determination unit 24b determines the gain of the filter 23a based on at least one of the feature information detected by the detection unit 24a and the sound source information acquired by the acquisition unit 21.

[0142] Specifically, the determination unit 24b corrects the threshold range obtained from the first wave front law based on at least one of the feature information and the sound source information, and determines the gain based on the corrected threshold range. Here, the process of determining the gain based on the first wave front law will be described with reference to FIG. 7.

[0143] Fig. 7 is a diagram for explaining the gain determination process by the determination unit 24b. As shown in Fig. 7, the determination unit 24b first corrects the threshold range obtained from the first wave front law based on the feature information (channel relation information, duration and frequency) and sound source information.

[0144] For example, the regions R1 and R2, which are the threshold ranges shown in the middle graph of Fig. 7, are set as the reference ranges. In this case, the determination unit 24b corrects the regions R1 and R2 by correcting the thresholds TH1 and TH2 on the two axes of time and amplitude based on the feature information and sound source information.

[0145] For example, the larger the channel relation information, the larger the determination unit 24b corrects the thresholds TH1 and TH2 to increase the time and amplitude. That is, the larger the channel relation information, the larger the determination unit 24b corrects the regions R1 and R2.

[0146] On the other hand, the smaller the channel relation information, the more the determination unit 24b corrects the thresholds TH1 and TH2 in the direction of decreasing the time and amplitude. In other words, the smaller the channel relation information, the more the determination unit 24b corrects the regions R1 and R2.

[0147] The channel relation information increases as the inter-channel level difference, which is the channel relation information, increases, or as the inter-channel time difference increases, or as the cross-correlation decreases (as the number of uncorrelated components increases).

[0148] In addition, the longer the duration of the sound component, the larger the thresholds TH1 and TH2 are corrected by the determination unit 24b. In other words, the longer the duration of the sound component, the larger the regions R1 and R2 are corrected by the determination unit 24b.

[0149] On the other hand, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude become smaller as the duration of the sound component becomes shorter. In other words, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude become smaller as the duration of the sound component becomes shorter.

[0150] In addition, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude become larger as the frequency (center of gravity) of the sound component becomes lower. In other words, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude become larger as the frequency (center of gravity) of the sound component becomes lower.

[0151] On the other hand, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude become smaller as the frequency (center of gravity) of the sound component becomes higher. In other words, the determination unit 24b corrects the regions R1 and R2 so that they become smaller as the frequency (center of gravity) of the sound component becomes higher.

[0152] Furthermore, when the sound source signal is classical based on the sound source information, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction in which the time and amplitude increase. In other words, when the sound source signal is classical, the determination unit 24b corrects the regions R1 and R2 to be larger.

[0153] On the other hand, when the sound source signal is determined to be a voice from the sound source information, the determination unit 24b corrects the thresholds TH1 and TH2 in a direction that reduces the time and the amplitude. In other words, when the sound source signal is a voice, the determination unit 24b corrects the regions R1 and R2 to be smaller.

[0154] Then, based on the corrected threshold range, the determination unit 24b determines the gain of the filter 23a used in the downstream output control unit 23. Specifically, the determination unit 24b determines the gain so that the pseudo reverberation sound generated by the filter 23a falls within the corrected threshold range.

[0155] That is, when the areas R1 and R2 are corrected to be larger, the determiner 24b determines the gain so that the time and amplitude of the artificial reverberation sound are increased. On the other hand, when the areas R1 and R2 are corrected to be smaller, the determiner 24b determines the gain so that the time and amplitude of the artificial reverberation sound are decreased.

[0156] In this way, the determination unit 24b can determine the gain for generating an optimal pseudo-reverberation sound by determining the gain from the threshold range of the first wave front law corrected based on the feature information and the sound source information.

[0157] Then, the output control unit 23 generates and outputs a reverberation signal indicative of a simulated reverberation sound, using the filter 23a to which the gain determined by the determination unit 24b is set.

[0158] As a result, in the second embodiment, even if the characteristics of the reverberation of the sound source are different, such as voice, percussion, classical music, etc., it is possible to generate an optimal pseudo reverberation sound for each sound source, and it is possible to perform optimal surround reproduction according to the characteristics of the sound source. That is, in the second embodiment, it is possible to perform high-quality surround reproduction.

[0159] <Control process of the audio device according to the second embodiment> Next, a specific processing procedure in the audio device 1 according to the second embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the processing procedure executed by the audio device 1 according to the second embodiment.

[0160] 8, the control unit 2 of the audio device 1 performs the same processes as those in the first embodiment in steps S10 and S11. Next, the control unit 2 acquires sound source information related to the sound source signal (step S11a).

[0161] Next, the control unit 2 detects feature information that is a feature of the acoustic component included in the sound source signal based on the acquired sound source signal and sound source information (step S11b). Next, the control unit 2 determines a threshold range in the first wave front law based on the detected feature information (step S11c).

[0162] Next, the control unit 2 determines the gain of the filter 23a based on the determined threshold range so that the pseudo reverberation sound falls within the threshold range (step S11d). Then, the control unit 2 executes the processes from step S12 onward using the filter 23a whose gain has been determined and set.

[0163] In the above, when removing the voice sound source signal from the sound source signal, the time-frequency analysis is performed on the Lch sound source signal and the Rch sound source signal, but the present invention is not limited to this. That is, the process of removing the voice sound source signal from the sound source signal may be performed using the difference or ratio between the Lch sound source signal and the Rch sound source signal, the LR channel correlation, etc., without performing the time-frequency analysis, thereby reducing the calculation.

[0164] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0165] 1 Sound equipment 2. Control section 22 Separation part 23 Output control section 24a Detector 24b Decision section

Claims

1. a separation unit that separates a predetermined sound source signal that does not require addition of a pseudo reverberation sound from a sound source signal and removes the separated predetermined sound source signal from the sound source signal; an output control unit that applies a filter for generating the pseudo reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation unit, and outputs the resultant signal; Equipped with The output control unit is applying the filter to the sound source signal from which the predetermined sound source signal has been removed and to the predetermined sound source signal, and outputting the artificial reverberation sound corresponding to the sound source signal from which the predetermined sound source signal has been removed such that a reverberation level of the artificial reverberation sound is higher than a reverberation level of the artificial reverberation sound corresponding to the predetermined sound source signal. sound equipment.

2. The output control unit is outputting the predetermined sound source signal so that a sound image of the sound due to the predetermined sound source signal separated by the separation unit is localized at a predetermined position; 2. An acoustic device according to claim 1.

3. The separation unit is performing a time-frequency analysis on the sound source signal to separate the predetermined sound source signal; 3. An acoustic device according to claim 1 or 2.

4. The sound source signal is A stereo signal that is played back in stereo on two channels. The separation unit is calculating difference information indicating a difference between the sound source signals reproduced on the two channels, and separating the predetermined sound source signal based on the calculated difference information; An acoustic device according to any one of claims 1 to 3.

5. The sound source signal is A stereo signal that is played back in stereo on two channels. The separation unit is Separating a first sound source signal including a component of a sound to be reproduced in one of the two channels from the sound source signal, and a second sound source signal including a component of a sound to be reproduced in the other of the two channels; The output control unit is outputting the first sound source signal separated by the separation unit from one of the channels to control a sound image width of the sound due to the first sound source signal, and outputting the second sound source signal from the other of the channels to control a sound image width of the sound due to the second sound source signal; An acoustic device according to any one of claims 1 to 4.

6. a detection unit that detects, based on the sound source signal, feature information indicating features of an acoustic component included in the sound source signal; a determination unit that determines a gain of the filter based on the feature information detected by the detection unit; Further equipped with The output control unit is generating and outputting a reverberation signal indicative of the artificial reverberation sound using the filter to which the gain determined by the determination unit is set; An acoustic device according to any one of claims 1 to 5.

7. Acquisition unit for acquiring vehicle status Further equipped with The output control unit is applying the filter to the sound source signal from which the predetermined sound source signal has been removed in accordance with the state of the vehicle acquired by the acquisition unit, and outputting the sound source signal from a channel disposed at least one of behind and above a listener. An acoustic device according to any one of claims 1 to 6.

8. The predetermined sound source signal includes a voice component. An acoustic device according to any one of claims 1 to 7.

9. a separation step of separating a predetermined sound source signal that does not require addition of a pseudo reverberation sound from the sound source signal, and removing the separated predetermined sound source signal from the sound source signal; an output control step of applying a filter for generating the pseudo reverberation sound to the sound source signal from which the predetermined sound source signal has been removed by the separation step, and outputting the resultant signal; Including, The output control step includes: applying the filter to the sound source signal from which the predetermined sound source signal has been removed and to the predetermined sound source signal, and outputting the artificial reverberation sound corresponding to the sound source signal from which the predetermined sound source signal has been removed such that a reverberation level of the artificial reverberation sound is higher than a reverberation level of the artificial reverberation sound corresponding to the predetermined sound source signal. Acoustic control methods.

Images