Mixing device and mixing method

Abstract

The present invention relates to a mixing device and a mixing method. A mixing device includes: a weighting circuit configured to weight the amplitude spectrum of a priority audio signal and a non-priority audio signal in a specific control target frequency band in consideration of frequency masking and time masking; an amplitude converter configured to relatively amplify the amplitude spectrum of the priority audio signal in the control target frequency band based on the amplitude spectrum to which the weighting is applied; a phase converter configured to make the phase spectrum of the non-priority audio signal close to the phase spectrum of the priority audio signal in the control target frequency band based on the amplitude spectrum to which the weighting is applied; and a mixer configured to mix the priority audio signal and the non-priority audio signal after the amplitude spectrum and the phase spectrum are changed.

Description

Technical Field

[0001] The invention relates to a sound mixing device and a sound mixing method. Background Art

[0002] A known audio mixing device is configured to mix audio signals input from multiple sound source circuits. In this audio mixing device, when a music signal is input from a vehicle audio device and an audio guidance signal is input from a navigation device, the music signal is temporarily muted and mixed with the muted audio guidance signal. This allows the driver to clearly hear the audio guidance. However, a passenger who is currently listening to music may experience inconvenience because the volume of the music may be temporarily reduced.

[0003] Conventionally, there is known a sound mixing device configured to convert each of the audio signals input from a plurality of sound source circuits into data on a time-frequency plane, and adjust the amplitude and phase represented by the converted data on the time-frequency plane to ensure the audibility of the sound of the sound source circuit that the listener wishes to hear, without significantly changing the amplitude of the sound signal output from each sound source circuit. An example of this conventional sound mixing device is disclosed in Japanese Provisional Patent Application No. 2013-51589.

[0004] In conventional mixing devices, the audibility of the audio guidance is ensured even when music clips are played in the vehicle. Summary of the Invention

[0005] However, according to the conventional sound mixing apparatus, there may be a case where a sound corresponding to a sound source circuit is masked by another sound corresponding to another sound source circuit, and a listener cannot clearly hear a sound that the listener wants to clearly hear.

[0006] Hereinafter, the sound of a sound source circuit that the listener wishes to hear clearly (i.e., the sound with higher priority) will be referred to as “priority sound”, and the sound of another sound source circuit having lower priority than the priority sound will be referred to as “non-priority sound”.

[0007] In view of the above situation, the technology according to the present invention is proposed, and its purpose is to provide a mixing device and mixing method that can suppress the non-priority sound from masking the priority sound when playing the non-priority sound and ensure the audibility of the priority sound.

[0008] According to an exemplary embodiment of the present invention, a mixing device is configured to mix a first audio signal representing a first sound with a second audio signal representing a second sound so that a listener can listen to the first sound in a manner that is more prioritized than the second sound. The mixing device includes: a weighting circuit configured to weight the amplitude spectrum of at least one of the first audio signal and the second audio signal within a specific control target frequency band in consideration of at least one of frequency masking and time masking; an amplitude converter configured to perform amplitude spectrum change processing, the amplitude spectrum change processing relatively amplifies the amplitude spectrum of the first audio signal within the control target frequency band by changing the amplitude spectrum of at least one of the first audio signal and the second audio signal based on the amplitude spectrum applied with the weighting; a phase converter configured to perform phase spectrum change processing, the phase spectrum change processing brings the phase spectrum of the second audio signal close to the phase spectrum of the first audio signal within the control target frequency band by changing the phase spectrum of at least one of the first audio signal and the second audio signal based on the amplitude spectrum applied with the weighting; and a mixer configured to mix the first audio signal and the second audio signal after the amplitude spectrum change processing and the phase spectrum change processing have been performed.

[0009] Therefore, the amplitude spectrum of at least one of the first and second audio signals is weighted in consideration of at least one of frequency masking and time masking, the amplitude spectrum of the first audio signal is relatively amplified, and the phase spectrum of the second audio signal is brought close to the phase spectrum of the first audio signal within a control target frequency band based on the weighted amplitude spectrum. The first and second audio signals are then mixed. Consequently, when the second sound is played, masking of the first sound by the second sound can be suppressed, thereby ensuring the audibility of the first sound.

[0010] The weighting circuit may be configured to weight the amplitude spectrum by taking frequency masking into consideration, widening a shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal within a control target frequency band so that the shape simulates a range in which a sound having an amplitude spectrum masks other sounds on a frequency axis.

[0011] Therefore, by performing weighting in consideration of frequency masking, when the second sound is played, masking of the first sound by the second sound can be further suppressed, and audibility of the first sound can be ensured.

[0012] The weighting circuit may be configured to weight the amplitude spectrum by taking time masking into consideration, widening a shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal within a control target frequency band so that the shape simulates a range in which a sound having the amplitude spectrum masks other sounds on a time axis.

[0013] Therefore, by performing weighting in consideration of temporal masking, when the second sound is played, masking of the first sound by the second sound can be further suppressed, and audibility of the first sound can be ensured.

[0014] The sound mixing apparatus according to the exemplary embodiment of the present invention may be configured to include a calculator configured to calculate a specific control value based on an amplitude ratio of the first audio signal and the second audio signal to which weighting is applied within a control target frequency band.

[0015] In this case, the amplitude converter may be configured to amplify the amplitude spectrum of the first audio signal and attenuate the amplitude spectrum of the second audio signal within a control target frequency band based on the specific control value.

[0016] Therefore, by amplifying the amplitude spectrum of the first audio signal and reducing the amplitude spectrum of the second audio signal within the control target frequency band based on the above control value, when the first sound is played, the masking of the first sound by the second sound can be further suppressed, and the audibility of the first sound can be ensured.

[0017] The phase converter may be configured to change only the phase spectrum of the second audio signal based on the specific control value, and match the phase spectrum of the second audio signal with the phase spectrum of the first audio signal within a control target frequency band.

[0018] Therefore, by changing only the phase spectrum of the second audio signal based on the above-mentioned control value and matching the phase spectrum of the second audio signal with the phase spectrum of the first audio signal within the control target frequency band, when the second sound is played, the masking of the first sound by the second sound can be further suppressed, and the audibility of the first sound can be ensured.

[0019] According to an exemplary embodiment of the present invention, a mixing device may include: a first converter configured to overlap each of a first audio signal and a second audio signal to convert into a frequency domain signal; and a second converter configured to overlap each of the first audio signal and the second audio signal to convert into a time domain signal after an amplitude spectrum changing process of an amplitude converter and a phase spectrum changing process of a phase converter. In that case, a weighting circuit weights the amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into a frequency domain signal by the first converter. The amplitude converter changes the amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into a frequency domain signal by the first converter. The phase converter changes the phase spectrum of at least one of the first audio signal and the second audio signal that have been converted into a frequency domain signal by the first converter.

[0020] Therefore, when the first converter converts the first audio signal and the second audio signal into frequency domain signals, and when the second converter converts the first audio signal and the second audio signal into time domain signals, overlapping processing is performed on the first audio signal and the second audio signal, thereby smoothing the discontinuity of the waveforms generated by the amplitude spectrum changing processing of the amplitude converter and the phase changing processing of the phase spectrum converter.

[0021] A mixing method according to an exemplary embodiment of the present invention is a method for mixing a first audio signal representing a first sound with a second audio signal representing a second sound so that a listener can listen to the first sound in a manner that is more prioritized than the second sound. The mixing method includes: a weighting step of weighting the amplitude spectrum of at least one of the first audio signal and the second audio signal within a specific control target frequency band in consideration of at least one of frequency masking and time masking; an amplitude conversion step of performing an amplitude spectrum change process, wherein the amplitude spectrum change process relatively amplifies the amplitude spectrum of the first audio signal within the control target frequency band by changing the amplitude spectrum of at least one of the first audio signal and the second audio signal based on the amplitude spectrum applied with the weighting; a phase conversion step of performing a phase spectrum change process, wherein the phase spectrum change process brings the phase spectrum of the second audio signal close to the phase spectrum of the first audio signal within the control target frequency band by changing the phase spectrum of at least one of the first audio signal and the second audio signal based on the amplitude spectrum applied with the weighting; and a mixing step of mixing the first audio signal and the second audio signal after the amplitude spectrum change process and the phase spectrum change process have been performed.

[0022] According to exemplary embodiments of the present invention, there is provided a sound mixing apparatus and a sound mixing method capable of suppressing masking of a first sound by a second sound and ensuring audibility of the first sound even when the second sound flows. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a block diagram illustrating a functional configuration of a sound mixing apparatus according to an exemplary embodiment of the present invention.

[0024] Figure 2 2 is a diagram illustrating an audio signal input to an FFT (Fast Fourier Transform) circuit provided in a sound mixing device according to an exemplary embodiment of the present invention, a Fourier transform length N, and an overlap length M when a short-time Fourier transform process is performed on the audio signal.

[0025] Figure 3A is a schematic diagram showing the amplitude spectrum at time t0.

[0026] Figure 3B is a schematic diagram showing the amplitude spectrum at time t1.

[0027] Figure 3C is a schematic diagram showing the amplitude spectrum at time t2.

[0028] Figure 4A is a schematic diagram showing temporal variation of the amplitude of the amplitude spectrum at frequency f0.

[0029] Figure 4B is a schematic diagram showing temporal variation of the amplitude of the amplitude spectrum at frequency f1.

[0030] Figure 4C is a schematic diagram showing temporal variation of the amplitude of the amplitude spectrum at frequency f2.

[0031] Figure 5 is a diagram illustrating filtering characteristics of a BPF (Band Pass Filter) and a BSF (Band Stop Filter) provided in a sound mixing device according to an exemplary embodiment of the present invention.

[0032] Figure 6 is a block diagram showing a functional configuration of a weight calculator provided in a sound mixing apparatus according to an embodiment of the present invention.

[0033] Figure 7A is a schematic diagram illustrating a configuration and operation of a frequency-axis weight calculator provided in a weight calculator according to an exemplary embodiment of the present invention.

[0034] Figure 7B is a diagram showing an amplitude spectrum at time t0 to be input to LPF F0.

[0035] Figure 7C is a diagram showing an amplitude spectrum at time t0 to be input to the LPF F1.

[0036] Figure 7D is shown to be input to the LPF F k Schematic diagram of the amplitude spectrum at time t0.

[0037] Figure 8 is a diagram illustrating characteristics of each LPF (Low Pass Filter) provided in the frequency-axis weight calculator according to an exemplary embodiment of the present invention.

[0038] Figure 9 is a schematic diagram illustrating an amplitude spectrum obtained by weighting an amplitude spectrum of a sine wave having a plurality of frequency components in consideration of frequency masking according to an exemplary embodiment of the present invention.

[0039] Figure 10is a block diagram illustrating a configuration of a time axis weight calculator provided in a weight calculator according to an exemplary embodiment of the present invention.

[0040] Figure 11A is a schematic diagram illustrating a function held by a control gain calculator provided in a weight calculator according to an exemplary embodiment of the present invention.

[0041] Figure 11B is a schematic diagram illustrating a function held by a control gain calculator provided in a weight calculator according to an exemplary embodiment of the present invention.

[0042] Figure 12A is a block diagram illustrating a configuration of a spectrum controller included in a sound mixing apparatus according to an exemplary embodiment of the present invention.

[0043] Figure 12B is a block diagram illustrating a configuration of a spectrum controller included in a sound mixing apparatus according to an exemplary embodiment of the present invention.

[0044] Figure 13A is a block diagram illustrating a configuration of an amplitude controller and a phase controller provided in a spectrum controller according to an exemplary embodiment of the present invention.

[0045] Figure 13B is a block diagram illustrating a configuration of an amplitude controller and a phase controller provided in a spectrum controller according to an exemplary embodiment of the present invention.

[0046] Figure 14A is a schematic diagram illustrating a function held by a weight coefficient calculator provided in a phase controller according to an exemplary embodiment of the present invention.

[0047] Figure 14B is a schematic diagram illustrating a function held by a weight coefficient calculator provided in a phase controller according to an exemplary embodiment of the present invention.

[0048] Figure 15A is a block diagram illustrating a configuration of an amplitude controller and a phase controller provided in a spectrum controller according to an exemplary embodiment of the present invention.

[0049] Figure 15B is a block diagram illustrating a configuration of an amplitude controller and a phase controller provided in a spectrum controller according to an exemplary embodiment of the present invention.

[0050] Figure 16A is a schematic diagram illustrating a function held by a weight coefficient calculator provided in a phase controller according to an exemplary embodiment of the present invention.

[0051] Figure 16Bis a schematic diagram illustrating a function held by a weight coefficient calculator provided in a phase controller according to an exemplary embodiment of the present invention.

[0052] Figure 17A is a diagram illustrating an amplitude spectrum of a priority audio signal itself that has not been subjected to a mixing process according to an exemplary embodiment of the present invention.

[0053] Figure 17B is a diagram illustrating an amplitude spectrum of a priority audio signal subjected to a mixing process according to an exemplary embodiment of the present invention.

[0054] Figure 18A is a diagram illustrating an amplitude spectrum of a non-priority audio signal itself that has not been subjected to a mixing process according to an exemplary embodiment of the present invention.

[0055] Figure 18B is a diagram illustrating an amplitude spectrum of a non-priority audio signal subjected to a mixing process according to an exemplary embodiment of the present invention.

[0056] Figure 19 is a diagram illustrating a difference between a phase spectrum of a non-priority audio signal itself that is not subjected to a mixing process according to an exemplary embodiment of the present invention and a phase spectrum of a non-priority audio signal that is subjected to a mixing process according to an exemplary embodiment of the present invention.

[0057] Figure 20A is a diagram illustrating a mixed signal obtained by mixing a priority audio signal itself that is not subjected to a mixing process according to an exemplary embodiment of the present invention and a non-priority audio signal itself that is not subjected to a mixing process according to an exemplary embodiment of the present invention.

[0058] Figure 20B is shown when the priority audio signal is executed Figure 17B The amplitude amplification shown is performed on the non-priority audio signal Figure 18B The reductions shown and Figure 19 Schematic diagram of a mixed signal between a priority audio signal and a non-priority audio signal output from an IFFT (Inverse Fast Fourier Transform) circuit provided in an audio mixing device according to an exemplary embodiment of the present invention when a phase is changed. DETAILED DESCRIPTION

[0059] Hereinafter, an exemplary embodiment of the present invention will be described with reference to the accompanying drawings. Hereinafter, a sound mixing device installed in a vehicle will be described as an example of an embodiment of the present invention. It should be noted that the sound mixing device according to the present invention is not limited to a sound mixing device installed in a vehicle.

[0060] Figure 1 1 is a block diagram showing a functional configuration of a sound mixing apparatus 1 according to an embodiment of the present invention. The sound mixing apparatus 1 is configured to mix a priority audio signal of a priority sound that the apparatus intends to make a listener listen to with high priority with a non-priority audio signal of a non-priority sound having a lower priority than the priority audio signal. Figure 1 As shown, the audio mixing device 1 includes: FFT circuits 10A and 10B, frequency band dividers 12A and 12B, phase calculators 14A and 14B, amplitude calculators 16A and 16B, a weight calculator 18, spectrum controllers 20A and 20B, complex number calculators 22A and 22B, adders 24A, 24B, and 26, an IFFT circuit 28, a system controller 30, and an operating device 32. Priority sounds and non-priority sounds are examples of first and second sounds, respectively. Priority audio signals and non-priority audio signals are examples of first and second audio signals, respectively. The audio mixing device 1 is configured to mix the priority audio signals with the non-priority audio signals so that a listener can listen to the priority sounds with higher priority than the non-priority sounds.

[0061] Audio signals from different sound source circuits are input to FFT circuits 10A and 10B, respectively. According to the present embodiment, an audio signal (more specifically, an audio guidance signal) from a navigation device (not shown) is input to FFT circuit 10A, while a music signal from a vehicle audio device (not shown) is input to FFT circuit 10B. Each of the navigation device and the vehicle audio device is an example of a sound source circuit.

[0062] When an audio guidance signal is input from the navigation device, the mixing device 1 mixes the audio guidance signal with the music signal and outputs the mixed signal to the speakers installed in the vehicle. Therefore, when music is being played in the vehicle, the audio guidance is reproduced. According to the present embodiment, when the audio guidance is reproduced while music is being played in the vehicle in this manner, the masking of the audio guidance by the music can be suppressed, and the audibility of the audio guidance can be ensured. In the following description, the audio guidance will be referred to as "priority sound", and the music will be referred to as "non-priority sound". In addition, the audio guidance signal, which is a signal of the priority sound, will be referred to as the "priority audio signal", and the music signal, which is a signal of the non-priority sound, will be referred to as the "non-priority audio signal".

[0063] The system controller 30 is configured to determine an audio signal input from a navigation device (ie, an audio guidance signal) as a priority audio signal, determine an audio signal input from a vehicle audio device (ie, a music signal) as a non-priority audio signal, and control each circuit of the sound mixing device 1 .

[0064] For example, the system controller 30 determines whether an audio signal is a priority audio signal or a non-priority audio signal based on the sound source circuit of the audio signal (in this embodiment, based on whether the sound source circuit is a navigation device or a vehicle audio device). Note that the system controller 30 may be configured to determine whether an audio signal is a priority audio signal or a non-priority audio signal based on meta-information included in the audio signal (e.g., based on information indicating whether the audio signal is a priority audio signal or a non-priority audio signal).

[0065] It should be noted that the listener may be allowed to arbitrarily set from which sound source circuit the audio signal is the priority audio signal and the non-priority audio signal, respectively, by operating the operating device 32 .

[0066] The FFT circuits 10A and 10B are configured to apply overlapping processing and weighting processing using a window function to the audio signals input from the respective sound source circuits. The FFT circuits 10A and 10B convert the audio signals, to which the overlapping processing and weighting processing using the window function have been applied, from the time domain to the frequency domain through short-time Fourier transform processing, and output the converted signals as spectrums to the frequency band dividers 12A and 12B, respectively. That is, each of the FFT circuits 10A and 10B functions as a first converter, which is configured to overlap each of the priority audio signal and the non-priority audio signal and convert the overlapped signals into frequency domain signals.

[0067] Figure 2 1 is a diagram showing an audio signal input to the FFT circuit 10A or 10B, a Fourier transform length N, and an overlap length M when a short-time Fourier transform process is performed on the audio signal. Figure 2 In the figure, the waveform represents the waveform of the audio signal in the time domain, and each solid line under the waveform represents the Fourier transform length N. Figure 2 As shown, the FFT circuit 10A or 10B performs short-time Fourier transform while shifting time by the difference time between the Fourier transform length N and the overlap length M. Therefore, n frequency spectra shifted time by the difference time are obtained.

[0068] Figures 3A to 3C is a schematic diagram showing the amplitude spectrum of each time shift. Figure 3A shows the amplitude spectrum at time t0, Figure 3B shows the amplitude spectrum at time t1, Figure 3C The amplitude spectrum at time t2 is shown. Figures 3A to 3C Each of the frequencies (ie, f0, f1, f2, ..., f N-2 、f N-1 ). In the case of Fourier transform length N, the total number of amplitude spectra is N. Since the music signal and the audio guide signal are non-stationary signals, Figures 3A to 3CAs shown, the amplitude spectrum fluctuates according to the shift time.

[0069] Figures 4A to 4C Shown is a temporal change in the amplitude of the amplitude spectrum at each frequency when the amplitude spectrum fluctuates according to the shift time. Figure 4A shows the time variation of the amplitude at frequency f0, Figure 4B shows the time variation of the amplitude at frequency f1, Figure 4C shows the amplitude at frequency f N-1 time changes. Figures 4A to 4C Each of shows the amplitude at each time instant (t0, t1, t2, ...). The amplitude spectrum of the time shift interval is sampled and output as a frequency spectrum.

[0070] Each of the stripers 12A and 12B includes a BPF 120 and a BSF 122, respectively. Figure 5 The filtering characteristics of BPF 120 and BSF 122 are shown. Figure 5 In the figure, the vertical axis represents power (unit: dB) and the horizontal axis represents frequency (unit: Hz). Figure 5 In FIG. 1 , the dotted line represents the filtering characteristics of the BPF 120 , the dashed line represents the filtering characteristics of the BSF 122 , and the solid line represents the combined characteristics of the filtering characteristics of the BPF 120 and the BSF 122 .

[0071] The BPF 120 is configured to pass only the signal having the frequency band to be controlled (hereinafter referred to as the “control target frequency band”) among the signals input from the FFT circuit 10A (or the FFT circuit 10B), and output the passed signal to both the phase calculator 14A and the amplitude calculator 16A (or both the phase calculator 14B and the amplitude calculator 16B).

[0072] The BSF 122 is configured to block signals within the control target frequency band and pass signals having frequency bands other than the control target frequency band among signals input from the FFT circuit 10A (or the FFT circuit 10B), and output the passed signals to the adder 24A (or the adder 24B).

[0073] The control target frequency band is the frequency band of the priority audio signal. In this embodiment, since the priority audio signal is an audio guide signal (which is a human voice signal), 130 Hz to 8000 Hz is set as the control target frequency band.

[0074] When the listener changes the sound source circuit of the priority audio signal by operating the operating device 32, the control target frequency band is set to the frequency band of the audio signal of the changed sound source circuit. It should be noted that the listener can arbitrarily change the control target frequency band by operating the operating device 32.

[0075] The phase calculator 14A and the amplitude calculator 16A are configured to respectively calculate the phase spectrum and the amplitude spectrum within the control target frequency band based on the frequency spectrum within the control target frequency band input from the frequency band divider 12A. Hereinafter, for the sake of convenience, the phase spectrum within the control target frequency band calculated by the phase calculator 14A will be referred to as "phase spectrum pA", and the amplitude spectrum within the control target frequency band calculated by the amplitude calculator 16A will be referred to as "amplitude spectrum aA". The phase calculator 14A is configured to output the calculated phase spectrum pA within the control target frequency band to each of the spectrum controllers 20A and 20B. The amplitude calculator 16A is configured to output the calculated amplitude spectrum aA within the control target frequency band to the weight calculator 18 and the spectrum controller 20A.

[0076] The phase calculator 14B and the amplitude calculator 16B are configured to respectively calculate the phase spectrum and the amplitude spectrum within the control target frequency band based on the frequency spectrum within the control target frequency band input from the frequency band divider 12B. Hereinafter, for the sake of convenience, the phase spectrum within the control target frequency band calculated by the phase calculator 14B will be referred to as "phase spectrum pB", and the amplitude spectrum within the control target frequency band calculated by the amplitude calculator 16B will be referred to as "amplitude spectrum aB". The phase calculator 14B is configured to output the calculated phase spectrum pB within the control target frequency band to the spectrum controllers 20A and 20B. The amplitude calculator 16B is configured to output the calculated amplitude spectrum aB within the control target frequency band to the weight calculator 18 and the spectrum controller 20B.

[0077] Figure 6 is a block diagram showing the functional configuration of the weight calculator 18. Figure 6 As shown, the weight calculator 18 includes frequency-axis weight calculators 180A and 180B, time-axis weight calculators 182A and 182B, and a control gain calculator 183 .

[0078] The frequency-axis weight calculators 180A and 180B are configured to weight the amplitude spectra aA and aB within the control target frequency band input from the amplitude calculators 16A and 16B, respectively, while taking frequency masking into account. Specifically, the frequency-axis weight calculators 180A and 180B weight the amplitude spectra aA and aB, respectively, so that each of the amplitude spectra aA and aB simulates the range of frequency masking that occurs between two sounds of the two sound source circuits.

[0079] 7A to 7D A schematic diagram illustrating the functional configuration and operation of the frequency-axis weight calculator 180A is shown. Figure 7A 180A. Since the configuration of the frequency-axis weight calculator 180B is the same as that of the frequency-axis weight calculator 180A, a detailed description thereof will be omitted.

[0080] like Figure 7A As shown, the frequency axis weight calculator 180A includes a plurality of LPFs (F0, F1, ..., F k ) and adder AD. Figures 7B to 7C Schematic diagrams showing amplitude spectra to be input to a plurality of LPFs are shown. In each LPF, an amplitude spectrum of each frequency within a control target frequency band is input. Figures 7B to 7C Shown Figure 3A The amplitude spectrum at time t0 of FIG. 1 is taken as an example of the amplitude spectrum to be input to the LPF.

[0081] Figure 8 The characteristics of each LPF provided by the frequency axis weight calculator 180A are shown. Figure 8 In , the vertical axis represents power (dB) and the horizontal axis represents normalized frequency (unitless). As an example, Figure 8 Characteristics of LPFs applied to amplitude spectra with different frequencies (125 Hz, 250 Hz, 500 Hz, 1000 Hz, 2,000 Hz, 4,000 Hz, 8,000 Hz) in one octave, respectively, are shown.

[0082] Each LPF provided by the frequency axis weight calculator 180A is configured to perform low-pass filtering on the input amplitude spectrum and output it to the adder AD. The adder AD is configured to add the amplitude spectra input from each LPF (i.e., the amplitude spectra of each frequency after filtering). As a result, the amplitude spectra of each frequency after filtering are summed at the adder AD, and a weighted amplitude spectrum that takes frequency masking into account is obtained for the control target frequency band.

[0083] Hereinafter, for convenience of explanation, the signals output from the frequency-axis weight calculators 180A and 180B (ie, weighted magnitude spectra considering frequency masking) will be referred to as "magnitude spectrum fmA" and "magnitude spectrum fmB," respectively.

[0084] Generally speaking, when filtering is applied to a signal in a frequency spectrum, the output of certain frequency components has uneven delays. In this embodiment, when the amplitude spectrum passing through each LPF is unevenly delayed, the amplitude spectrum within the control target frequency band obtained by adding multiple amplitude spectrums by adder AD will not become smooth as a whole. Therefore, in this embodiment of the present invention, frequency axis weight calculator 180A uses an LPF that does not cause phase distortion (i.e., does not cause delay).

[0085] Figure 9 1 shows an amplitude spectrum fmA obtained by weighting the amplitude spectrum of a sine wave having frequency components of 125 Hz, 250 Hz, 500 Hz, 1000 Hz, 2000 Hz, 4000 Hz, and 8000 Hz by the frequency axis weight calculator 180A in consideration of frequency masking. Figure 9 In the figure, the vertical axis represents power (dB) and the horizontal axis represents frequency (unit: Hz). Figure 9 In the figure, the solid lines represent the line spectra (i.e., frequency components) contained in the sine wave, the dotted lines represent the amplitude spectrum fmA, and the dotted lines represent the 1 / 3 octave bandwidth centered on each line spectrum.

[0086] exist Figure 9 In the example, by signal processing in the frequency axis weight calculator 180A, the shape of each frequency component included in the sine wave is converted into a spectral shape with a width (i.e., a spectral shape with a bandwidth of approximately 1 / 3 times the frequency). This shape (i.e., the shape of the amplitude spectrum fmA) is similar to the shape of the human auditory filter. In addition, this shape simulates the range of other sounds on the frequency axis (i.e., other sounds with frequencies different from the sine wave) that the sound of the sine wave masks, and the higher the frequency, the wider the bandwidth of the range. Therefore, it can be seen that in the frequency axis weight calculator 180A, a weight that takes frequency masking into account is applied to the amplitude spectrum aA within the control target frequency band.

[0087] The time-axis weight calculators 182A and 182B are configured to weight the amplitude spectra fmA and fmB input from the frequency-axis weight calculators 180A and 180B, respectively, while taking into account time masking. Specifically, the time-axis weight calculators 182A and 182B weight the amplitude spectra fmA and fmB, respectively, so that each of the amplitude spectra fmA and fmB simulates the range of time masking that occurs between two sounds of the two sound source circuits.

[0088] Hereinafter, for the sake of convenience, the signals output from the time axis weight calculators 182A and 182B (i.e., the amplitude spectrum weighted considering the time mask in addition to the frequency mask) will be referred to as "amplitude spectrum tmA" and "amplitude spectrum ftmB", respectively.

[0089] Weighting that takes into account temporal masking can be performed using conventional techniques such as those described in Japanese Patent No. 5898534, the teachings of which are incorporated herein by reference. Figure 10 182B is the same as that of the time axis weight calculator 182A, and a detailed description thereof will be omitted.

[0090] like Figure 10 As shown, the time axis weight calculator 182A includes an attack sound controller 1821 , a reverb controller 1822 , an adder 1823 , and a limiter 1824 .

[0091] The magnitude spectrum fmA is input from the frequency axis weight calculator 180A to each of the attack sound controller 1821 and the reverberation controller 1822 .

[0092] like Figure 10 As shown, the attack sound controller 1821 includes a HPF (High Pass Filter) 1821 a , a limiter 1821 b , and a gain circuit 1821 c .

[0093] For each amplitude spectrum, HPF 1821a is configured to perform high-pass filtering on the input amplitude spectrum fmA. Limiter 1821b sets the amplitude of the negative side of the high-pass filtered amplitude spectrum to zero. By setting the amplitude of the negative side to zero, the rising component of the signal of each amplitude spectrum, i.e., the attack component (attack sound), can be detected.

[0094] According to this embodiment, the higher the cutoff frequency set in HPF 1821a, the shorter the attack sound suppression time. In other words, the lower the cutoff frequency, the longer the attack sound suppression time. For example, the cutoff frequency is set based on the sound source circuit of the audio signal and the frequency band of the audio signal estimated from the sound source circuit (for example, the frequency band of the human voice if the sound source circuit is a navigation device). The cutoff frequency can be arbitrarily set by the listener operating the operating device 32.

[0095] The gain circuit 1821 c is configured to perform weighting of the attack component of the amplitude spectrum detected by the limiter 1821 b and output the amplitude spectrum to the adder 1823 .

[0096] In addition to the weighted attack component amplitude spectrum fmA, adder 1823 also receives the unweighted amplitude spectrum fmA from frequency axis weight calculator 180A. Adder 1823 adds these amplitude spectra. Therefore, when the attack component weight is positive, the attack sound of amplitude spectrum fmA is enhanced, while when the weight is negative, the attack sound is reduced. In other words, the greater the positive value of the weight, the greater the attack sound is enhanced, while the greater the negative value of the weight, the greater the attack sound is reduced.

[0097] For example, the weighting value for attack sounds can be set to a value of -1 or greater, or +1 or less. For example, the weighting value is set based on the sound source circuit of the audio signal and the frequency band of the audio signal estimated from the sound source circuit. The listener can arbitrarily set the weighting value by operating the operating device 32.

[0098] like Figure 10 As shown, the reverberation controller 1822 includes a HPF 1822a, an amplitude inverter 1822b, a limiter 1822c, and a gain circuit 1822d.

[0099] For each magnitude spectrum, the HPF 1822a is configured to perform a high-pass filtering process on the input magnitude spectrum fmA. The magnitude inverter 1822b multiplies the magnitude spectrum of the high-pass filtering at the HPF 1822a by -1 to invert its magnitude.

[0100] The limiter 1822c is configured to set the amplitude of the negative side of the amplitude spectrum after the amplitude inversion to 0. By setting the amplitude of the negative side to 0, the descending component of the signal of each amplitude spectrum, that is, the reverberation component (reverberation) that lasts after the attack sound, can be detected.

[0101] The higher the cutoff frequency value set in the HPF 1822a, the shorter the reverberation control time. In other words, the lower the value, the longer the reverberation control time. For example, the cutoff frequency is set based on the audio signal's source circuit and the frequency band of the audio signal estimated from the source circuit. The listener can arbitrarily set the cutoff frequency by operating the operating device 32.

[0102] The gain circuit 1822 d is configured to perform weighting of the reverberation component of the magnitude spectrum detected by the limiter 1822 c and output the magnitude spectrum to the adder 1823 .

[0103] In addition to the reverberation component weighted magnitude spectrum fmA, adder 1823 receives the unweighted magnitude spectrum fmA from frequency axis weight calculator 180A. Adder 1823 is configured to add these magnitude spectra. Therefore, when the reverberation component weight is positive, the reverberation is enhanced relative to magnitude spectrum fmA, while when the weight is negative, the reverberation is reduced. In other words, the greater the positive value of the weighting, the greater the reverberation is enhanced, while the greater the negative value of the weighting, the greater the reverberation is reduced.

[0104] For example, the reverberation weighting value is set to a value of -1 or greater and +1 or less. For example, the weighting value is set based on the sound source circuit of the audio signal and the frequency band of the audio signal estimated from the sound source circuit. The weighting value can be arbitrarily set by the listener operating the operating device 32.

[0105] The adder 1823 is configured to add the magnitude spectrum weighted for the attack component and the magnitude spectrum weighted for the reverberation component to the magnitude spectrum fmA to obtain a magnitude spectrum in which the attack sound and reverberation are enhanced or reduced, and output the obtained magnitude spectrum to the limiter 1824 .

[0106] The limiter 1824 is configured to set the amplitude of the negative side of the amplitude spectrum input from the adder 1823 to zero and output the amplitude spectrum (i.e., the amplitude spectrum ftmA weighted in consideration of frequency masking and time masking) to each of the control gain calculator 183 and the spectrum controllers 20A and 20B. It should be noted that the time axis weight calculator 182B outputs only the amplitude spectrum ftmB to the control gain calculator 183.

[0107] Therefore, the time axis weight calculator 182A can adjust the duration of the rising component (i.e., the attack component) and the duration of the falling component (i.e., the reverberation component) of the audio signal (i.e., the priority audio signal in the time axis weight calculator 182A) by changing the amplitude spectrum fmA input from the frequency axis weight calculator 180A (more specifically, the amplitude spectrum fmA of the n frequency spectra obtained by the FFT circuit 10A) according to the cutoff frequencies of the HPF 1821a and the HPF 1822a and the weighted amounts of the attack sound and the reverberation.

[0108] For example, the time axis weight calculator 182B can simulate the range of the music masking the audio guide on the time axis by extending the duration of the falling component of the non-priority audio signal. Therefore, it can be seen that the time axis weight calculators 182A and 182B weight the amplitude spectrum within the control target frequency band while taking time masking into account.

[0109] According to the present embodiment, in order to obtain the effect of suppressing the masking of the audio guidance by the music to ensure the audibility of the audio guidance, the amplitude spectra aA and aB within the control target frequency band are weighted in consideration of both frequency masking and time masking. In another embodiment, in order to obtain this effect, only one of the frequency masking and time masking may be considered to weight one or both of the amplitude spectra aA and aB within the control target frequency band. In such an embodiment, the same effect can be obtained (however, a better effect will be obtained by weighting both the amplitude spectra aA and aB within the control target frequency band in consideration of both frequency masking and time masking).

[0110] As described above, the weight calculator 18 operates as a weighting circuit that performs weighting on at least one of the amplitude spectra aA and aB within the control target frequency band, while taking into account at least one of frequency masking and time masking. More specifically, the frequency axis weight calculators 180A and 180B perform weighting while taking into account frequency masking to widen the shapes of the amplitude spectra aA and aB within the control target frequency band, respectively, so that each shape simulates the range of other sounds on the frequency axis that are masked by the sound having the amplitude spectrum aA or aB. In addition, the frequency axis weight calculators 180A and 180B weight the amplitude spectra aA and aB within the control target frequency band, respectively, so as to further widen the bandwidth as the frequency increases. In addition, the time axis weight calculators 182A and 182B widen the shapes of the amplitude spectra aA and aB within the control target frequency band, respectively, by performing weighting while taking into account time masking, so that each shape simulates the range of other sounds on the time axis that are masked by the sound having the amplitude spectrum aA or aB.

[0111] Figure 11A and Figure 11B The function of the control gain calculator 183 is shown. Figure 11A and Figure 11B As shown, the control gain calculator 183 has a first gain function and a second gain function. The control gain calculator 183 is configured to calculate the amplitude ratio between the amplitude spectrum ftmA input from the time axis weight calculator 182A and the amplitude spectrum ftmB input from the time axis weight calculator 182B for each frequency. The control gain calculator 183 then inputs the calculated amplitude ratio into the first gain function to obtain the control gain gA for the priority audio signal, and also inputs the amplitude ratio into the second gain function to obtain the control gain gB for the non-priority audio signal. The control gain calculator 183 outputs the control gain gA to the spectrum controller 20A and outputs the control gain gB to the spectrum controller 20B.

[0112] That is, the control gain calculator 183 functions as a calculator for calculating a specific control value based on the amplitude ratio in the control target frequency band between the priority audio signal and the non-priority audio signal after the weighting process.

[0113] According to the first gain function and the second gain function, as the level of the priority audio signal becomes lower than the level of the non-priority audio signal, the control gain gA becomes higher and the control gain gB becomes lower. In other words, as the level of the priority audio signal becomes higher than the level of the non-priority audio signal, the control gain gA becomes lower and the control gain gB becomes higher. It should be noted that the control gain gA is a positive gain and the control gain gB is a negative gain.

[0114] Figure 12A and Figure 12B1 and 2 are block diagrams showing the functional configurations of the spectrum controllers 20A and 20B, respectively. Figure 12A As shown, the spectrum controller 20A includes an amplitude controller 200A and a phase controller 202A. Figure 12B As shown, the spectrum controller 20B includes an amplitude controller 200B and a phase controller 202B.

[0115] In the amplitude controller 200A, the amplitude spectrum aA is input from the amplitude calculator 16A, the amplitude spectrum ftmA is input from the time axis weight calculator 182A, and the control gain gA is input from the control gain calculator 183. In the phase controller 202A, the phase spectra pA and pB are input from the phase calculators 14A and 14B, respectively, the amplitude spectrum ftmA is input from the time axis weight calculator 182A, and the control gain gA is input from the control gain calculator 183.

[0116] In the amplitude controller 200B, the amplitude spectrum aB is input from the amplitude calculator 16B, the amplitude spectrum ftmA is input from the time axis weight calculator 182A, and the control gain gB is input from the control gain calculator 183. In the phase controller 202B, the phase spectra pA and pB are input from the phase calculators 14A and 14B, respectively, the amplitude spectrum ftmA is input from the time axis weight calculator 182A, and the control gain gB is input from the control gain calculator 183.

[0117] Figure 13A and Figure 13B Block diagrams of the amplitude controller 200A and the phase controller 202A are shown separately.

[0118] like Figure 13A As shown, the amplitude controller 200A includes: multipliers 2000A and 2001A, a comparator 2002A, a negation circuit 2003A, a multiplier 2004A, and an adder 2005A.

[0119] The multiplier 2000A is configured to multiply the amplitude spectrum aA input from the amplitude calculator 16A by the control gain gA input from the control gain calculator 183 and output the multiplied amplitude spectrum aA to the multiplier 2001A.

[0120] The comparator 2002A is configured to compare the amplitude spectrum ftmA input from the time axis weight calculator 182A with the threshold K, and output 1 when the amplitude spectrum ftmA is greater than the threshold K, and output 0 when the amplitude spectrum ftmA is equal to or less than the threshold K.

[0121] Multiplier 2001A is configured to multiply the amplitude spectrum input from multiplier 2000A by the output from comparator 2002A (i.e., 0 or 1), and output the multiplication result to adder 2005A. That is, only when the amplitude spectrum ftmA is greater than the threshold value K, does multiplier 2001A output the amplitude spectrum aA multiplied by the control gain gA to adder 2005A.

[0122] The multiplier 2004A is configured to multiply the amplitude spectrum aA input from the amplitude calculator 16A by the output of the NOT circuit 2003A (i.e., the inverted output of the comparator 2002A), and output the multiplication result to the adder 2005A. That is, the multiplier 2004A outputs the amplitude spectrum aA, which has not been multiplied by the control gain gA, to the adder 2005A only when the amplitude spectrum ftmA is equal to or less than the threshold value K.

[0123] The adder 2005A is configured to add the amplitude spectrum input from the multiplier 2001A and the amplitude spectrum input from the multiplier 2004A, and output the added result to the complex number calculator 22A.

[0124] That is, the amplitude controller 200A amplifies the amplitude spectrum aA input from the amplitude calculator 16A by multiplying it by the control gain gA only within the frequency band where the amplitude spectrum ftmA is greater than the threshold value K, and outputs the amplified spectrum aA to the complex calculator 22A. Hereinafter, for the sake of convenience of description, the amplitude spectrum output to the complex calculator 22A will be referred to as "amplitude spectrum AA."

[0125] According to the present embodiment, the initial low-power frequency band of the audio guide (specifically, the frequency band in which the amplitude spectrum ftmA is equal to or lower than the threshold value K) is considered to be a frequency band of a sound component that is basically acceptable to the listener even if the listener cannot hear it, and the amplitude spectrum aA is not amplified by the control gain gA. In another embodiment, the entire control target frequency band of the audio guide can be amplified by the control gain gA.

[0126] Therefore, the amplitude controller 200A performs a multiplication process to amplify the amplitude of the priority audio signal and suppress the masking of the priority sound by the non-priority sound by using a control gain gA (i.e., a parameter whose value corresponds to the amplitude ratio of the amplitude spectrum ftmA and the amplitude spectrum ftmB weighted in consideration of frequency masking and time masking). However, in order to avoid making the non-priority sound difficult to hear by amplifying the amplitude of the priority audio signal, the amplitude controller 200A does not excessively amplify the amplitude of the priority audio signal while limiting the frequency band in which amplification is performed to the control target frequency band. Illustratively, when the amplitude spectrum ftmA is completely greater than the amplitude spectrum ftmB, the amplitude controller 200A does not amplify the amplitude of the priority audio signal.

[0127] like Figure 13B As shown, the phase controller 202A includes a weight coefficient calculator 2020A, complex number calculators 2021A and 2022A, an adder 2023A, a phase calculator 2024A, a comparator 2025A, a multiplier 2026A, a negation circuit 2027A, a multiplier 2028A, and an adder 2029A.

[0128] Figure 14A and Figure 14B The functions of the weight coefficient calculator 2020A are shown. Figure 14A and Figure 14B As shown, weight coefficient calculator 2020A includes weight function 1A and weight function 2A. Weight coefficient calculator 2020A is configured to input control gain gA from control gain calculator 183 to each of weight function 1A and weight function 2A to calculate weight coefficients mA and nA, and to output weight coefficient nA to complex number calculator 2022A while outputting weight coefficient mA to complex number calculator 2021A. It should be noted that each of weight coefficient mA and weight coefficient nA takes a value between 0 and 1 so that the sum of weight coefficient mA and weight coefficient nA is equal to 1.

[0129] The complex number calculator 2021A is configured to calculate a frequency spectrum cAA using the phase spectrum pA input from the phase calculator 14A and the weight coefficient mA input from the weight coefficient calculator 2020A (in this case, the weight coefficient mA is considered to be an amplitude spectrum), and output the calculated frequency spectrum cAA to the adder 2023A. The complex number calculator 2022A is configured to calculate a frequency spectrum cBA using the phase spectrum pB input from the phase calculator 14B and the weight coefficient nA input from the weight coefficient calculator 2020A (in this case, the weight coefficient nA is considered to be an amplitude spectrum), and output the calculated frequency spectrum cBA to the adder 2023A.

[0130] The adder 2023A is configured to add the frequency spectrum cAA and the frequency spectrum cBA respectively input from the complex calculators 2021A and 2022A, and output the addition result to the phase calculator 2024A.

[0131] Phase calculator 2024A is configured to obtain an argument in the complex plane from the result of adding spectrum cAA and spectrum cBA. Since the argument (i.e., phase) is calculated for each frequency, phase calculator 2024A obtains a phase spectrum in which spectrum cAA and spectrum cBA are synthesized.

[0132] The comparator 2025A is configured to compare the amplitude spectrum ftmA input from the time axis weight calculator 182A with the threshold L, and output 1 when the amplitude spectrum ftmA is greater than the threshold L, and output 0 when the amplitude spectrum ftmA is equal to or less than the threshold L.

[0133] Multiplier 2026A is configured to multiply the phase spectrum input from phase calculator 2024A by the output (i.e., 0 or 1) from comparator 2025A and output the multiplication result to adder 2029A. That is, multiplier 2026A outputs the phase spectrum input from phase calculator 2024A to adder 2029A only when the amplitude spectrum ftmA is greater than threshold value L.

[0134] Multiplier 2028A is configured to multiply the phase spectrum pA input from phase calculator 14A by the output of NOT circuit 2027A (i.e., the inverted output of comparator 2025A), and output the multiplication result to adder 2029A. That is, multiplier 2028A outputs phase spectrum pA to adder 2029A only when the amplitude spectrum ftmA is equal to or less than threshold value L.

[0135] Adder 2029A is configured to add the phase spectrum input from multiplier 2026A and the phase spectrum input from multiplier 2028A. Adder 2029A outputs the phase spectrum PA obtained by the addition to complex calculator 22A. In this embodiment, weight coefficient mA and weight coefficient nA are constants of 1 and 0, respectively. Therefore, phase spectrum PA is the same as phase spectrum pA input from phase calculator 14A to complex calculator 2021A.

[0136] According to this embodiment, the phase controller 202A does not change the phase of the priority audio signal to enhance the audibility of the priority sound. In another embodiment, the phase controller 202A can change the phase of the priority audio signal within a range in which the priority sound is not difficult to hear.

[0137] Figure 15A and Figure 15B Block diagrams of the amplitude controller 200B and the phase controller 202B are shown separately.

[0138] like Figure 15A As shown, the amplitude controller 200B includes: multipliers 2000B and 2001B, a comparator 2002B, a NOT circuit 2003B, a multiplier 2004B, and an adder 2005B. Figure 15BAs shown, phase controller 202B includes: weight coefficient calculator 2020B, complex number calculators 2021B and 2022B, adder 2023B, phase calculator 2024B, comparator 2025B, multiplier 2026B, NOT circuit 2027B, multiplier 2028B, and adder 2029B. It should be noted that the configuration of amplitude controller 200B is the same as that of amplitude controller 200A, and the configuration of phase controller 202B is the same as that of phase controller 202A. Therefore, detailed descriptions of amplitude controller 200B and phase controller 202B will be omitted.

[0139] The amplitude controller 200B is configured to reduce the amplitude spectrum aB input from the amplitude calculator 16B by multiplying the amplitude spectrum ftmA by the control gain gB within the frequency band where the amplitude spectrum ftmA is greater than the threshold K, and output the reduced amplitude spectrum aB to the complex calculator 22B. In the following, for ease of explanation, the amplitude spectrum output to the complex calculator 22B will be referred to as "amplitude spectrum AB."

[0140] The amplitude controller 200B is configured to reduce the amplitude of the non-priority audio signal by performing a multiplication process using a control gain gB (i.e., a parameter whose value corresponds to the amplitude ratio of the amplitude spectrum ftmA and the amplitude spectrum ftmB weighted in consideration of frequency masking and time masking), thereby suppressing the non-priority sound from masking the priority sound. However, in order to avoid making the non-priority sound difficult to hear by reducing the amplitude of the non-priority audio signal, the amplitude controller 200B does not excessively reduce the amplitude of the non-priority audio signal while limiting the frequency band in which the reduction is performed to the control target frequency band. Illustratively, when the amplitude spectrum ftmA is completely greater than the amplitude spectrum ftmB, the amplitude controller 200B does not reduce the amplitude of the non-priority audio signal.

[0141] In this embodiment, by amplifying the amplitude spectrum aA while simultaneously reducing the amplitude spectrum aB, the amplitude spectrum aA is relatively amplified in the controlled target frequency band, thereby suppressing the masking of the audio guidance by the music, thereby ensuring the audibility of the audio guidance. In another embodiment, only one of amplifying the amplitude spectrum aA and reducing the amplitude spectrum aB may be performed to relatively amplify the amplitude spectrum aA in the controlled target frequency band.

[0142] The adder 2029B is configured to add the phase spectrum output from the multiplier 2026B and the phase spectrum output from the multiplier 2028B, and output the phase spectrum PB obtained thereby to the complex number calculator 22B.

[0143] Figure 16A and Figure 16B The functions of the weight coefficient calculator 2020B are shown. Figure 16A and Figure 16BAs shown, weight coefficient calculator 2020B has weight function 1B and weight function 2B. Weight coefficient calculator 2020B is configured to calculate weight coefficient mB and weight coefficient nB (which are values ​​corresponding to control gain gB input from control gain calculator 183), and output weight coefficient mB to complex number calculator 2021B and weight coefficient nB to complex number calculator 2022B. Like weight coefficient mA and weight coefficient nA, weight coefficient mB and weight coefficient nB also take values ​​between 0 and 1 so that the sum of weight coefficient mB and weight coefficient nB is equal to 1.

[0144] According to weight function 1B and weight function 2B, the smaller the control gain gB is (in other words, the smaller the amplitude spectrum ftmA is relative to the amplitude spectrum ftmB), the larger the weight coefficient mB is, and the smaller the weight coefficient nB is.

[0145] The weight coefficient calculator 2020B is configured to calculate the spectrum cAB using the phase spectrum pA and the weight coefficient mB (here, the weight coefficient mB is considered to be the amplitude spectrum). When the amplitude spectrum ftmA is small relative to the amplitude spectrum ftmB, the weight coefficient mB has a larger value. In addition, the weight coefficient calculator 2020B is configured to calculate the spectrum cBB using the phase spectrum pB and the weight coefficient nB (here, the weight coefficient nB is considered to be the amplitude spectrum). When the amplitude spectrum ftmA is large relative to the amplitude spectrum ftmB, the weight coefficient nB has a smaller value. Therefore, in the control target frequency band, the phase spectrum pB of the non-priority audio signal matches the phase spectrum pA of the priority audio signal. By matching the phase spectrum pB with the phase spectrum pA, it is possible to prevent the priority sound from being offset by the non-priority sound and becoming difficult to hear.

[0146] Although it is ideal that the phase spectrum pB is completely consistent with the phase spectrum pA, when the phase spectrum pB is adjusted to be close to the phase spectrum pA, the effect of reducing the cancellation of the priority sound by the non-priority sound is obtained, thereby improving the audibility of the priority sound.

[0147] Each of the amplitude controllers 200A and 200B operates as an amplitude modification circuit, which is configured to change the amplitude spectrum of at least one of the priority audio signal and the non-priority audio signal based on the amplitude spectrum after weighting processing by the weighting circuit (more specifically, based on the control value obtained according to the amplitude spectrum after weighting processing) to relatively amplify the amplitude spectrum of the priority audio signal in the control target frequency band.

[0148] In addition, each of the phase controllers 202A and 202B operates as a phase converter, which is configured to change the phase spectrum of at least one of the priority audio signal and the non-priority audio signal based on the amplitude spectrum after the weighting operation by the weighting circuit (more specifically, based on the control value obtained according to the amplitude spectrum after the weighting operation) so that within the control target frequency band, the phase spectrum of the non-priority audio signal is close to the phase spectrum of the priority audio signal.

[0149] The complex calculator 22A is configured to calculate a frequency spectrum within the control target band based on the amplitude spectrum AA input from the amplitude controller 200A and the phase spectrum PA input from the phase controller 202A, and output the calculated frequency spectrum to the adder 24A.

[0150] The adder 24A is configured to add the spectrum in the control target band of the priority audio signal input from the complex calculator 22A and the spectrum in the band other than the control target band of the priority audio signal input from the BSF 122 of the band divider 12A, and output the addition result to the adder 26.

[0151] The complex calculator 22B is configured to calculate a frequency spectrum in the control target band based on the amplitude spectrum AB input from the amplitude controller 200B and the phase spectrum PB input from the phase controller 202B, and output the calculated frequency spectrum to the adder 24B.

[0152] The adder 24B is configured to add the spectrum in the control target band of the non-priority audio signal input from the complex calculator 22B and the spectrum in the band other than the control target band of the non-priority audio signal input from the BSF 122 of the band divider 12B, and output the addition result to the adder 26.

[0153] The adder 26 is configured to add the spectrum of the priority audio signal input from the adder 24A, whose amplitude within the control target frequency band has been changed, and the spectrum of the non-priority audio signal input from the adder 24B, whose amplitude and phase within the control target frequency band have been changed, and output the addition result to the IFFT circuit 28.

[0154] That is, the adder 26 operates as a mixer configured to mix the priority audio signal and the non-priority audio signal after a change process of the amplitude spectrum by the amplitude converter and a change process of the phase spectrum by the phase converter.

[0155] The IFFT circuit 28 is configured to convert the spectrum input from the adder 26 from the frequency domain to the time domain through an inverse short-time Fourier transform process, perform overlapping processing and weighting processing using a window function, and output it to a speaker (not shown) embedded in the vehicle. In other words, the IFFT circuit 28 operates as a second converter, which is configured to overlap at least one of the priority audio signal and the non-priority audio signal after the amplitude spectrum change processing and the phase spectrum change processing by the spectrum controller 20A or 20B, and convert the spectrum into a time domain signal.

[0156] By the overlapping processing in the FFT circuits 10A and 10B and the IFFT circuit 28, discontinuities in the waveforms of the priority audio signal and the non-priority audio signal caused by changes in the amplitude spectrum and the phase spectrum in the spectrum controllers 20A and 20B are smoothed.

[0157] Therefore, according to this embodiment, the amplitude spectra of the audio guidance signal and the music signal are weighted, taking into account frequency and time masking. Based on the weighted amplitude spectrum (in other words, taking into account the masking between the audio guidance and the music), the amplitude of the audio guidance signal is reduced within the human voice frequency band (i.e., the control target frequency band), while the amplitude of the music is amplified, and the phase of the music signal is matched to the phase of the audio guidance signal. Therefore, when the music is played, the masking of the audio guidance by the music is suppressed, and the audibility of the audio guidance can be ensured.

[0158] In this embodiment, the mixing process is performed using data of a frequency spectrum with a small number of dimensions. Therefore, the processing load is suppressed compared to the processing load of the conventional mixing process in which the mixing process is performed using data on a time-frequency plane with a large number of dimensions.

[0159] 17A to 20B A specific processing example is shown.

[0160] Figure 17A Shown is the amplitude spectrum of the priority audio signal on which the mixing process according to the present embodiment is not performed. Figure 17B : is a graph showing the amplitude spectrum of the priority audio signal on which the mixing process according to the present embodiment has been performed. Figure 18A Shown is the amplitude spectrum of a non-priority audio signal on which the mixing process according to the present embodiment has not been performed. Figure 18B Shown is the amplitude spectrum of the non-priority audio signal on which the mixing process according to the present embodiment has been performed.

[0161] exist Figure 17A 、 Figure 17B 、 Figure 18A and Figure 18B (further described later) Figure 20A and Figure 20B), each vertical axis represents frequency (unit: kHz), and each horizontal axis represents time (unit: second). In these figures, a higher power frequency band is represented by a higher brightness, and a lower power frequency band is represented by a lower brightness.

[0162] Compare Figure 17A and Figure 17B It can be seen that Figure 17B The control target frequency band is white as a whole (i.e., represented by a higher brightness), that is, the amplitude is amplified. Figure 18A and Figure 18B It can be seen that Figure 18B The control target frequency band is entirely black (ie, represented by a lower brightness), that is, the amplitude is cut.

[0163] Figure 19 : shows the phase difference between the phase spectrum of the non-priority audio signal on which the mixing according to the present embodiment is not performed and the phase spectrum of the non-priority audio signal on which the mixing according to the present embodiment is performed. Figure 19 In the figure, the vertical axis represents frequency (unit: kHz) and the horizontal axis represents time (unit: seconds). Figure 19 In the display, a frequency band having a larger phase difference between a frequency band subjected to the mixing process according to this embodiment and a frequency band subjected to the mixing process not according to this embodiment (i.e., the amount of phase change given to a non-priority audio signal in the mixing process according to this embodiment) is indicated with a higher brightness, and a frequency band having a smaller phase difference is indicated with a lower brightness. A frequency band having a phase difference of zero is displayed in black.

[0164] like Figure 19 As shown, it can be seen that the phase difference occurs in the control target frequency band.

[0165] Figure 20A A mixed signal obtained by mixing a priority audio signal on which the mixing process according to the present embodiment is not performed and a non-priority audio signal on which the mixing process according to the present embodiment is not performed is shown. Figure 20B shows when priority audio signal is applied Figure 17B The amplitude shown is amplified and applied to the non-priority audio signal Figure 18B The reductions shown and Figure 19 The mixed signal of the priority audio signal and the non-priority audio signal is output from the IFFT circuit 28 when the phase is changed as shown.

[0166] Compare Figure 20A and Figure 20B, it can be seen that in the mixed signal output by IFFT circuit 28, the spectrogram shape of the priority audio signal is clearly shown, and the amplitude of the non-priority audio signal is not significantly reduced overall. This is because in the control target frequency band, the amplitude of the priority audio signal is amplified, the amplitude of the non-priority audio signal is reduced, and the phase of the non-priority audio signal matches the phase of the priority audio signal. Therefore, it can be understood that when the non-priority sound is played, the non-priority sound is suppressed from masking the priority sound, and the audibility of the priority sound is guaranteed.

[0167] The above is a description of an exemplary embodiment of the present invention. It should be noted that the embodiments of the present invention are not limited to the above embodiments, and various modifications can be made within the scope of the technical concept of the present invention. For example, in the embodiments of the present application, appropriate combinations of the examples, obvious examples, etc. exemplarily described in the specification are included.

[0168] Incidentally, each process in the sound mixing apparatus 1 is performed by cooperation between software and hardware provided in the sound mixing apparatus 1. At least the OS (Operating System) of the software included in the sound mixing apparatus 1 is provided as an embedded system, but other components (for example, a software module for performing mixing of priority audio signals and non-priority audio signals) may be provided as applications that can be distributed on a network or saved in a recording medium such as a memory card.

Claims

1. A sound mixing device configured to mix a first audio signal representing a first sound with a second audio signal representing a second sound so that a listener can listen to the first sound in a manner that gives priority to the second sound, the sound mixing device comprising: a weighting circuit configured to weight the amplitude spectrum of at least one of the first audio signal and the second audio signal in consideration of at least one of frequency masking and time masking within a specific control target frequency band; an amplitude converter configured to perform an amplitude spectrum changing process of relatively amplifying the amplitude spectrum of the first audio signal within the control target frequency band by performing one or both of amplification of the amplitude spectrum of the first audio signal and reduction of the amplitude spectrum of the second audio signal based on the amplitude spectrum to which weighting is applied; a phase converter configured to perform a phase spectrum changing process of bringing a phase spectrum of the second audio signal close to a phase spectrum of the first audio signal within a control target frequency band by changing a phase spectrum of at least one of the first audio signal and the second audio signal based on an amplitude spectrum to which weighting is applied; as well as A mixer is configured to mix the first audio signal and the second audio signal after the amplitude spectrum changing process and the phase spectrum changing process have been performed.

2. The sound mixing device according to claim 1, wherein: The weighting circuit weights the amplitude spectrum by considering frequency masking, widening the shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal within the control target frequency band so that the shape simulates a range in which the sound having the amplitude spectrum masks other sounds on the frequency axis.

3. The sound mixing device according to claim 1 or 2, wherein: The weighting that takes frequency masking into consideration is a weighting that more broadens the bandwidth of the shape of the amplitude spectrum as the frequency increases.

4. The sound mixing device according to any one of claims 1 to 2, wherein: The weighting circuit weights the amplitude spectrum by considering time masking, widening the shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal within the control target frequency band so that the shape simulates a range in which the sound having the amplitude spectrum masks other sounds on the time axis. 5 . The sound mixing apparatus according to claim 1 , further comprising a calculator configured to calculate a specific control value based on an amplitude ratio of the first audio signal and the second audio signal to which weighting is applied within a control target frequency band.

6. The sound mixing device according to claim 5, wherein: The amplitude converter amplifies the amplitude spectrum of the first audio signal and cuts the amplitude spectrum of the second audio signal within a control target frequency band based on the specific control value.

7. The sound mixing device according to claim 5 or 6, wherein: The phase converter changes the phase spectrum of the second audio signal based only on the specific control value, and matches the phase spectrum of the second audio signal with the phase spectrum of the first audio signal within a control target frequency band.

8. The sound mixing device according to any one of claims 1, 2 and 5, further comprising: a first converter configured to overlap each of the first audio signal and the second audio signal to convert to a frequency domain signal; as well as a second converter configured to overlap each of the first audio signal and the second audio signal to convert into a time domain signal after an amplitude spectrum changing process of the amplitude converter and a phase spectrum changing process of the phase converter, wherein the weighting circuit weights the amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into a frequency domain signal by the first converter; wherein the amplitude converter changes the amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into the frequency domain signal by the first converter; The phase converter changes the phase spectrum of at least one of the first audio signal and the second audio signal which have been converted into the frequency domain signal by the first converter.

9. A sound mixing method for mixing a first audio signal representing a first sound with a second audio signal representing a second sound so that a listener can listen to the first sound in a manner that gives priority to the second sound, the sound mixing method comprising: a weighting step of weighting the amplitude spectrum of at least one of the first audio signal and the second audio signal in consideration of at least one of frequency masking and time masking within a specific control target frequency band; an amplitude conversion step of performing an amplitude spectrum changing process of relatively amplifying the amplitude spectrum of the first audio signal within the control target frequency band by performing one or both of amplification of the amplitude spectrum of the first audio signal and reduction of the amplitude spectrum of the second audio signal based on the amplitude spectrum to which weighting is applied; a phase transforming step of performing a phase spectrum changing process of bringing a phase spectrum of the second audio signal close to a phase spectrum of the first audio signal within a control target frequency band by changing a phase spectrum of at least one of the first audio signal and the second audio signal based on an amplitude spectrum to which weighting is applied; A mixing step of mixing the first audio signal and the second audio signal after the amplitude spectrum changing process and the phase spectrum changing process have been performed.

10. The audio mixing method according to claim 9, wherein: In the weighting step, the amplitude spectrum is weighted by considering frequency masking, and the shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal is widened within the control target frequency band so that the shape simulates a range in which the sound having the amplitude spectrum masks other sounds on the frequency axis.

11. The mixing method according to claim 9 or 10, wherein: The weighting that takes frequency masking into consideration is a weighting that more broadens the bandwidth of the shape of the amplitude spectrum as the frequency increases.

12. The sound mixing method according to any one of claims 9 to 10, wherein: In the weighting step, the amplitude spectrum is weighted by considering time masking, and the shape of the amplitude spectrum of at least one of the first audio signal and the second audio signal is widened within the control target frequency band so that the shape simulates the range of other sounds masked by the sound having the amplitude spectrum on the time axis.

13. The sound mixing method according to claim 9, further comprising a calculation step of calculating a specific control value based on an amplitude ratio of the first audio signal and the second audio signal to which weighting is applied within a control target frequency band.

14. The audio mixing method according to claim 13, wherein: In the amplitude conversion step, the amplitude spectrum of the first audio signal is amplified and the amplitude spectrum of the second audio signal is reduced within the control target frequency band based on the specific control value.

15. The sound mixing method according to claim 13 or 14, wherein: In the phase converter, the phase spectrum of the second audio signal is changed based only on the specific control value, and the phase spectrum of the second audio signal is matched with the phase spectrum of the first audio signal within the control target frequency band.

16. The sound mixing method according to any one of claims 9, 10 and 13, further comprising: a first conversion step of overlapping each of the first audio signal and the second audio signal to convert into a frequency domain signal; as well as a second conversion step of overlapping each of the first audio signal and the second audio signal to convert into a time domain signal after the amplitude spectrum change process in the amplitude conversion step and the phase spectrum change process in the phase conversion step, wherein, in the weighting step, the amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into a frequency domain signal by the first converter is weighted; In the amplitude conversion step, an amplitude spectrum of at least one of the first audio signal and the second audio signal that have been converted into the frequency domain signal by the first converter is changed; In the phase transforming step, a phase spectrum of at least one of the first audio signal and the second audio signal, which have been converted into a frequency domain signal by the first converter, is changed.

Images