Acoustic signal processing device, acoustic signal processing method, and acoustic signal processing program

Abstract

This allows for the assurance of good call quality across multiple frequency bands without the need for tuning for each individual frequency band. [Solution] This is an acoustic signal processing device that transmits a signal in a predetermined frequency band, the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device. It comprises a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band wider than the first frequency band, based on a reference signal transmitted through the receiving-side signal path, and a band-specific countermeasure unit that takes countermeasures according to the frequency band detected by the band detection unit.

Description

Technical Field

[0001] The present invention relates to an acoustic signal processing apparatus, an acoustic signal processing method, and an acoustic signal processing program.

Background Art

[0002] In Patent Document 1, every time a sample point of a received signal transmitted through a received signal path that transmits a signal to a sound generation unit is acquired, based on the received signal acquired within a predetermined period before the time when the sample point is acquired, an optimal mask is sequentially generated or selected from basic masks that are one or more masks generated based on a learning signal. Every time the optimal mask is generated or selected, it is sequentially detected whether it is in a double-talk state based on the result of comparing the input signal with the optimal mask. When it is detected that no speech is input to the sound detection sensor and the received signal includes speech, an acoustic signal processing apparatus that sequentially performs a process of suppressing echo on the input signal is disclosed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The frequency band of a communication network varies depending on the country, region, communication carrier, usage environment, etc., and the performance also differs. In a conventional apparatus such as the invention described in Patent Document 1, since it is necessary to perform pre-learning in the frequency band of all assumed conditions and perform tuning according to the situation of the communication network, region, etc., there is a problem that an enormous amount of tuning time is required.

[0005] This invention has been made in view of these circumstances, and aims to provide an acoustic signal processing device, an acoustic signal processing method, and an acoustic signal processing program that can ensure good call quality across multiple frequency bands without tuning for each frequency band. [Means for solving the problem]

[0006] To solve the above problems, the acoustic signal processing device according to the present invention is an acoustic signal processing device that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, and is characterized by comprising: a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path; and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit.

[0007] To solve the above problems, the present invention provides an acoustic signal processing method that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, and is characterized by including the steps of detecting whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path, and taking countermeasures according to the detected frequency band.

[0008] To solve the above problems, the acoustic signal processing program according to the present invention is an acoustic signal processing program that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generation unit via a receiving-side signal path to produce sound from the sound generation unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, wherein the computer comprises a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path, and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit. It is characterized by being designed to function as such.

[0009] According to the above embodiment of the present invention, based on the reference signal transmitted through the receiver-side signal path, it is detected whether the receiver frequency band is a first frequency band (narrowband) or a second frequency band (wider frequency band than the first frequency band), and countermeasures are taken according to the detected frequency band.

[0010] Conventionally, tuning was performed in advance for each frequency band, and the acoustic signal processing device 1 performed acoustic processing based on the results. However, this was time-consuming and costly, and required highly skilled personnel and time. Therefore, tuning was sometimes performed in a wider second frequency band, and the tuning results from the wider frequency band were used for the narrower first frequency band. However, when nonlinear echoes or reflection components are large, nonlinear echoes are likely to occur due to distortion and vibration of the speaker and amplifier, and reflections occur inside the enclosure, so the tuning results from the second frequency band may not be usable in the first frequency band.

[0011] In contrast, according to the above embodiment of the present invention, instead of using the tuning results in the second frequency band in the first frequency band, countermeasures are taken according to the detected frequency band, thus ensuring good call quality across multiple frequency bands without tuning for each frequency band.

[0012] The system includes a frequency analyzer that converts the reference signal into a frequency domain signal to obtain a bandwidth detection signal. The bandwidth detection unit detects whether the signal is in the first frequency band or the second frequency band based on the magnitude of the signal in a predetermined range of frequency bands within the bandwidth detection signal. The predetermined range may be a range higher than the upper limit of the frequency bin of the frequency band to be detected among the first and second frequency bands. By using a frequency band signal, high frequency resolution can be achieved, and the accuracy of bandwidth detection can be increased. Furthermore, by using a range higher than the upper limit of the frequency bin of the frequency band to be detected, the processing load can be reduced.

[0013] The bandwidth detection unit may detect that the receiving frequency band is the first frequency band if, in a range higher than the upper limit of the frequency bins of the receiving frequency band, the sum of the magnitudes of the bandwidth detection signals is less than or equal to a first threshold. This enables accurate bandwidth detection.

[0014] The system includes a double-talk detection unit that sequentially detects whether or not speech has been input to the sound detection sensor based on a double-talk detection mask, and a nonlinear echo suppression unit that performs processing to suppress nonlinear echoes in the signal from which linear echoes have been removed. In the case of single-talk (receiving only) and the receiving frequency band is the first frequency band, the nonlinear echo suppression unit may suppress nonlinear echoes more strongly for frequency bins not included in the first frequency band than for frequency bins included in the first frequency band. In other words, as a countermeasure according to the detected frequency band, the suppression of nonlinear echoes is changed according to the frequency band (first frequency band and second frequency band). This ensures that unwanted nonlinear echoes are reliably suppressed.

[0015] The double-talk detection unit may, when the receiving frequency band is the first frequency band, use a first detection mask to detect whether or not speech has been input from the sound detection sensor for frequency bins included in the first frequency band, and use a second detection mask larger than the first detection mask to detect whether or not speech has been input from the sound detection sensor for frequency bins not included in the first frequency band. In other words, as a countermeasure corresponding to the detected frequency band, the mask for double-talk detection is changed according to the frequency band (first frequency band and second frequency band). This prevents false detection of whether or not speech has been transmitted.

[0016] The device includes a volume adjustment unit that adjusts the volume of the signal transmitted through the receiver signal path, and the band-specific countermeasure unit may control the volume adjustment unit to gradually reduce the volume to a predetermined value when it detects that the receiver frequency band is the second frequency band. In this way, the overall volume can be reduced to improve the call quality.

[0017] The system includes a compressor for compressing the signal transmitted through the receiver signal path, and the band-specific countermeasures unit may control the compressor to gradually lower the compressor threshold at which the compressor begins compression when it is detected that the receiver frequency band is the second frequency band. In this way, the overall volume can be reduced to improve the call quality.

[0018] The system includes a linear echo removal unit that suppresses linear echoes generated when the sound output from the sound generation unit is input to the sound detection sensor. The linear echo removal unit may remove linear echoes using the results of a learning process performed with a first step size when it is detected that the receiving frequency band is the first frequency band, and may remove linear echoes using the results of a learning process performed with a second step size larger than the first step size when it is detected that the receiving frequency band is the second frequency band. In this way, by increasing the step size and bringing the echo output closer to zero, distortion can be brought closer to zero and the call quality can be improved.

[0019] The per-band countermeasure unit generates the first detection mask and the second detection mask by using an estimated echo function calculated based on a second learning received signal obtained by converting a learning received signal transmitted through the received signal path into a frequency domain, and a second learning signal obtained by converting a learning signal transmitted through the transmission signal path when sound output from the sound generation unit enters the sound detection sensor by the learning received signal. The estimated echo function uses, as variables, the logarithm of the magnitude at each frequency of the reference signal, the frequency of the reference signal, the logarithm of the total received value that is the sum of the magnitudes of the reference signal or the sum of the reference signals in an arbitrary frequency range, and the logarithm of the envelope of the total received value. The first detection mask may be generated by inputting a signal generated using the sound source in the second frequency band into the received signal path. Thereby, an accurate first detection mask and second detection mask can be obtained.

[0020] The second detection mask is generated by adding an addition value to the first detection mask. The addition value may be generated by inputting a signal generated using the sound source in the first frequency band into the received signal path and acquiring an echo signal in the frequency band of the second frequency band. Thereby, the parameters (coefficients) can be reduced as compared with the case where the first detection mask and the second detection mask are generated separately.

Advantages of the Invention

[0021] According to the present invention, good call quality can be ensured in a plurality of frequency bands without performing tuning for each frequency band.

Brief Description of the Drawings

[0022] [Figure 1] FIG. schematically shows an audio communication system 100 including an acoustic signal processing device 1. [Figure 2] FIG. shows an outline of functional blocks of the acoustic signal processing device 1. [Figure 3]This is a flowchart showing the processing flow (Method 1) of the bandwidth detection unit 13. [Figure 4] This diagram schematically illustrates the process (Method 1) for determining the frequency band. [Figure 5] This is a flowchart showing the processing flow (Method 2) of the bandwidth detection unit 13. [Figure 6] This diagram schematically illustrates the process (Method 2) for determining the frequency band. [Figure 7] This is a flowchart showing the processing flow (Method 3) of the bandwidth detection unit 13. [Figure 8] This diagram schematically illustrates the process for determining the frequency band (Method 3). [Figure 9] This diagram schematically illustrates the processing performed by the double-talk detection unit 15. [Figure 10] This diagram shows a schematic of the functional blocks used to determine the function for calculating estimated echoes in the acoustic signal processing device 1. [Figure 11] This is an example of a scatter plot of the logarithmic power spectrum of the learning receiver signal at each frequency and the logarithmic power spectrum of the learning signal at each frequency. [Figure 12] This is an example of a scatter plot of the logarithm of the frequency of the learning receiver signal and the logarithm of the power spectrum of the transmitted signal. [Figure 13] This is an example of a scatter plot of the logarithm of the total received power spectrum and the logarithm of the transmitted power spectrum of a learning received signal. [Figure 14] This is an example of a scatter plot of the logarithm of the envelope of the total received power spectrum of a learning received signal and the logarithm of the transmitted power spectrum. [Figure 15] This flowchart shows the processing flow for double talk detection and nonlinear echo suppression depending on the bandwidth. [Figure 16] This flowchart shows the sequence of processes for determining the value to be added to the threshold T3. [Figure 17] This flowchart shows the sequence of processes for determining thresholds T3 and T4. [Figure 18] This diagram shows a schematic representation of the functional blocks of the acoustic signal processing device 2. [Figure 19] This diagram shows a schematic representation of the functional blocks of the acoustic signal processing device 3. [Figure 20] This diagram shows a schematic representation of the functional blocks of the acoustic signal processing device 4. [Figure 21] This graph shows the frequency characteristics of the reference signal input from the communication device 53 to the acoustic signal processing device 1, with the horizontal axis representing frequency and the vertical axis representing magnitude. (B) is an enlarged view of a portion of (A). [Figure 22] This is an example of the relationship between the reference signal and the echo signal when learning (tuning) in wideband mode. (A) shows the case when a wideband reference signal is input, and (B) shows the case when a narrowband reference signal is input. [Modes for carrying out the invention]

[0023] Hereinafter, embodiments of the acoustic signal processing device according to the present invention will be described in detail with reference to the drawings. <First Embodiment> Figure 1 is a schematic diagram showing a voice communication system 100 including an acoustic signal processing device 1 according to the first embodiment. The voice communication system 100 mainly comprises a terminal 50 provided in a closed space 90, the acoustic signal processing device 1, and a communication device 53.

[0024] Terminal 50 has a sound detection sensor 51 (corresponding to the first sound detection sensor of the present invention) and a sound generation unit 52 (corresponding to the first sound generation unit of the present invention). The sound detection sensor 51 is a sensor capable of detecting sound, such as a microphone or a vibration sensor. The sound generation unit 52 is a component that generates sound by vibration or the like, such as a speaker or an exciter that produces sound by vibrating a contact surface. In this embodiment, each terminal 50 has one sound detection sensor 51 and one sound generation unit 52, but the number of sound detection sensors 51 and sound generation units 52 that a terminal 50 has is not limited to this.

[0025] The enclosed space 90 is a closed indoor space that is not open, such as a car cabin, office, or room in a house. In this embodiment, the enclosed space 90 is a car cabin. The terminal 50 is located inside the enclosed space 90 and outputs sound into the enclosed space 90 or detects sound within the enclosed space 90.

[0026] It is not essential that the sound detection sensor 51 and the sound generation unit 52 constitute the terminal 50. For example, the sound detection sensor 51 and the sound generation unit 52 may be separate components, and they may be provided at different locations inside the closed space 90.

[0027] The sound detection sensor 51 and sound generation unit 52 of terminal 50 are each connected to the acoustic signal processing device 1. The acoustic signal processing device 1 is a device that includes a processing unit that executes a memory program containing a program for operating terminal 50, and is, for example, a DSP (Digital Signal Processor). The acoustic signal processing device 1 may be integrated with terminal 50 or may be provided in a separate location from terminal 50. Furthermore, the acoustic signal processing device 1 may be provided in a location other than the enclosed space 90.

[0028] The acoustic signal processing device 1 functionally comprises an acoustic processing unit and a parameter storage unit. The acoustic processing unit is the functional unit that runs the program that enables each functional block (see Figure 2) of the acoustic signal processing device 1 to function. The parameter storage unit stores various parameters used by the acoustic processing unit.

[0029] The acoustic signal processing device 1 is connected to the communication device 53. The acoustic signal processing device 1 is a device that transmits signals in a predetermined frequency band (corresponding to the receiving frequency band of the present invention) acquired via the communication device 53 to the sound generation unit 52 via the receiving-side signal path (described in detail later) to produce sound from the sound generation unit 52, and also acquires voice input via the sound detection sensor 51 via the transmitting-side signal path (described in detail later) and outputs it to the communication device 53.

[0030] The communication device 53 enables a near-end speaker A using terminal 50 to communicate with a far-end speaker B using terminal 54 (far-end terminal) located at a distance in the enclosed space 90. The sound generation unit 52 amplifies the voice signal from far-end speaker B input via terminal 54, and the sound detection sensor 51 collects the voice emitted by near-end speaker A on the near-end side and transmits it to far-end speaker B. This allows near-end speaker A to make a loudspeaker call (hands-free call) without holding terminal 50.

[0031] The communication device 53 enables voice communication using a predetermined frequency band. The predetermined frequency band is generally one of the following: narrowband (NB), wideband (WB), superwideband (SWB), or fullband (FB).

[0032] The frequency bands used for voice communication have expanded from narrowband to wideband, superwideband, and fullband. Currently, voice communication uses a variety of frequency bands, and the acoustic signal processing device 1 needs to support all of them. To this end, conventionally, tuning (adjustment work such as parameter setting) was performed for each frequency band in advance, the acoustic signal processing device 1 stored the results, and the acoustic signal processing device 1 performed acoustic processing based on those results.

[0033] However, increasing the number of tuning cycles increases the adjustment, evaluation, and management workloads, leading to increased time and costs. Furthermore, tuning requires specialized knowledge of signal processing, thus demanding highly skilled personnel and time. For this reason, attempts are being made to reduce the number of adjustments and parameters by performing tuning over a wider frequency band (e.g., full band) and using the tuning results from the wider frequency band for narrower frequency bands (e.g., narrowband).

[0034] However, when nonlinear echoes or reflection components are large, it can be difficult to use the results of tuning over a wide frequency range in a narrow frequency range. For example, vehicles are equipped with on-board devices in which the microphone and speaker are integrated into a single housing. However, on-board devices are prone to nonlinear echoes due to distortion and vibration of the speaker and amplifier, and reflections occur inside the housing, so the results of tuning over a wide frequency range may not be usable in a narrow frequency range. This will be explained using Figures 15 and 16.

[0035] Figure 21 is a graph showing the frequency characteristics of the reference signal input from the communication device 53 to the acoustic signal processing device 1, with the horizontal axis representing frequency and the vertical axis representing magnitude. Figure 21(B) is an enlarged view of a portion of Figure 21(A). In the in-vehicle hands-free emergency call system (ITU-T P.1140 NB / WB), the total volume of the wideband sound (gray line in Figure 21) and the total volume of the narrowband sound (black line in Figure 21) are the same (-16dB). m0 As a result, in the central frequency band, the narrowband sound is slightly louder than the wideband sound. For example, in automotive equipment, even a slight increase in volume in a specific frequency band can increase nonlinear echoes, so using the results of a wideband (e.g., wideband) adjustment in a narrowband (e.g., narrowband) environment may degrade call quality.

[0036] Figure 22 shows an example of the relationship between the reference signal and the echo signal when learning (tuning) in wideband mode. (A) shows the case when a wideband reference signal is input, and (B) shows the case when a narrowband reference signal is input. Here, the echo signal is the signal that is detected by the sound detection sensor 51 and input to the acoustic signal processing device 1 when the reference signal is output as sound from the sound generation unit 52 via the acoustic signal processing device 1. It includes both speech and distortion. In Figure 22, the gray line is the reference signal, and the black line is the echo signal. The horizontal axis in Figure 22 is frequency, and the vertical axis is magnitude. In Figure 22, the high-frequency band of 4 kHz is enclosed by a dotted line.

[0037] As shown in Figure 22(A), when the reference signal is wideband, the increase in the echo signal relative to the reference signal at 7kHz is an average of +14dB. In other words, at high frequencies, the sound (magnitude of the reference signal) is greater than the distortion (magnitude of the echo signal), and the distortion is less affected by the distortion because it is buried in the sound.

[0038] In contrast, as shown in Figure 22(B), when the reference signal is narrowband, no reference signal is input in the high-frequency band (it is -120 (negative infinity) in Figure 22), so the echo signal should also be -120. However, in reality, an echo signal of about -100 is input. This is a problem that occurs because the results of tuning for wideband are applied to narrowband, indicating that echo rejection learning has not been performed correctly. If noise (such as voice distortion) is introduced into the reference signal under these circumstances, the call quality may deteriorate.

[0039] In this embodiment, the acoustic signal processing device 1 can address this problem by detecting the bandwidth of the reference signal (receiver frequency band) and performing processing according to the detection result (details will be described later).

[0040] Returning to the explanation of Figure 1, the acoustic signal processing device 1 may consist of, for example, a computer system including an arithmetic unit such as a CPU (Central Processing Unit) for performing information processing, a storage device such as RAM (Random Access Memory) or ROM (Read Only Memory), a dedicated board mounted on various terminals (e.g., in-vehicle devices, conference systems, mobile terminals), and software (tuning program). The acoustic signal processing program may also be pre-stored in a storage medium such as an HDD built into a computer or ROM in a microcomputer with a CPU, and then installed on the computer from there. The acoustic signal processing program may also be temporarily or permanently stored (remembered) in a removable storage medium such as semiconductor memory, memory card, optical disk, magneto-optical disk, or magnetic disk.

[0041] Figure 2 is a schematic diagram of the functional blocks of the acoustic signal processing device 1. Functionally, the acoustic signal processing device 1 mainly comprises an echo removal unit 11, frequency analyzers (FFT units) 12 and 18, a bandwidth detection unit 13, a bandwidth-specific countermeasure unit 14, a double talk detection unit 15, a nonlinear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. In Figure 2, the upper signal path is the transmitting-side signal path that transmits the input signal received from the sound detection sensor 51, and the lower signal path is the receiving-side signal path that transmits the signal to the sound generation unit 52. Note that the functional components of the acoustic signal processing device 1 may be further classified into more components depending on the processing content, or one component may perform the processing of multiple components.

[0042] The echo removal unit 11 is a functional unit that removes echoes using, for example, an adaptive filter. The echo removal unit 11 removes echoes by updating filter coefficients according to a given procedure, generating a pseudo-echo signal from the signal transmitted through the receiver-side signal path, and subtracting the pseudo-echo signal from the signal transmitted through the transmitter-side signal path. Since adaptive filters are already well known, their explanation will be omitted.

[0043] In this embodiment, an adaptive filter is applied to the echo removal unit 11, but other known echo removal techniques can also be applied to the echo removal unit 11. Furthermore, although the echo removal unit 11 is not essential in this embodiment, it is desirable to provide it because it allows for more accurate detection of near-end utterances (utterances from near-end speaker A (see Figure 1)).

[0044] The frequency analyzers (FFT units) 12 and 18 are functional units that perform a Fast Fourier Transform (FFT) on a signal. The FFT unit 12 performs a Fast Fourier Transform on the signal transmitted through the transmitting signal path, in this case the signal that has passed through the echo rejection unit 11, and the FFT unit 18 performs a Fast Fourier Transform on the reference signal transmitted through the receiving signal path. The FFT units 12 and 18 convert a signal arranged in time series (time domain) into a signal represented by a set of frequencies (frequency domain). Hereinafter, a time-dependent signal is denoted by ...[t] and a frequency-dependent signal is denoted by ...[i].

[0045] The Fast Fourier Transform (FFT) is a computationally optimized version of the Discrete Fourier Transform (DFT) with a modified algorithm, resulting in faster calculations. Similar results can be obtained using either the FFT or the DFT. In this embodiment, the FFT is used as the frequency conversion method, but the DFT can also be used.

[0046] The bandwidth detection unit 13 is a functional unit that detects the frequency band of a signal acquired via the communication device 53 (corresponding to the first frequency band of the present invention) based on a reference signal transmitted through the receiver-side signal path. In this embodiment, the bandwidth detection unit 13 detects the frequency band based on a reference signal (corresponding to the bandwidth detection signal of the present invention) that has been converted into a frequency domain signal by the FFT unit 18. By using a reference signal converted into a frequency domain signal by the FFT unit 18, the bandwidth detection unit 13 can achieve high frequency resolution with low computational load, and as a result, the accuracy of bandwidth detection is improved.

[0047] For example, the bandwidth detection unit 13 detects whether the frequency band is a narrowband (corresponding to the first frequency band of the present invention) or a wideband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the narrowband. Alternatively, the bandwidth detection unit 13 may detect whether the frequency band is a narrowband wideband (corresponding to the first frequency band of the present invention) or a superwideband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the wideband, or whether the frequency band is a narrowband superwideband (corresponding to the first frequency band of the present invention) or a fullband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the superwideband. Furthermore, the bandwidth detection unit 13 can combine the above embodiments to detect which of the three frequency bands—narrowband, wideband, and superwideband—is being used, or which of the three frequency bands—wideband, superwideband, and fullband—is being used. The bandwidth detection unit 13 will be described in detail later.

[0048] The bandwidth-specific countermeasures unit 14 is a functional unit that performs countermeasures according to the detected frequency band based on the detection results of the bandwidth detection unit 13. For example, if the bandwidth-specific countermeasures unit 14 detects that the reference signal is a signal in a wider frequency band (second frequency band), it performs different processing (countermeasures for the second frequency band) than when it detects that the signal is in a narrower frequency band (first frequency band). The bandwidth-specific countermeasures unit 14 will be described in detail later.

[0049] The double talk detection unit 15 is a functional unit that detects whether or not a double talk state is in place based on the signal input from the FFT unit 12 and the reference signal input from the FFT unit 18. Here, a double talk state is a state in which both the near-end speaker A and the far-end speaker B are speaking, that is, a signal containing speech is being transmitted simultaneously to both the receiving-side signal path and the transmitting-side signal path.

[0050] The nonlinear echo suppression unit 16 performs echo suppression processing (processing that strongly suppresses echoes) on the input signal. The nonlinear echo suppression unit 16 performs different processing based on the detection result of the double talk detection unit 15 (described in detail later). For example, the nonlinear echo suppression unit 16 enables echo suppression processing when it is a single talk consisting only of far-end utterances, and disables it in other cases. Also, for example, the nonlinear echo suppression unit 16 strongly suppresses echoes when it is a single talk consisting only of far-end utterances, and weakly suppresses echoes in other cases. Since echo suppression processing is already well known, a detailed explanation will be omitted. The nonlinear echo suppression unit 16 receives the result of detecting whether or not a double talk state is present from the double talk detection unit 15 at unit time intervals. Therefore, the nonlinear echo suppression unit 16 switches between enabling and disabling echo suppression processing at unit time intervals.

[0051] Furthermore, the double-talk detection unit 15 and the nonlinear echo suppression unit 16 perform different processing depending on whether the signal is a narrowband signal (corresponding to the first frequency band of the present invention) or a signal with a wider frequency band (corresponding to the second frequency band of the present invention), based on the countermeasures taken by the band-specific countermeasures unit 14 (details will be described later).

[0052] The IFFT unit 17 performs an inverse FFT (IFFT) on the input signal that has passed through the nonlinear echo suppression unit 16.

[0053] The volume control unit 20 is a functional unit that adjusts the gain of the input signal. Specifically, the volume control unit 20 adjusts the degree to which it amplifies the input signal (gain) to output a signal of the desired level (magnitude). The volume control unit 20 adjusts the gain according to the magnitude of the input signal on the receiving side (communication device 53). The volume control unit 20 is, for example, an AGC (Automatic Gain Control) circuit, and since it is already known, its explanation is omitted. However, the volume control unit 20 is not essential in this embodiment.

[0054] The compressor 21 is a functional unit that adjusts the dynamic range (sound variation range) of the signal output from the volume control unit 20. The compressor 21 also changes its compression characteristics based on noise and other factors in the environment in which the terminal 50 is installed. Since the compressor 21 is already known, its description is omitted. However, the compressor 21 is not essential in this embodiment.

[0055] Next, the processing of the bandwidth detection unit 13 will be explained in detail using a specific example. The bandwidth detection unit 13 detects the frequency band of the reference signal using one of the following methods 1, 2, or 3.

[0056] <Method 1> Figure 3 is a flowchart showing the processing flow (Method 1) of the bandwidth detection unit 13. First, the bandwidth detection unit 13 associates the upper limit of each frequency band (i_Band[i]_Up) with the frequency bin (step S11). For example, i in the narrowband upper limit i_Band[0]_Up is 109, and i in the wideband i_Band[1]_Up is 224.

[0057] Next, the bandwidth detection unit 13 detects the largest frequency bin that exceeds the threshold T1 for the acquired reference signal (step S12). The threshold T1 may be set experimentally in advance.

[0058] Next, the bandwidth detection unit 13 compares the upper limit and frequency bin of each frequency band with the largest frequency bin that exceeds the threshold T1 to determine the frequency band of the reference signal (step S13). For example, if the largest frequency bin that exceeds the threshold T1 is less than or equal to the upper limit of an arbitrary frequency band (frequency band A), the bandwidth detection unit 13 determines that the frequency band of the reference signal is frequency band A. If the largest frequency bin that exceeds the threshold T1 is not less than or equal to the upper limit of an arbitrary frequency band (frequency band A), the unit performs a comparison and determination for the next frequency band.

[0059] The band detection unit 13 performs comparison starting from the frequency bin at the upper limit of a narrow frequency band. That is, first, it compares with the upper limit i_Band[0]_Up of the narrow band, then with the i_Band[1]_Up of the wide band, then with the i_Band[2]_Up of the super-wide band, and finally with the i_Band[3]_Up of the full band. Note that the comparison with the i_Band[3]_Up of the full band is not essential.

[0060] Regarding the process of step S13, a specific example will be used for explanation. FIG. 4 is a graph schematically showing the process of determining the frequency band. For example, in the graph of the black line in FIG. 4, for the signal shown, the maximum frequency bin i exceeding the threshold T1 is 108. First, the band detection unit 13 compares i = 108 with the upper limit i = 109 of the narrow band. Since the upper limit (i = 108) of this signal is smaller than the upper limit i = 109 of the narrow band, it is determined that this signal is a narrow-band signal (the maximum frequency bin < i_Band[0]_Up:NB exceeding the threshold T1).

[0061] For example, in the graph of the gray line in FIG. 3, for the signal shown, the maximum frequency bin i exceeding the threshold T1 is 223. First, the band detection unit 13 compares i = 223 with the upper limit i = 109 of the narrow band. Since the upper limit (i = 223) of this signal is larger than the upper limit i = 109 of the narrow band, it is detected that this signal is a signal in a frequency band wider than the narrow band. Next, the band detection unit 13 compares i = 223 with the upper limit i = 224 of the wide band, which is the next wider band after the narrow band. Since the upper limit (i = 223) of this signal is smaller than the upper limit i = 224 of the wide band, it is determined that this signal is a wide-band signal (i_Band[0]_Up ≤ the maximum frequency bin exceeding the threshold T1 < i_Band[1]_Up: WB).

[0062] Also, when the band detection unit 13 detects that a certain signal (signal I) is a signal in a frequency band wider than the wide band, the maximum frequency bin i of signal I exceeding the threshold T1 I is compared with the upper limit frequency bin of the super-wide band, which is the next wider band after the wide band. The frequency bin i of signal II A signal is determined to be a superwideband signal when its frequency is smaller than the upper frequency bin of the superwideband (i_Band[1]_Up ≤ the largest frequency bin above the threshold T1). <i_Band[2]_Up:SWB)。

[0063] Furthermore, if the bandwidth detection unit 13 detects that a certain signal (signal II) is a signal with a frequency band wider than the wideband, it will determine the largest frequency bin i of signal II that exceeds the threshold T1. II We compare this with the upper frequency bin of the superwideband, which is the next widest band, and the frequency bin i of signal II. II If the signal is greater than the upper frequency bin of the superwideband, it is determined that this signal is a full-band signal (the largest frequency bin above the threshold T1 ≥ i_Band[2]_Up:FB).

[0064] Note that the frequency bin i of signal II II When the frequency bin i is greater than the upper frequency bin of the super wideband, the bandwidth detection unit 13 detects the frequency bin i II We compare this with the upper frequency bin of the full band, which is the next widest band after the super wideband, and frequency bin i II A signal may be determined to be a full-band signal when it is smaller than the upper frequency bin of the full band (i_Band[2]_Up ≤ the largest frequency greater than or equal to T1). <i_Band[3]_Up:FB)。

[0065] Generally, the difference in frequency bands (the difference between narrowband and wideband) is mainly at the high-frequency end, with little difference at the low-frequency end. Furthermore, at the low-frequency end, the reference signal is often obscured by noise. Therefore, by comparing the upper limit and frequency bin of each frequency band with the largest frequency bin exceeding the threshold T1, the frequency band of the reference signal can be determined. Using only the high-frequency end allows for accurate determination while reducing processing load.

[0066] <Method 2> Figure 5 is a flowchart showing the processing flow (Method 2) of the bandwidth detection unit 13. First, the bandwidth detection unit 13 associates the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) of each frequency band with the frequency bin (Step S14).

[0067] Next, the bandwidth detection unit 13 obtains the number of frequency bins that exceed the threshold T2 for each frequency band, within the range outside the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) (below the lower limit, above the upper limit) for the acquired reference signal (step S15). The threshold T2 may be set experimentally in advance.

[0068] Next, the bandwidth detection unit 13 compares the number of frequency bins acquired in step S15 with a predetermined threshold N to determine the frequency bandwidth of the reference signal (step S16). For example, the bandwidth detection unit 13 determines that the frequency band is A if the number of frequency bins exceeding the threshold T2 outside the range outside of the arbitrary frequency band (frequency band A) acquired in step S15 is less than or equal to the threshold N. If the number exceeds the threshold N, it performs a comparison over a wider frequency band than frequency band A.

[0069] The bandwidth detection unit 13 performs comparisons in order from the narrowest frequency band. Specifically, it first compares the number of frequency bins above threshold T2 with threshold N in the range outside the narrowband, then compares the number of frequency bins above threshold T2 with threshold N in the range outside the wideband, then compares the number of frequency bins above threshold T2 with threshold N in the superwideband, and finally compares the number of frequency bins above threshold T2 with threshold N in the fullband. Note that fullband comparison is not mandatory.

[0070] FIG. 6 is a diagram schematically showing a process of obtaining the number of frequency bins exceeding a threshold value T2. In the case of the signal shown by the black line graph in FIG. 6, the number n1 of frequency bins exceeding the threshold value T2 in the range outside the lower limit (i_Band[0]_Lo = 2) and the upper limit (i_Band[0]_Up = 109) of the narrow band (the shaded portion in FIG. 6) is 1. Since the number n1 is smaller than the threshold value N (for example, 5), it can be seen that the signal shown by the black line graph in FIG. 6 is a narrow band (the number of values above the threshold value T2 below i_Band[0]_Lo and above i_Band[0]_Up < N: NB). Note that when the number of frequency bins exceeding the threshold value T2 in the range between the lower limit of the narrow band and the lower limit of the wide band and between the upper limit of the narrow band and the upper limit of the wide band is smaller than the threshold value N, it is also possible to determine that the signal is a narrow band (the number of values above the threshold value T2 below i_Band[0]_Lo and greater than i_Band[1]_Lo and above i_Band[0]_Up and less than i_Band[1]_Up (outside the band 1) < N: NB).

[0071] In the case of the signal shown by the gray line graph in FIG. 6, in the range outside the lower limit and the upper limit of the narrow band, the number n2 of frequency bins exceeding the threshold value T2 is 115. Since the number n2 is greater than the threshold value N, it can be seen that this signal is a signal in a frequency band wider than the narrow band. Next, for the signal shown by the gray line graph, the number n3 of frequency bins exceeding the threshold value T2 in the range outside the lower limit and the upper limit of the wide band is obtained. Since the number n3 (for example, 1) is smaller than the threshold value N, it can be seen that the signal shown by the gray line graph in FIG. 6 is a wide band (the number of values above the threshold value T2 below i_Band[1]_Lo and above i_Band[1]_Up < N: WB). Note that when the number of frequency bins exceeding the threshold value T2 in the range between the lower limit of the wide band and the lower limit of the super-wide band and between the upper limit of the wide band and the upper limit of the super-wide band is smaller than the threshold value N, it is also possible to determine that the signal is a wide band (the number of values above the threshold value T2 below i_Band[1]_Lo and greater than i_Band[2]_Lo and above i_Band[1]_Up and less than i_Band[2]_Up (outside the band 2) < N: WB).

[0072] Further, when the band detection unit 13 detects that a certain signal (signal I) is a signal in a frequency band wider than the wide band, for signal I, it obtains the number n4 of frequency bins exceeding the threshold value T2 in the range outside the lower limit and the upper limit of the super wide band, and when the number n4 is smaller than the threshold value N, it determines that signal I is in the super wide band (the number of thresholds T2 or more outside the band (band outside 3) at i_Band[2]_Up or more (<N:SWB)), and when the number n4 is greater than or equal to the threshold value N, it determines that signal I is in the full band (the number of thresholds T2 or more outside the band (band outside 3) at i_Band[2]_Up or more (≧N:FB)).

[0073] In addition, for signal I, the band detection unit 13 obtains the number n5 of frequency bins exceeding the threshold value T2 in the range between the lower limit of the super wide band and the lower limit of the full band and between the upper limit of the super wide band and the upper limit of the full band, and when the number n5 is smaller than the threshold value N, it determines that signal I is in the super wide band (the number of thresholds T2 or more at i_Band[2]_Lo or less and greater than i_Band[3]_Lo and i_Band[2]_Up or more and less than i_Band[3]_Up (band outside 3) <N:SWB), and when the number n5 is greater than or equal to the threshold value N, it may determine that signal I is in the full band (the number of thresholds T2 or more at i_Band[2]_Lo or less and greater than i_Band[3]_Lo and i_Band[2]_Up or more and less than i_Band[3]_Up (band outside 3) ≧N:FB).

[0074] Basically, the difference in the frequency band is mainly on the high-frequency side, and there is not much difference on the low-frequency side. Therefore, it is not essential to compare the number of frequency bins exceeding the threshold value T2 with the threshold value N in the range outside the frequency band (narrow band, etc.) (low-frequency side and high-frequency side), and it is also possible to compare the number of frequency bins exceeding the threshold value T2 with the threshold value N on the high-frequency side of the frequency band. By using only the high-frequency side, the processing load can be reduced while maintaining the accuracy.

[0075] <Method 3> Figure 7 is a flowchart showing the processing flow (Method 3) of the bandwidth detection unit 13. First, the bandwidth detection unit 13 associates the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) of each frequency band with the frequency bin (Step S14).

[0076] Next, the bandwidth detection unit 13 calculates the sum of volume power for each frequency band of the acquired reference signal within the range outside the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) (below the lower limit, above the upper limit) (step S17).

[0077] Next, the bandwidth detection unit 13 compares the sum of volume power calculated in step S17 with a predetermined threshold M to determine the frequency band of the reference signal (step S18). For example, if the sum of volume power in the range outside of the arbitrary frequency band (frequency band A) acquired in step S17 is less than or equal to the threshold M, it is determined that the reference signal is in frequency band A. If it exceeds the threshold M, a comparison is made with the next widest frequency band after frequency band A.

[0078] The bandwidth detection unit 13 performs comparisons in order from the narrowest frequency band. Specifically, it first compares the sum of volume power with the threshold M in the range outside the narrowband, then compares the sum of volume power with the threshold M in the range outside the wideband, then compares the sum of volume power with the threshold M in the superwideband, and finally compares the sum of volume power with the threshold M in the fullband. Note that fullband comparison is not mandatory.

[0079] FIG. 8 is a diagram schematically showing a process of calculating the sum of volume powers. In the case of the signal shown by the black-line graph in FIG. 8, the sum of volume powers S1 is calculated in the range outside the lower limit (i_Band[0]_Lo = 2) and the upper limit (i_Band[0]_Up = 109) of the narrow band (the shaded part in FIG. 6). Since the sum S1 (for example, 1) is smaller than the threshold value M (for example, 5), it can be seen that the signal shown by the black-line graph in FIG. 8 is a narrow band (sum S1 < M in the range below i_Band[0]_Lo and above i_Band[0]_Up: NB). Note that when the sum of volume powers S1 in the range between the lower limit of the narrow band and the lower limit of the wide band and between the upper limit of the narrow band and the upper limit of the wide band is smaller than the threshold value M, it may be determined that the signal is a narrow band (sum S1 < M in the range below i_Band[0]_Lo and greater than i_Band[1]_Lo and above i_Band[0]_Up and less than i_Band[1]_Up (outside the band 1): NB).

[0080] In the case of the signal shown by the gray-line graph in FIG. 6, in the range outside the lower limit and the upper limit of the narrow band, the sum of volume powers S2 (for example, 10) is greater than the threshold value M, so it can be seen that this signal is a signal in a frequency band wider than the narrow band. Next, for the signal shown by the gray-line graph, the sum of volume powers S3 is calculated in the range outside the lower limit and the upper limit of the wide band. Since the sum S3 (for example, 3) is below the threshold value M, it can be seen that the signal shown by the gray-line graph in FIG. 8 is a wide band (sum S2 < M in the range below i_Band[1]_Lo and above i_Band[1]_Up: WB). Note that when the sum S2 in the range between the lower limit of the wide band and the lower limit of the super-wide band and between the upper limit of the wide band and the upper limit of the super-wide band is smaller than the threshold value M, it may be determined that the signal is a wide band (sum S2 < M in the range below i_Band[1]_Lo and greater than i_Band[2]_Lo and above i_Band[1]_Up and less than i_Band[2]_Up (outside the band 2): WB).

[0081] Further, when the band detection unit 13 detects that a certain signal (signal I) is a signal in a frequency band wider than the wide band, for signal I, the total volume power S3 is obtained in the range outside the lower and upper limits of the super wide band, and when the total sum S3 is smaller than the threshold value M, it is determined that signal I is in the super wide band (the total sum S3 < M in the super wide band (outside the band 3) at i_Band[2]_Up or higher), and when the total sum S3 is greater than or equal to the threshold value M, it is determined that signal I is in the full band (the total sum S3 ≥ M in the super wide band (outside the band 3) at i_Band[2]_Up or higher).

[0082] In addition, for signal I, the band detection unit 13 may obtain the total volume power S4 in the range between the lower limit of the super wide band and the lower limit of the full band and between the upper limit of the super wide band and the upper limit of the full band, and when the total sum S4 is smaller than the threshold value M, signal I is considered to be in the super wide band (the total sum S4 < M in the range where i_Band[2]_Lo or lower, greater than i_Band[3]_Lo, and i_Band[2]_Up or higher and less than i_Band[3]_Up (outside the band 3): SWB), and when the total sum S4 is greater than or equal to the threshold value M, signal I is considered to be in the full band (the total sum S3 ≥ M in the range where i_Band[2]_Lo or lower, greater than i_Band[3]_Lo, and i_Band[2]_Up or higher and less than i_Band[3]_Up (outside the band 3): FB).

[0083] Basically, the difference in the frequency band is mainly on the high-frequency side, and there is not much difference on the low-frequency side. Therefore, it is not essential to compare the total volume power with the threshold value M in the range outside the frequency band (low-frequency side and wide-frequency side), and the total volume power can be compared with the threshold value M on the high-frequency side of the frequency band.

[0084] Using one of the methods 1, 2, or 3 described above, the bandwidth detection unit 13 detects the frequency band of the reference signal, i.e., the signal acquired via the communication device 53. Method 1 is the simplest and requires the least computation, but there is a risk of misjudgment due to noise. Method 2 is more accurate than Method 1, but there is a risk of misjudgment if the frequency bins are coarse (low resolution). Method 3 requires the most computation but is the most accurate. In this embodiment, the bandwidth detection unit 13 detects the frequency band using Method 3.

[0085] Next, the bandwidth-specific countermeasure unit 14 will be described. The bandwidth-specific countermeasure unit 14 calculates a threshold value corresponding to the frequency band and outputs it to the double-talk detection unit 15 and the nonlinear echo suppression unit 16.

[0086] First, the processing performed by the double talk detection unit 15 and the nonlinear echo suppression unit 16 using the threshold values ​​obtained by the band-specific countermeasures unit 14 and the double talk detection unit 15 is explained using a specific example. Figure 9 schematically shows the processing performed by the double talk detection unit 15. The horizontal axis of Figure 9 is time, and the vertical axis is the power of the signal (suppressed signal) after the linear echo has been removed by the echo removal unit 11. The light gray bar graph in Figure 9 represents the echo component, and the dark gray represents the speech component within the closed space 90. Figure 9 also illustrates the power of the frequency bins that have exceeded the upper limit of the narrowband frequency bins when the band detection unit 13 detects that it is a narrowband.

[0087] Typically, for example, when the reference signal is a sound in the narrowband frequency range, a narrowband double-talk detection mask (corresponding to the first detection mask of the present invention, threshold T3 in Figure 9) is used to detect whether or not speech has been input (whether or not there is transmission) from the sound detection sensor 51 for signals in the narrowband frequency range among the suppressed signals.

[0088] In contrast, when the reference signal is in the narrowband frequency range, an echo signal may be input to signals outside the narrowband frequency range of the suppressed signal (for example, in the high-frequency band), even though no sound should be present (see the area enclosed by the dashed line in Figure 22(B)). In this way, in frequency bands outside the frequency band detected by the band detection unit 13, the echo signal becomes larger than it should be, which can cause the power of the suppressed signal to exceed the threshold T3 even though only an echo signal is present (near-end utterance, i.e., no transmission), potentially leading to a false detection of transmission (see the circle in Figure 9).

[0089] Therefore, when the reference signal is a sound in the narrowband frequency range, for signals in the frequency band outside the narrowband frequency range of the suppressed signal, a threshold T4 (corresponding to the second detection mask of the present invention), which is greater than the threshold T3 which is the double-talk detection mask in the narrowband frequency band, is used to detect whether or not speech has been input from the sound detection sensor 51. In this embodiment, threshold T4 is made greater than threshold T3 by adding an arbitrary fixed value to threshold T3.

[0090] Here, we will explain the threshold T3. The threshold T3 is determined based on a previously determined estimated echo function. Figure 10 is a schematic diagram of the functional blocks used in the acoustic signal processing device 1 to determine the function for calculating the estimated echo. The acoustic signal processing device 1 functionally includes an estimated echo calculation unit 25.

[0091] The calculation process for estimated echoes will be explained in detail. First, after the echo removal unit 11 has sufficiently completed learning the adaptive filter, under conditions where there is no near-end utterance and background noise is sufficiently low, a learning received signal is transmitted to the receiving-side signal path, and a far-end one-sided utterance (single talk) is repeatedly performed in which the sound generation unit 52 outputs sound based on the learning received signal. The signal transmitted to the transmitting-side signal path during single talk is then used as the learning signal. In the acoustic signal processing device 1, the signal from which the echo has been removed by the echo removal unit 11 becomes the learning signal. Note that a sound source with a wide frequency band is used for the far-end one-sided utterance. As a result, the learning received signal and the learning signal also have a wide frequency band. For example, when the acoustic signal processing device 1 is used in narrowband and wideband modes, a wideband sound source is used for the far-end one-sided utterance.

[0092] A time-dependent learning signal (hereinafter referred to as learning signal [t]) is converted by the FFT unit 12 into a frequency-dependent learning signal (hereinafter referred to as learning signal [i]) and input to the estimated echo calculation unit 25. Similarly, a time-dependent learning received signal (hereinafter referred to as learning received signal [t]) is converted by the FFT unit 18 into a frequency-dependent learning received signal (hereinafter referred to as learning received signal [i]) and input to the estimated echo calculation unit 25.

[0093] The estimated echo calculation unit 25 calculates the power spectra of the learning signal [i] and the learning receiver signal [i] at regular intervals to obtain multiple learning power spectra. Here, a regular interval is a predetermined time domain that is arbitrarily defined. The power spectrum P[i] is expressed as the square of the Fourier spectrum X[i] obtained by the Fast Fourier Transform (see equation (1)). P[i]=|X[i]|2=|X[i]|×|X[i]| ···(1)

[0094] The estimated echo calculation unit 25 creates multiple scatter plots of the learning signal [i] and the learning received signal [i] based on the learning signal [i], the learning received signal [i], and the learning power spectrum. Even if the power spectrum of the learning signal is the same, the power spectrum of the learning signal, i.e., the echo, will vary. Therefore, in this embodiment, the estimated echo is calculated not only based on the power spectrum of the learning signal, but also based on multiple scatter plots with different horizontal axes.

[0095] Here, the power spectrum at each frequency of the learning signal refers to the power spectrum of the echo produced by the learning receiver signal. The total receiver power spectrum is the sum of the power spectra at each frequency of the learning signal, i.e., the sum of the power spectra of the learning receiver signal [t] before passing through the FFT section 18, and is expressed by the following equation (2).

[0096]

number

[0097] The total received power spectrum may also be the sum of the power spectra at each frequency within an arbitrary frequency range of the learning signal. In this case, the total received power spectrum is expressed by the following formula (3). Here, A is greater than or equal to 0, and B is less than the maximum frequency (A>0, B <F_MAX)。

number

[0098] When the double-talk detection unit 15 performs speech detection (described in detail later), it may be more accurate to use the sum of the power spectra of an arbitrary frequency range of the learning signal (equation (3)) rather than the sum of the power spectra of all frequencies of the learning signal (equation (2)). Therefore, in such cases, it is desirable for the estimated echo calculation unit 25 to use equation (3) to obtain the total received power spectrum.

[0099] A certain relationship exists between the logarithmic and frequency information of the learning receiver signal and the power spectrum of the learning signal, i.e., the echo. In this embodiment, a sufficient amount of learning signal [i] and learning receiver signal [i] are acquired in advance, and the estimated echo amount is determined based on the certain relationship between them. The estimated echo calculation unit 25 calculates the estimated echo function using the following formula (4). The estimated echo function (estimated echo power spectrum [i]) is a frequency-dependent signal and is expressed as a function with variables the logarithm of the magnitude of the learning receiver signal at each frequency, the frequency of the learning receiver signal, the logarithm of the total receiver power spectrum of the learning receiver signal, and the logarithm of the envelope of the total receiver value of the learning receiver signal.

[0100] Estimated echo power spectrum [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum ... (4)

[0101] The calculation of the estimated echo function will be explained in detail using Figures 11 to 14. The estimated echo calculation unit 25 sequentially calculates the coefficient α of the received power spectrum [i], the coefficient β of the frequency, the coefficient γ of the total received power spectrum, and the coefficient δ of the envelope of the total received power spectrum. The coefficients α, β, γ, and δ of each variable are obtained based on data obtained by removing outliers from the learning signal [i].

[0102] Figure 11 is a scatter plot of the logarithm of the power spectrum at each frequency of the learning receiver signal (hereinafter referred to as the receiver power spectrum) and the logarithm of the power spectrum at each frequency of the learning signal (hereinafter referred to as the transmission power spectrum). In Figure 11, the measured data is plotted, and α is shown as a line.

[0103] α represents the relationship between the logarithm of the received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. α is determined by removing outliers from a scatter plot of the logarithms of the received and transmitted power spectra. α can be expressed as a linear function (without conditional branching) or a nonlinear function (with conditional branching).

[0104] As shown in Figure 11, a larger power spectrum in the received signal does not necessarily mean a larger power spectrum in the transmitted signal (i.e., echo); rather, the echo decreases when the power spectrum of the received signal exceeds a certain level. This is due to the characteristics of the sound generation unit 52 (there is a region where sound cannot be produced) and the fact that the echo removal unit 11 is provided before the FFT unit 12. In the example shown in Figure 5, α is given by equations (5) and (6). Thus, α is a nonlinear function.

[0105] When the logarithm of the power spectrum of the receiver is <-1 α = 0.5 × logarithm of the power spectrum of the receiver -0.5···(5)

[0106] When the logarithm of the power spectrum of the receiver is greater than or equal to -1 α = -1.0 × logarithm of the power spectrum of the receiver -2.0···(6)

[0107] In the case where the echo removal unit 11 is not provided, the peak of the line representing α shifts to the right compared to the example shown in Figure 11, and the slope of the descending line after the peak becomes smaller, but α remains a nonlinear function (with conditional branching).

[0108] Once α is calculated, the estimated echo calculation unit 25 calculates β. Figure 12 is a scatter plot of the logarithm of the frequency of the learning receiver signal (hereinafter referred to as the receiver frequency) and the logarithm of the power spectrum of the transmission. In Figure 12, the result of subtracting the α component from the measured data is plotted, and β is shown as a line.

[0109] β represents the relationship between the receiver frequency and the maximum value of the logarithm of the transmit power spectrum. β is determined by removing outliers from a scatter plot of the receiver frequency and the logarithm of the transmit power spectrum. β can be expressed as a linear or nonlinear function.

[0110] The sound generating unit 52 has the characteristic of being unable to produce low or high frequencies, so in Figure 11, the echo is small for low and high frequencies. Also, when the terminal 50 is installed inside a vehicle, as shown in Figure 12, there is a dip around 1 kHz where the echo is small due to the influence of the intermediate environment (reflection, etc.). Therefore, β is a nonlinear function.

[0111] Once β is calculated, the estimated echo calculation unit 25 calculates γ. Figure 13 is a scatter plot of the logarithm of the total received power spectrum of the learning received signal and the logarithm of the transmitted power spectrum. In Figure 13, the results obtained by subtracting the α and β components from the measured data are plotted, and γ is shown as a line.

[0112] For example, when the sound generation unit 52 outputs sounds at 100Hz and 110Hz, the sound generation unit 52 may also emit a sound at 105Hz in addition to 100Hz and 110Hz. Therefore, in order to refer to information on whether sounds other than the intended frequency are being emitted, in this embodiment, a term with the logarithm of the total received power spectrum as a variable is added to the estimated echo function (equation (4)).

[0113] γ represents the relationship between the logarithm of the total received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. γ is determined by removing outliers from a scatter plot of the logarithms of the received frequency and transmitted power spectrum. γ can be expressed as a linear or nonlinear function. In the example shown in Figure 13, γ is a nonlinear function.

[0114] Once γ is calculated, the estimated echo calculation unit 25 calculates δ. Figure 14 is a scatter plot of the logarithm of the envelope of the total received power spectrum of the learning received signal and the logarithm of the transmitted power spectrum. In Figure 14, the results obtained by subtracting the α, β, and γ components from the measured data are plotted, and δ is shown as a line.

[0115] Because sound reflections inside the vehicle and vibrations of the sound generating unit 52 are output as sound from the sound generating unit 52, echoes can exist even without a learning received signal. Therefore, it is necessary to estimate the echo by referring not only to the total received power spectrum at the current time, but also to the learning signals over a recent period. For this reason, in this embodiment, a term in which the logarithm of the envelope of the total received power spectrum is used as a variable is added to the estimated echo function (equation (4)).

[0116] Envelope A is the maximum value over a recent period and is calculated gradually using the time constant B and the total received power spectrum C, as shown in the following equation (7). In this embodiment, the time constant B is set to 0.5 to 1.

[0117] If(A <C): A=C Else: A = B × A + (1 - B) × C ... (7)

[0118] δ represents the relationship between the logarithm of the envelope of the total received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. δ is determined by removing outliers from a scatter plot of the logarithms of the received frequency and transmitted power spectrum. δ can be expressed as a linear or nonlinear function. In the example shown in Figure 14, δ is a linear function.

[0119] Once the estimated echo function (a function representing the estimated echo power spectrum [i]) is calculated in this manner, the estimated echo calculation unit 25 stores the four coefficients α, β, γ, and δ of this function as parameters for the threshold T3 in the storage unit 14a of the band-specific countermeasure unit 14.

[0120] Furthermore, the memory unit 14a stores an arbitrary fixed value X (corresponding to the added value in this invention) that is added to the threshold T3 when determining the threshold T4.

[0121] Figure 15 is a flowchart showing the processing flow for double talk detection and nonlinear echo suppression according to the frequency band. First, the double talk detection unit 15 determines whether or not there is a call received based on the presence or absence of a reference signal (step S21). If there is no reference signal, i.e., there is no call received (NO in step S21), the nonlinear echo suppression unit 16 does not perform suppression processing, and the series of processes in Figure 15 ends.

[0122] If a reference signal is present, i.e., if there is a call received (YES in step S21), the double talk detection unit 15 obtains the frequency band detected by the band detection unit 13 and thresholds T3 and T4 from the band-specific countermeasures unit 14 (step S22). Thresholds T3 and T4 can be generated using formulas (8) and (9) stored in the memory unit 14a. In calculation formulas (8) and (9), the power of the reference signal (received signal) used to detect the frequency band in step S21 is applied to the received power.

[0123] Threshold T3 calculation formula [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum ... (8) Threshold T4 calculation formula [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum + fixed value X...(9)

[0124] Furthermore, not only the threshold T4 for wideband, but also the thresholds for superwideband and fullband can be determined in the same way. The threshold for superwideband can be determined by adding a fixed value X1 to the threshold T3, and the threshold for fullband can be determined by adding a fixed value X2 to the threshold T3.

[0125] Next, the double talk detection unit 15 calculates a threshold T3 or threshold T4 based on the calculation formula obtained in step S22, and determines whether or not there is a transmission based on the threshold T3 or threshold T4 (step S23).

[0126] For example, if the frequency band of the reference signal is determined to be narrowband by the band-specific countermeasures unit 14 in step S22, the double talk detection unit 15 compares the threshold T3 with the suppressed signal for frequency bins included in the narrowband frequency band, and compares the threshold T4 with the suppressed signal for frequency bins outside the narrowband frequency band. Also, for example, if the frequency band of the reference signal is determined to be wideband by the band-specific countermeasures unit 14 in step S22, the double talk detection unit 15 compares the threshold T3 with the suppressed signal for frequency bins included in the wideband frequency band, and compares the threshold for superwideband with the suppressed signal for frequency bins outside the wideband frequency band.

[0127] For example, the double talk detection unit 15 compares the magnitude of the suppressed signal with the magnitude of threshold T3 or threshold T4 for each frequency bin. If the number of frequency bins in which the magnitude of the suppressed signal is greater than or equal to the magnitude of threshold T3 or threshold T4 is less than or equal to a predetermined number, it determines that there is no transmission. Otherwise, it determines that there is a transmission.

[0128] If there is no transmission (NO in step S23), i.e., a single-talk state with only reception, the double-talk detection unit 15 outputs to the nonlinear echo suppression unit 16 that it is a single-talk state with only reception, and the nonlinear echo suppression unit 16 suppresses the nonlinear echo (strong suppression) to reduce the magnitude of the transmission signal to an acceptable value (step S24), and the series of processes shown in Figure 15 is completed.

[0129] For example, in step S24, if the band-specific countermeasures unit 14 has determined in step S22 that the frequency band of the reference signal is narrowband, the nonlinear echo suppression unit 16 suppresses nonlinear echoes for frequency bins included in the narrowband frequency band such that the suppressed signal with a magnitude of threshold T3 becomes an acceptable value (e.g., near 0), and suppresses nonlinear echoes for frequency bins outside the narrowband frequency band such that the suppressed signal with a magnitude of threshold T4 (threshold T4 > threshold T3) becomes an acceptable value.

[0130] In step S24, the suppression of nonlinear echoes is strengthened for frequency bins outside the frequency band detected by the band detection unit 13 compared to frequency bins included in the frequency band detected by the band detection unit 13.

[0131] For example, when the reference signal is a sound in the narrowband frequency range, signals outside the narrowband frequency range of the suppressed signal (e.g., high-frequency bands) may receive echo signals even though they should not produce any sound (see the area enclosed by the dashed line in Figure 22(B)). Therefore, by suppressing the echo more strongly than the original nonlinear echo suppression, it is possible to address nonlinear echoes in frequency bands where sound should not normally be produced.

[0132] In step S24, the nonlinear echo suppression unit 16 may choose not to suppress nonlinear echoes for frequency bins included in the narrowband frequency range, but to strongly suppress nonlinear echoes for frequency bins outside the narrowband frequency range (for example, by reducing them by 30 dB). In this case as well, the suppression of nonlinear echoes is stronger for frequency bins outside the frequency band detected by the band detection unit 13 than for frequency bins included in the frequency band detected by the band detection unit 13.

[0133] If there is a transmission (YES in step S23), that is, if there is a double talk state with both receiving and transmitting, the double talk detection unit 15 outputs that there is a double talk state to the nonlinear echo suppression unit 16, and the nonlinear echo suppression unit 16 suppresses the nonlinear echo more weakly than in the case of single talk (weak suppression) (step S25), and the series of processes shown in Figure 15 is completed.

[0134] For example, in step S25, if the frequency band of the reference signal is determined to be narrowband by the band-specific countermeasures unit 14 in step S22, the nonlinear echo suppression unit 16 will not suppress nonlinear echoes for frequency bins included in the narrowband frequency band, and will suppress nonlinear echoes for frequency bins outside the narrowband frequency band to a degree weaker than the suppression (step S24) where the suppressed signal of threshold T3 is at an acceptable level (for example, 50% of strong suppression).

[0135] In step S25, the nonlinear echo suppression unit 16 may suppress nonlinear echoes for frequency bins outside the narrowband frequency range to a weaker level (e.g., 50% of strong suppression) than the suppression level at which the suppressed signal of threshold T4 magnitude becomes acceptable. Alternatively, if the nonlinear echo suppression unit 16 performed strong suppression in step S24 to reduce the nonlinear echo by a certain value (e.g., 30 dB) for frequency bins outside the narrowband frequency range, it may perform weak suppression to reduce the nonlinear echo by a smaller value (e.g., 10 dB) than in step S24.

[0136] In step S25, as in step S24, the suppression of nonlinear echoes is strengthened more for frequency bins outside the frequency band detected by the band detection unit 13 than for frequency bins included in the frequency band detected by the band detection unit 13.

[0137] The process shown in Figure 15 is performed each time a sample point is acquired. After the process in Figure 15 is performed, the IFFT unit 17 performs an inverse FFT on the signal output from the nonlinear echo suppression unit 16, and the signal converted to the time domain by the inverse FFT is output to the communication device 53.

[0138] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasure unit 14 performs countermeasures according to the bandwidth (changing the degree of nonlinear echo suppression depending on whether the signal input from the communication device 53 is in the frequency bin of the frequency band). As a result, good call quality can be ensured across multiple frequency bands without tuning for each frequency band.

[0139] For example, in situations where nonlinear echoes and reflections are likely to occur, such as in an in-vehicle device where the microphone and speaker are integrated, if the band-specific countermeasures unit 14 does not take countermeasures according to the frequency band, using the adjustment results for a wideband environment in a narrowband environment may result in degraded sound quality and make it difficult to use. In contrast, in this embodiment, the band-specific countermeasures unit 14 changes the degree of nonlinear echo suppression depending on whether the signal input from the communication device 53 is in the frequency bin of the band, so that call quality can be guaranteed even when the adjustment results for a wideband environment are used in a narrowband environment.

[0140] Furthermore, according to this embodiment, in the case of single talk where only receiving is possible, the nonlinear echo suppression unit 16 suppresses nonlinear echoes more strongly for frequency bins outside the frequency band detected by the frequency band detected by the frequency band detected by the frequency band detection unit 13 than for frequency bins included in the frequency band detected by the frequency band detection unit 13, thereby reliably suppressing unwanted nonlinear echoes.

[0141] Furthermore, according to this embodiment, the double talk detection unit 15 detects the presence or absence of transmission using a threshold T3 for frequency bins included in the frequency band detected by the band detection unit 13, and detects the presence or absence of transmission using a threshold T4 (threshold T4 > threshold T3) for frequency bins outside the frequency band detected by the band detection unit 13, thereby preventing false detection of the presence or absence of transmission.

[0142] Furthermore, in this embodiment, the frequency band of the reference signal (the signal input from the communication device 53) is detected based on the magnitude of the signal outside the frequency band of the frequency band to be detected, among the signals obtained by converting the reference signal transmitted through the receiver-side signal path into a frequency domain signal in the FFT unit 18. Therefore, the frequency band can be detected using only the reference signal. In particular, by detecting the frequency band based on the sum of the power of the frequency bins outside the frequency band, accurate detection can be achieved.

[0143] In this embodiment of the present invention, the threshold for double talk detection is changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 (threshold T3 for frequency bins in the frequency band detected by the band detection unit 13, and threshold T4 for frequency bins outside the frequency band detected by the band detection unit 13 (threshold T4 > threshold T3)), and the strength of suppression is changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 in the case of single talk (for frequency bins outside the frequency band detected by the band detection unit 13, nonlinear echo suppression is stronger than for frequency bins in the frequency band detected by the band detection unit 13). However, it is not essential to change the threshold depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13. For example, double talk may be detected using threshold T3 for all frequency bins, and the strength of suppression may be changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 in the case of single talk (receiving only).

[0144] Furthermore, in this embodiment of the present invention, a fixed value is added to the threshold T3 to obtain the threshold T4, but the method for determining the threshold T4 is not limited to this. The following describes some variations in the method for determining the threshold T4.

[0145] <Example 1> Modification 1 is a form in which the summation value to be added to the threshold T3 is determined using the coefficient γ1 of the total received power spectrum and the coefficient δ1 of the envelope of the total received power spectrum. The summation value is determined by the estimated echo calculation unit 25. Figure 16 is a flowchart showing the sequence of processes for determining the summation value. Here, narrowband and wideband are used as examples.

[0146] The estimated echo calculation unit 25 performs calibration using a wideband sound source and determines the four coefficients α, β, γ, and δ used in the threshold T3 calculation formula using the method already described (step S31). The threshold T3 calculation formula [i] is equation (8) already described.

[0147] Next, the estimated echo calculation unit 25 performs far-end utterance using a narrowband sound source and obtains a pair value of a reference signal (learning received signal) and an echo signal (learning signal) outside the narrowband frequency band (step S32). In other words, in step SP32, even though only narrowband sound is being played, an echo signal that is outside the narrowband frequency band and within the wideband frequency band is obtained in correspondence with the reference signal from which that signal was obtained.

[0148] Next, the estimated echo calculation unit 25 subtracts the threshold values ​​(α component and β component) obtained in step S31 from the data obtained in step S32 (step S33). However, the result of the subtraction is not made to be less than or equal to 0. As a result, the result shown in Figure 13 is obtained.

[0149] Next, the estimated echo calculation unit 25 extracts the maximum value obtained by subtracting the α and β components from the logarithm of the transmitted power for each logarithm of the transmitted / received power in the results obtained in step S33 (step S34), and determines the coefficient γ1 for wideband (step S35). The method for determining the coefficient γ1 is the same as the method for determining the coefficient γ. Alternatively, the coefficient γ1 may be determined by regression analysis with the result obtained in step S33 as the dependent variable and the logarithm of the transmitted / received power as the independent variable.

[0150] Next, the estimated echo calculation unit 25 subtracts the coefficient γ1 component obtained in step S35 from the result obtained in step S33 (step S36). In the result obtained in step S36, for each logarithm of the most recent maximum value of each transmit / receive power, the maximum value obtained by subtracting the α component, β component, and γ1 component from the logarithm of the transmit power is extracted (step S37), and the coefficient δ1 for wideband is determined (step S38). The method for determining the coefficient δ1 is the same as the method for determining the coefficient δ. This completes the process of determining the parameter set for the summation value. The estimated echo calculation unit 25 stores the coefficients α, β, γ, δ and coefficients γ1, δ1 as parameters for the summation value in the storage unit 14a of the band-specific countermeasure unit 14.

[0151] The bandwidth-specific countermeasure unit 14 calculates the threshold T4 using the coefficients γ1 and δ1 stored in the memory unit 14a and formulas (10) and (11). However, a fixed value Y is not required.

[0152] Threshold T4 = Threshold T3 + Addition value ... (10) The sum [i] = γ1 × total received power spectrum + δ1 × envelope of total received power spectrum + fixed value Y...(11)

[0153] Thus, the sum is generated by inputting a signal generated using a sound source in a narrow first frequency band (in this case, narrowband) into the receiver's signal path, and by acquiring an echo signal in a frequency band wider than the first frequency band (in this case, wideband).

[0154] The estimated echo calculation unit 25 calculates not only the summation value [i] for wideband, but also for superwideband and fullband. The method for calculating the coefficients γ1' and δ1' for superwideband and the coefficients γ1'' and δ1'' for fullband is the same as for the coefficients γ1 and δ1 for wideband.

[0155] This modified method allows for the determination of more accurate summation values ​​compared to methods that simply add fixed values. Furthermore, it can be performed with a small number of parameters: a threshold T3 and three summation values.

[0156] In this modified example, by modifying the threshold T4[i] (threshold T3[i] and summation value[i]) as shown in the following formula (12), the double talk detection unit 15 can perform speech detection without conditional branching between using threshold T3 and using threshold T4. α × received power spectrum[i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum + (γ1 × total received power spectrum + δ1 × envelope of total received power spectrum) = α × received power spectrum[i] + β × frequency + γz × total received power spectrum + δz × envelope of total received power spectrum (where γz = γ + γ1, δz = δ + δ1) ... (12)

[0157] <Modification 2> Modification 2 is a form in which threshold values ​​for each frequency band are determined separately, rather than determining the sum to be added to the threshold T3. The threshold T3 is as already explained (see equation (8)). The formula for calculating the wideband threshold T4 is given by equation (13). Note that the method for determining the coefficients γ1 and δ1 in equation (13) is the same as in the case of equation (11).

[0158] Threshold T4 calculation formula [i] = γ1 × total received power spectrum + δ1 × envelope of total received power spectrum + fixed value Y...(13)

[0159] Thus, the threshold for the second frequency band (in this case, wideband), which is wider than the narrow first frequency band, is generated by inputting a signal generated using a sound source in the first frequency band (in this case, narrowband) into the receiver-side signal path, and by acquiring an echo signal in the frequency band of the second frequency band.

[0160] Furthermore, the estimated echo calculation unit 25 can similarly determine thresholds for both super-wideband and full-band applications.

[0161] According to this modified example, the method for calculating the threshold T4 is the same as for the threshold T3, but the number of coefficients (parameters) can be reduced.

[0162] <Variation 3> Modification 3, like Modification 2, is a form in which thresholds for each frequency band are determined separately. Figure 17 is a flowchart showing the sequence of processes for determining thresholds T3 and T4. Here, the frequency bands are narrowband and wideband.

[0163] The estimated echo calculation unit 25 performs calibration using a wideband sound source and determines the four coefficients α, β, γ, and δ used in the calculation formula for the threshold T3 using the method already described (step S31).

[0164] Next, the estimated echo calculation unit 25 performs far-end speech using a narrowband sound source and obtains pair values ​​of a reference signal (learning received signal) and an echo signal (learning signal) outside the narrowband frequency band (step S32).

[0165] Next, the estimated echo calculation unit 25 calculates the coefficients α2, β2, γ2, and δ2 based on the data obtained in step S32 (step S37). The method for calculating the coefficients α2, β2, γ2, and δ2 is the same as for the coefficients α, β, γ, and δ. This completes the process of calculating the parameters for the thresholds T3 and T4. The estimated echo calculation unit 25 stores the coefficients α, β, γ, and δ, and the coefficients α2, β2, γ2, and δ2, as parameters for addition in the storage unit 14a of the band-specific countermeasure unit 14.

[0166] The bandwidth-specific countermeasure unit 14 uses the coefficients α2, β2, γ2, δ2 stored in the memory unit 14a and formula (14) to determine the threshold T4.

[0167] Threshold T4 calculation formula [i] = α² × received power spectrum [i] + β² × frequency + γ² × total received power spectrum + δ² × envelope of total received power spectrum ... (14)

[0168] Thus, the threshold for the second frequency band (in this case, wideband), which is wider than the narrow first frequency band, is generated by inputting a signal generated using a sound source in the first frequency band (in this case, narrowband) into the receiver-side signal path, and by acquiring an echo signal in the frequency band of the second frequency band.

[0169] Furthermore, the estimated echo calculation unit 25 can similarly determine thresholds for both super-wideband and full-band applications.

[0170] According to this modified example, the method for calculating threshold T4 is exactly the same as that for threshold T3, making threshold adjustment easy.

[0171] Here, we will explain why, in modified examples 2 and 3, the threshold T4 calculated separately from the threshold T3 by the calculation formula is a larger value than the threshold T3. First, the coefficients α, β, γ, and δ of the threshold T3 are determined using a sound source in the second frequency band (here, wideband) (see Figure 22(A)), and then the coefficients γ1 and δ1 are determined using a sound source in the first frequency band (here, narrowband) (see Figure 22(B)). When determining the threshold T3, the coefficients α, β, γ, and δ are determined in that order, so the coefficient α is dominant. However, outside the narrowband, the logarithm of the received power [i] (corresponding to α) is 0 (see Figure 22(A)), so the threshold T3 is approximately 0 (threshold T3 ≈ 0). When determining the coefficients γ1 and δ1, learning is performed with the logarithm of the received power [i] being 0 (see Figure 22(B)), so α is not considered at all and the others are the dominant factors in the calculation. Therefore, the threshold T4 is close to the echo value (>>0) (threshold T4>>0). Therefore, threshold T4 is greater than threshold T3.

[0172] <Second Embodiment> The following describes the acoustic signal processing device 2 according to the second embodiment. Parts identical to those in the acoustic signal processing device 1 according to the first embodiment are denoted by the same reference numerals, and their descriptions are omitted.

[0173] Figure 18 is a schematic diagram of the functional blocks of the acoustic signal processing device 2. Functionally, the acoustic signal processing device 2 mainly comprises an echo removal unit 11, frequency analyzers (FFT units) 12 and 18, a band detection unit 13, a band-specific countermeasure unit 14A, a nonlinear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. Note that the functional components of the acoustic signal processing device 2 may be further classified into more components depending on the processing content, or one component may perform processing for multiple components.

[0174] The frequency band-specific countermeasure unit 14A performs countermeasures according to the frequency band detected by the frequency band detection unit 13. In this embodiment, the frequency band-specific countermeasure unit 14A controls the volume adjustment unit 20 and the compressor 21 differently depending on whether the frequency band detected by the frequency band detection unit 13 is a narrowband signal (corresponding to the first frequency band of the present invention) or a wider frequency band signal (corresponding to the second frequency band of the present invention). The frequency band-specific countermeasure unit 14A performs processing each time a sample point is acquired.

[0175] The bandwidth detection unit 13 detects the frequency band based on the reference signal, and the bandwidth-specific countermeasure unit 14A acquires the result. If the bandwidth-specific countermeasure unit 14A acquires that it is a narrowband (for example, a narrowband), the volume adjustment unit 20 and the compressor 21 perform normal processing.

[0176] If the bandwidth-specific countermeasure unit 14A detects that the frequency band is wider than a narrowband (for example, a wideband), it controls the volume adjustment unit 20 to gradually reduce the volume to a level where distortion is acceptable. The level at which distortion is acceptable can be determined in advance and stored in the memory unit 14a. Furthermore, the level at which distortion is acceptable can be set to decrease as the frequency bandwidth of the frequency band widens, and stored in the memory unit 14a.

[0177] Furthermore, if the bandwidth-specific countermeasure unit 14A detects that the frequency band is wider than a narrowband (for example, wideband), it controls the compressor 21 to gradually lower the compressor threshold until the distortion level is acceptable. The acceptable distortion level can be determined in advance and stored in the memory unit 14a. The acceptable distortion level can also be set to decrease as the bandwidth of the frequency band widens and stored in the memory unit 14a.

[0178] Thus, when the frequency band is wider than the narrowband, the volume control unit 20 and compressor 21 reduce the overall volume in order to lower the level of the extraneous echo signal in the high-frequency band (the area enclosed by the dashed line) in Figure 22(B).

[0179] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasures unit 14A takes countermeasures according to the frequency band (if the frequency band of the signal input from the communication device 53 is wideband, the volume and compressor threshold are lowered). As a result, good call quality can be ensured across multiple frequency bands without tuning each frequency band.

[0180] In this embodiment, the volume reduction process was performed using both the volume adjustment unit 20 and the compressor 21. However, it is also possible to perform the overall volume reduction process using at least one of the volume adjustment unit 20 and the compressor 21.

[0181] Furthermore, in this embodiment, the volume adjustment unit 20 and the compressor 21 performed a process to reduce the overall volume, but it is also possible to reduce the volume for at least some of the frequency bins using the volume adjustment unit 20 and the compressor 21.

[0182] Furthermore, the process by which the frequency band-specific countermeasure unit 14A reduces the overall volume using the volume adjustment unit 20 and the compressor 21 may be performed in combination with the processing of the frequency band-specific countermeasure unit 14.

[0183] <Third Embodiment> The acoustic signal processing device 3 according to the third embodiment will be described below. Parts identical to those in the acoustic signal processing device 1 according to the first embodiment will be denoted by the same reference numerals, and their descriptions will be omitted.

[0184] Figure 19 is a schematic diagram of the functional blocks of the acoustic signal processing device 3. Functionally, the acoustic signal processing device 3 mainly comprises an echo removal unit 11, frequency analyzers (FFT units) 12 and 18, a bandwidth detection unit 13, a bandwidth-specific countermeasure unit 14B, a nonlinear echo suppression unit (echo suppressor) 16, and a restoration unit (IFFT unit) 17. Note that the functional components of the acoustic signal processing device 3 may be further classified into more components depending on the processing content, or one component may perform processing for multiple components.

[0185] The band-specific countermeasure unit 14A performs countermeasures according to the frequency band detected by the band detection unit 13. In this embodiment, the band-specific countermeasure unit 14B controls the echo removal unit 11 differently depending on whether the frequency band detected by the band detection unit 13 is a narrowband signal (corresponding to the first frequency band of the present invention) or a wider frequency band signal (corresponding to the second frequency band of the present invention). The band-specific countermeasure unit 14A performs processing each time a sample point is acquired.

[0186] The bandwidth detection unit 13 detects the frequency band based on the reference signal, and the bandwidth-specific countermeasure unit 14B acquires the result. If the bandwidth-specific countermeasure unit 14B acquires that it is a narrowband, the echo removal unit 11 performs normal processing.

[0187] The bandwidth-specific countermeasures unit 14B controls the echo removal unit 11 to increase the step size (update width) and perform learning processing of the adaptive filter when it detects that the frequency band is wider than a narrowband (for example, wideband). The step size is set to be smallest when the bandwidth is narrowband and to increase as the bandwidth widens, and is stored in the memory unit 14a.

[0188] A linear filter H(n) is expressed by the following equation (15), where μ is the step size, X(n) is the input vector, and e(n) is the error signal. H(n)=H(n-1)+μX(n)e(n) ···(15)

[0189] For example, in a normal learning process, the step size μ is μ1 (corresponding to the first step size of the present invention, e.g., 0.05), but when the frequency band is wider than a narrowband (e.g., wideband), the step size μ is increased to μ2 (corresponding to the second step size of the present invention, e.g., 0.1, μ2 > μ1). Increasing the step size brings the echo output closer to zero, so the sound quality (e.g., conversations inside a car) deteriorates somewhat, but the distortion (out-of-band noise (see the area enclosed by the dashed line in Figure 22(B))) can be brought closer to zero. As a result, call quality is greatly improved.

[0190] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasure unit 14A takes countermeasures appropriate to the bandwidth (if the frequency band of the signal input from the communication device 53 is wideband, the step size μ is increased). As a result, good call quality can be ensured across multiple frequency bands without tuning each frequency band.

[0191] In this embodiment, the process by which the bandwidth-specific countermeasure unit 14B increases the step size μ may be performed in combination with the processing of the bandwidth-specific countermeasure unit 14 and the bandwidth-specific countermeasure unit 14A.

[0192] <Fourth Embodiment> The following describes the acoustic signal processing device 4 according to the fourth embodiment. Note that parts identical to those in the acoustic signal processing device 1 according to the first to third embodiments are denoted by the same reference numerals, and their descriptions are omitted.

[0193] Figure 20 is a schematic diagram of the functional blocks of the acoustic signal processing device 4. Functionally, the acoustic signal processing device 4 mainly comprises an echo removal unit 11, a frequency analyzer (FFT unit) 12, a band detection unit 13A, a band-specific countermeasure unit 14C, a nonlinear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. Note that the functional components of the acoustic signal processing device 4 may be further classified into more components depending on the processing content, or one component may perform processing for multiple components.

[0194] The bandwidth detection unit 13A is a functional unit that detects the frequency band of a signal acquired via the communication device (corresponding to the first frequency band of the present invention) based on a reference signal transmitted through the receiver-side signal path. For example, the bandwidth detection unit 13A detects whether it is a narrowband (corresponding to the first frequency band of the present invention) or a wideband (corresponding to the second frequency band of the present invention) with a frequency band wider than the narrowband. Alternatively, the bandwidth detection unit 13A may detect whether it is a narrowband wideband (corresponding to the first frequency band of the present invention) or a superwideband (corresponding to the second frequency band of the present invention) with a frequency band wider than the wideband, or whether it is a narrowband superwideband (corresponding to the first frequency band of the present invention) or a fullband (corresponding to the second frequency band of the present invention) with a frequency band wider than the superwideband.

[0195] First, the bandwidth detection unit 13A applies a bandstop filter to the reference signal to isolate only the components outside the narrowband frequency range. Next, the bandwidth detection unit 13A measures the power of the signal, which now consists only of the out-of-band components of the narrowband, and determines whether this power is above a certain threshold. If the power is below the threshold, the bandwidth detection unit 13A determines that the reference signal is a narrowband signal.

[0196] If the power is not below a threshold, the bandwidth detection unit 13A determines that the reference signal is audio in a frequency band wider than narrowband. The bandwidth detection unit 13A then applies a bandstop filter to the reference signal to isolate only the components outside the wideband frequency band and measures the power of the components outside the wideband frequency band. Next, the bandwidth detection unit 13A determines whether this power is above an arbitrary threshold. If the power is below the threshold, it determines that the reference signal is a wideband signal; if the power is not below the threshold, it determines that the reference signal is audio in a frequency band wider than wideband.

[0197] If the band detection unit 13A determines that the reference signal is audio in a frequency band wider than the wideband, it applies a bandstop filter to the reference signal to isolate only the components outside the superwideband frequency band and measures the power of the components outside the superwideband frequency band. Next, the band detection unit 13A determines whether this power is above or below an arbitrary threshold. If the power is below the threshold, it determines that the reference signal is a signal in the superwideband frequency band; if the power is below the threshold, it determines that the reference signal is audio in a frequency band wider than the superwideband, i.e., the fullband frequency band.

[0198] Note that the filter used is not limited to a bandstop filter. For example, a high-pass filter may be applied to the reference signal to measure the power, leaving only the components wider than the frequency band to be detected.

[0199] The band-specific countermeasure unit 14C performs countermeasures according to the frequency band detected by the band detection unit 13A. For example, the band-specific countermeasure unit 14C controls the echo removal unit 11 to perform learning processing of the adaptive filter with a step size corresponding to the detected frequency band.

[0200] Furthermore, for example, the frequency band-specific countermeasure unit 14C controls the volume adjustment unit 20 to gradually reduce the volume to an acceptable level of distortion, according to the detected frequency band. Also, for example, the compressor 21 controls the compressor threshold to gradually reduce the compressor threshold to an acceptable level of distortion, according to the detected frequency band.

[0201] According to this embodiment, the frequency band of the signal input from the communication device 53 can be detected by the band detection unit 13A without converting the reference signal into a frequency domain signal. Furthermore, since the reference signal is not converted into a frequency domain signal when detecting the frequency band, low-latency frequency band detection is possible. In addition, since the band-specific countermeasures unit 14C is performed based on the detection result, good call quality can be ensured across multiple frequency bands without tuning for each frequency band.

[0202] In this embodiment, the band-specific countermeasure unit 14C controlled the echo removal unit 11, the volume adjustment unit 20, and the compressor 21 according to the frequency band detected by the band detection unit 13A. However, the band-specific countermeasure unit 14C only needs to control at least one of the echo removal unit 11, the volume adjustment unit 20, and the compressor 21.

[0203] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design modifications and the like are also included within the scope of the gist of this invention. Furthermore, although power was used as an indicator of the magnitude of the signal in each of the above embodiments, the magnitude of the signal can also be expressed using amplitude (power is the square of the amplitude), and the power in the above embodiments can be replaced with amplitude. [Explanation of symbols]

[0204] 1, 2, 3, 4: Acoustic signal processing devices 11: Echo removal section 12, 18:FFT section 13, 13A: Bandwidth detection unit 14, 14A, 14B, 14C: Bandwidth-specific countermeasures 14a: Storage section 15: Double Talk Detection Unit 16: Nonlinear echo suppression unit 17:IFFT section 20: Volume control section 21: Compressor 25: Estimated Echo Calculation Unit 50, 54: Terminals 51: Sound detection sensor 52: Sound generation unit 53: Communications device 90: Closed space 100: Voice communication system

Claims

1. An acoustic signal processing device that transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device for voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, A bandwidth detection unit detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band wider than the first frequency band, based on a reference signal transmitted through the receiving signal path. A band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit, An acoustic signal processing device characterized by having the following features.

2. The system includes a frequency analyzer that converts the aforementioned reference signal into a signal in the frequency domain to obtain a signal for band detection. The bandwidth detection unit detects whether the signal is in the first frequency band or the second frequency band based on the magnitude of the signal in a predetermined range of frequency bands among the bandwidth detection signals. The predetermined range is a range higher than the upper limit of the frequency bin of the frequency band to be detected among the first frequency band and the second frequency band. The acoustic signal processing device according to feature 1.

3. The band detection unit detects that the receiving frequency band is the first frequency band if, in a range higher than the upper limit of the frequency bins of the receiving frequency band, the sum of the magnitudes of the band detection signals is less than or equal to a first threshold. The acoustic signal processing device according to feature 2.

4. A double-talk detection unit sequentially detects whether or not speech has been input to the sound detection sensor based on a double-talk detection mask, It comprises a nonlinear echo suppression unit that performs processing to suppress nonlinear echoes in a signal from which linear echoes have been removed, The nonlinear echo suppression unit suppresses nonlinear echoes more strongly for frequency bins not included in the first frequency band than for frequency bins included in the first frequency band, in the case of single talk where only receiving is occurring and the receiving frequency band is the first frequency band. The acoustic signal processing device according to claim 2 or 3.

5. The double-talk detection unit, when the receiving frequency band is the first frequency band, detects whether or not speech has been input from the sound detection sensor using a first detection mask for frequency bins included in the frequency band of the first frequency band, and detects whether or not speech has been input from the sound detection sensor using a second detection mask larger than the first detection mask for frequency bins not included in the frequency band of the first frequency band. The acoustic signal processing device according to feature 4.

6. The device includes a volume control unit that adjusts the volume of the signal transmitted through the receiver-side signal path, When the band-specific countermeasure unit detects that the receiving frequency band is the second frequency band, it controls the volume adjustment unit to gradually reduce the volume to a predetermined value. The acoustic signal processing apparatus according to any one of claims 1 to 5.

7. The receiver side signal path includes a compressor that compresses the signal transmitted through the aforementioned receiver-side signal path. When the band-specific countermeasure unit detects that the receiving frequency band is the second frequency band, it controls the compressor to gradually lower the compressor threshold at which the compressor starts compression. The acoustic signal processing device according to any one of claims 1 to 6.

8. The system includes a linear echo removal unit that suppresses linear echoes generated when sound output from the sound generation unit is input to the sound detection sensor, The linear echo removal unit removes linear echoes using the results of a learning process performed with a first step size when it is detected that the receiving frequency band is the first frequency band, and removes linear echoes using the results of a learning process performed with a second step size larger than the first step size when it is detected that the receiving frequency band is the second frequency band. The acoustic signal processing device according to any one of claims 1 to 7.

9. The bandwidth-specific countermeasure unit generates the first detection mask and the second detection mask using an estimated echo function calculated based on a second learning receiving signal obtained by converting a learning receiving signal transmitted through the receiving-side signal path into the frequency domain, and a second learning signal obtained by converting a learning receiving signal transmitted through the transmitting-side signal path into the frequency domain when the sound output from the sound generation unit due to the learning receiving signal is input to the sound detection sensor, wherein the estimated echo function has as variables the logarithm of the magnitude of the reference signal at each frequency, the frequency of the reference signal, the logarithm of the total received value which is the sum of the magnitudes of the reference signal or the sum of the reference signal in an arbitrary frequency range, and the logarithm of the envelope of the total received value. The first detection mask is generated by inputting a signal generated using a sound source in the second frequency band into the receiver-side signal path. The acoustic signal processing device according to feature 5.

10. The second detection mask is generated by adding an additional value to the first detection mask. The summation value is generated by inputting a signal generated using the first frequency band sound source into the receiver-side signal path and acquiring an echo signal in the second frequency band frequency range. The acoustic signal processing device according to feature 9.

11. An acoustic signal processing method comprising: transmitting a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device for voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit; and acquiring voice input via a sound detection sensor via a transmitting-side signal path and outputting it to the communication device, A step of detecting whether the receiving frequency band is a narrowband first frequency band or a second frequency band wider than the first frequency band, based on a reference signal transmitted through the receiving signal path, The steps include taking measures corresponding to the detected frequency band, A method for processing acoustic signals, characterized by including the following:

12. An acoustic signal processing program that transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device for voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, Computers, A bandwidth detection unit detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band wider than the first frequency band, based on a reference signal transmitted through the receiving signal path. A band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit, An acoustic signal processing program characterized by functioning as such.

Images