Acoustic signal processing apparatus, acoustic signal processing method, and acoustic signal processing program

Abstract

The present invention makes it possible to ensure excellent call quality in a plurality of frequency bands without performing tuning for each frequency band.　This acoustic signal processing apparatus: transfers, to a sound generation unit via a reception-side signal path, a signal in a reception frequency band which is a prescribed frequency band acquired via a communication device for performing speech communication, and causes the sound generation unit to emit a sound; and acquires, via a transmission-side signal path, speech inputted via a sound detection sensor, and outputs the same to the communication device. The acoustic signal processing apparatus comprises: a band detection unit that detects, on the basis of a reference signal transferred over the reception-side signal path, whether the reception frequency band is a first frequency band that is a narrow band or a second frequency band that is wider than the first frequency band; and a band-specific countermeasure unit that takes a countermeasure according to the frequency band detected by the band detection unit.

Description

Acoustic signal processing device, acoustic signal processing method, and acoustic signal processing program

[0001] The present invention relates to an acoustic signal processing device, an acoustic signal processing method, and an acoustic signal processing program.

[0002] In Patent Document 1, each time a sample point of a received signal transmitted through a received signal path that transmits a signal to an audio generation unit is acquired, based on the received signals acquired within a predetermined period before the time when the sample point was acquired, an optimal mask is sequentially generated or selected from basic masks, which are one or more masks generated based on a learning signal. Each time the optimal mask is generated or selected, it is sequentially detected whether it is in a double-talk state based on the result of comparing the input signal with the optimal mask. When it is detected that no speech is input to the voice detection sensor and the received signal contains speech, a process of suppressing echo for the input signal is sequentially performed. An acoustic signal processing device is disclosed.

[0003] Japanese Unexamined Patent Application Publication No. 2021-168455

[0004] The frequency band of a communication network varies depending on the country, region, communication carrier, usage environment, etc., and the performance also differs. In a conventional device such as the invention described in Patent Document 1, since it is necessary to perform pre-learning in the frequency bands of all assumed conditions and perform tuning according to the situation of the communication network, region, etc., there is a problem that an enormous amount of tuning time is required.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide an acoustic signal processing device, an acoustic signal processing method, and an acoustic signal processing program that can ensure good call quality in a plurality of frequency bands without tuning for each frequency band.

[0006] To solve the above problems, the acoustic signal processing device according to the present invention is an acoustic signal processing device that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, and is characterized by comprising: a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path; and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit.

[0007] To solve the above problems, the present invention provides an acoustic signal processing method that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, and is characterized by including the steps of detecting whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path, and taking countermeasures according to the detected frequency band.

[0008] To solve the above problems, the acoustic signal processing program according to the present invention is an acoustic signal processing program that, for example, transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generation unit via a receiving-side signal path to produce sound from the sound generation unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, characterized in that the computer functions as a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path, and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit.

[0009] According to the above embodiment of the present invention, based on a reference signal transmitted through the receiver-side signal path, it is detected whether the receiver frequency band is a first frequency band (narrowband) or a second frequency band (wider frequency band than the first frequency band), and countermeasures are taken according to the detected frequency band.

[0010] Conventionally, tuning was performed in advance for each frequency band, and the acoustic signal processing device 1 performed acoustic processing based on the results. However, this was time-consuming and costly, and required highly skilled personnel and time. Therefore, tuning was sometimes performed in a wider second frequency band, and the tuning results from the wider frequency band were used for the narrower first frequency band. However, when nonlinear echoes or reflection components are large, nonlinear echoes are likely to occur due to distortion and vibration of the speaker and amplifier, and reflections occur inside the enclosure, so the tuning results from the second frequency band may not be usable in the first frequency band.

[0011] In contrast, according to the above embodiment of the present invention, instead of using the tuning results in the second frequency band in the first frequency band, countermeasures are taken according to the detected frequency band, thus ensuring good call quality across multiple frequency bands without tuning for each frequency band.

[0012] The system includes a frequency analyzer that converts the reference signal into a frequency domain signal to obtain a bandwidth detection signal. The bandwidth detection unit detects whether the signal is in the first frequency band or the second frequency band based on the magnitude of the signal in a predetermined range of frequency bands within the bandwidth detection signal. The predetermined range may be a range higher than the upper limit of the frequency bin of the frequency band to be detected among the first and second frequency bands. By using a frequency band signal, high frequency resolution can be achieved, and the accuracy of bandwidth detection can be increased. Furthermore, by using a range higher than the upper limit of the frequency bin of the frequency band to be detected, the processing load can be reduced.

[0013] The bandwidth detection unit may detect that the receiving frequency band is the first frequency band if, in a range higher than the upper limit of the frequency bins of the receiving frequency band, the sum of the magnitudes of the bandwidth detection signals is less than or equal to a first threshold. This enables accurate bandwidth detection.

[0014] The system includes a double-talk detection unit that sequentially detects whether or not speech has been input to the sound detection sensor based on a double-talk detection mask, and a nonlinear echo suppression unit that performs processing to suppress nonlinear echoes in the signal from which linear echoes have been removed. In the case of single-talk (receiving only) and the receiving frequency band is the first frequency band, the nonlinear echo suppression unit may suppress nonlinear echoes more strongly for frequency bins not included in the first frequency band than for frequency bins included in the first frequency band. In other words, as a countermeasure according to the detected frequency band, the suppression of nonlinear echoes is changed according to the frequency band (first frequency band and second frequency band). This ensures that unwanted nonlinear echoes are reliably suppressed.

[0015] The double talk detection unit may, when the receiving frequency band is the first frequency band, use a first detection mask to detect whether or not speech has been input from the sound detection sensor for frequency bins included in the first frequency band, and use a second detection mask larger than the first detection mask to detect whether or not speech has been input from the sound detection sensor for frequency bins not included in the first frequency band. In other words, as a countermeasure corresponding to the detected frequency band, the mask for double talk detection is changed according to the frequency band (first frequency band and second frequency band). This prevents false detection of whether or not speech has been transmitted.

[0016] The system includes a volume adjustment unit that adjusts the volume of the signal transmitted through the receiver signal path, and the band-specific countermeasure unit may control the volume adjustment unit to gradually reduce the volume to a predetermined value when it detects that the receiver frequency band is the second frequency band. In this way, the overall volume can be reduced to improve the call quality.

[0017] The system includes a compressor for compressing the signal transmitted through the receiver signal path, and the band-specific countermeasures unit may control the compressor to gradually lower the compressor threshold at which the compressor begins compression when it is detected that the receiver frequency band is the second frequency band. In this way, the overall volume can be reduced to improve the call quality.

[0018] The system includes a linear echo removal unit that suppresses linear echoes generated when the sound output from the sound generation unit is input to the sound detection sensor. The linear echo removal unit may remove linear echoes using the results of a learning process performed with a first step size when it is detected that the receiving frequency band is the first frequency band, and may remove linear echoes using the results of a learning process performed with a second step size larger than the first step size when it is detected that the receiving frequency band is the second frequency band. In this way, by increasing the step size and bringing the echo output closer to zero, distortion can be brought closer to zero and the call quality can be improved.

[0019] The band-specific countermeasure unit generates the first detection mask and the second detection mask using an estimated echo function calculated based on a second learning receiving signal obtained by converting a learning receiving signal transmitted through the receiving-side signal path into the frequency domain, and a second learning signal obtained by converting a learning receiving signal transmitted through the transmitting-side signal path into the frequency domain when the sound output from the sound generation unit due to the learning receiving signal is input to the sound detection sensor, the estimated echo function having as variables the logarithm of the magnitude of the reference signal at each frequency, the frequency of the reference signal, the logarithm of the total received value which is the sum of the magnitudes of the reference signal or the sum of the reference signal in an arbitrary frequency range, and the logarithm of the envelope of the total received value. The first detection mask may be generated by inputting a signal generated using a sound source in the second frequency band into the receiving-side signal path. This makes it possible to obtain accurate first and second detection masks.

[0020] The second detection mask is generated by adding an additional value to the first detection mask, and the additional value may be generated by inputting a signal generated using a sound source in the first frequency band to the receiver-side signal path and acquiring an echo signal in the frequency band of the second frequency band. This reduces the number of parameters (coefficients) compared to when the first detection mask and the second detection mask are generated separately.

[0021] According to the present invention, good call quality can be ensured across multiple frequency bands without tuning each frequency band individually.

[0022] This is a schematic diagram of the voice communication system 100 including the acoustic signal processing device 1. This is a diagram showing the outline of the functional blocks of the acoustic signal processing device 1. This is a flowchart showing the processing (method 1) of the band detection unit 13. This is a schematic diagram showing the processing (method 1) for determining the frequency band. This is a flowchart showing the processing (method 2) of the band detection unit 13. This is a schematic diagram showing the processing (method 2) for determining the frequency band. This is a flowchart showing the processing (method 3) of the band detection unit 13. This is a schematic diagram showing the processing (method 3) for determining the frequency band. This is a schematic diagram showing the processing performed by the double talk detection unit 15. This is a diagram showing the outline of the functional blocks when determining the function for calculating the estimated echo in the acoustic signal processing device 1. This is an example of a scatter plot of the logarithm of the power spectrum at each frequency of the learning receiver signal and the logarithm of the power spectrum at each frequency of the learning signal. This is an example of a scatter plot of the logarithm of the frequency of the learning receiver signal and the logarithm of the power spectrum of the transmission. This is an example of a scatter plot of the logarithm of the total received power spectrum of the learning receiver signal and the logarithm of the power spectrum of the transmission. This is an example of a scatter plot of the logarithm of the envelope of the total received power spectrum of the learning received signal and the logarithm of the transmitted power spectrum. This is a flowchart showing the processing flow for double talk detection and nonlinear echo suppression according to the bandwidth. This is a flowchart showing a series of processing flows for determining the sum value to be added to the threshold T3. This is a flowchart showing a series of processing flows for determining thresholds T3 and T4. This is a diagram showing the schematic of the functional blocks of the acoustic signal processing device 2. This is a diagram showing the schematic of the functional blocks of the acoustic signal processing device 3. This is a diagram showing the schematic of the functional blocks of the acoustic signal processing device 4. This is a graph showing the frequency characteristics of the reference signal input from the communication device 53 to the acoustic signal processing device 1, with the horizontal axis being frequency and the vertical axis being magnitude. (B) is an enlarged view of a part of (A). This is an example of the relationship between the reference signal and the echo signal when learning (tuning) is performed in wideband, with (A) being the case when a wideband reference signal is input and (B) being the case when a narrowband reference signal is input.

[0023] Hereinafter, embodiments of the acoustic signal processing device according to the present invention will be described in detail with reference to the drawings. <First Embodiment> Figure 1 is a schematic diagram showing a voice communication system 100 including an acoustic signal processing device 1 according to the first embodiment. The voice communication system 100 mainly comprises a terminal 50 provided in a closed space 90, an acoustic signal processing device 1, and a communication device 53.

[0024] The terminal 50 has a sound detection sensor 51 (corresponding to the first sound detection sensor of the present invention) and a sound generation unit 52 (corresponding to the first sound generation unit of the present invention). The sound detection sensor 51 is a sensor capable of detecting sound, such as a microphone or a vibration sensor. The sound generation unit 52 is a component that generates sound by vibration or the like, such as a speaker or an exciter that produces sound by vibrating a contact surface. In this embodiment, each terminal 50 has one sound detection sensor 51 and one sound generation unit 52, but the number of sound detection sensors 51 and sound generation units 52 that a terminal 50 has is not limited to this.

[0025] The enclosed space 90 is a closed indoor space that is not open, such as a car cabin, an office, or a room in a house. In this embodiment, the enclosed space 90 is a car cabin. The terminal 50 is installed inside the enclosed space 90 and outputs sound into the enclosed space 90 or detects sound inside the enclosed space 90.

[0026] It is not essential that the sound detection sensor 51 and the sound generation unit 52 constitute the terminal 50. For example, the sound detection sensor 51 and the sound generation unit 52 may be separate components, and they may be provided at different locations inside the closed space 90.

[0027] The sound detection sensor 51 and sound generation unit 52 of the terminal 50 are each connected to the acoustic signal processing device 1. The acoustic signal processing device 1 is a device that includes a processing unit that executes a memory program containing a program for operating the terminal 50, and is, for example, a DSP (Digital Signal Processor). The acoustic signal processing device 1 may be integrated with the terminal 50 or may be provided in a separate location from the terminal 50. Furthermore, the acoustic signal processing device 1 may be provided in a location other than the enclosed space 90.

[0028] The acoustic signal processing device 1 functionally comprises an acoustic processing unit and a parameter storage unit. The acoustic processing unit is a functional unit that operates the program that enables each functional block (see Figure 2) of the acoustic signal processing device 1 to function. The parameter storage unit stores various parameters used by the acoustic processing unit.

[0029] The acoustic signal processing device 1 is connected to the communication device 53. The acoustic signal processing device 1 is a device that transmits signals in a predetermined frequency band (corresponding to the receiving frequency band of the present invention) acquired via the communication device 53 to the sound generation unit 52 via the receiving-side signal path (described in detail later) to produce sound from the sound generation unit 52, and acquires voice input via the sound detection sensor 51 via the transmitting-side signal path (described in detail later) and outputs it to the communication device 53.

[0030] The communication device 53 enables a near-end speaker A using terminal 50 to communicate with a far-end speaker B using terminal 54 (far-end terminal) located at a distance in the enclosed space 90. The sound generation unit 52 amplifies the voice signal from far-end speaker B input via terminal 54, and the sound detection sensor 51 collects the voice emitted by near-end speaker A on the near-end side and transmits it to far-end speaker B, allowing near-end speaker A to make a loudspeaker call (hands-free call) without holding terminal 50.

[0031] The communication device 53 enables voice communication using a predetermined frequency band. The predetermined frequency band is generally one of the following: narrowband (NB), wideband (WB), super wideband (SWB), or fullband (FB).

[0032] The frequency bands used for voice communication have expanded from narrowband to wideband, superwideband, and fullband. Currently, voice communication uses various frequency bands, and the acoustic signal processing device 1 needs to support all frequency bands. To this end, conventionally, tuning (adjustment work such as parameter setting) was performed for each frequency band in advance, the acoustic signal processing device 1 stored the results, and the acoustic signal processing device 1 performed acoustic processing based on those results.

[0033] However, increasing the number of tuning cycles increases the adjustment, evaluation, and management workloads, leading to increased time and costs. Furthermore, tuning requires specialized knowledge of signal processing, thus demanding highly skilled personnel and time. For this reason, attempts are being made to reduce the number of adjustments and parameters by performing tuning over a wider frequency band (e.g., full band) and using the tuning results from the wider frequency band for narrower frequency bands (e.g., narrowband).

[0034] However, when nonlinear echoes or reflection components are large, it can be difficult to use the results of tuning over a wide frequency range in a narrow frequency range. For example, vehicles are equipped with on-board devices in which the microphone and speaker are integrated into a single housing. However, on-board devices are prone to nonlinear echoes due to distortion and vibration of the speaker and amplifier, and reflections occur inside the housing, so the results of tuning over a wide frequency range may not be usable in a narrow frequency range. This will be explained using Figures 15 and 16.

[0035] Figure 21 is a graph showing the frequency characteristics of the reference signal input from the communication device 53 to the acoustic signal processing device 1, with the horizontal axis representing frequency and the vertical axis representing magnitude. Figure 21(B) is an enlarged view of a portion of Figure 21(A). In the in-vehicle hands-free emergency call system (ITU-T P.1140 NB / WB), the total volume of the wideband sound (gray line in Figure 21) and the total volume of the narrowband sound (black line in Figure 21) are the same (-16dB). m0 As a result, in the central frequency band, the narrowband sound is slightly louder than the wideband sound. For example, in automotive equipment, even a slight increase in volume in a specific frequency band can increase nonlinear echoes, so using the results of a wideband (e.g., wideband) adjustment in a narrowband (e.g., narrowband) environment may degrade call quality.

[0036] Figure 22 shows an example of the relationship between the reference signal and the echo signal when learning (tuning) in wideband mode. (A) shows the case when a wideband reference signal is input, and (B) shows the case when a narrowband reference signal is input. Here, the echo signal is the signal that is detected by the sound detection sensor 51 and input to the acoustic signal processing device 1 when the reference signal is output as sound from the sound generation unit 52 via the acoustic signal processing device 1. It includes both speech and distortion. In Figure 22, the gray line represents the reference signal, and the black line represents the echo signal. The horizontal axis in Figure 22 is frequency, and the vertical axis is magnitude. In Figure 22, the high-frequency band of 4 kHz is enclosed by a dotted line.

[0037] As shown in Figure 22(A), when the reference signal is wideband, the increase in the echo signal relative to the reference signal at 7 kHz is an average of +14 dB. In other words, in the high-frequency range, the sound (magnitude of the reference signal) is greater than the distortion (magnitude of the echo signal), and the distortion is less affected by the distortion because it is buried in the sound.

[0038] In contrast, as shown in Figure 22(B), when the reference signal is narrowband, no reference signal is input in the high-frequency band (it is -120 (negative infinity) in Figure 22), so the echo signal should also be -120. However, in reality, an echo signal of about -100 is input. This is a problem that occurs because the results of tuning for wideband are applied to narrowband, indicating that echo rejection learning has not been performed correctly. If noise (such as voice distortion) is introduced into the reference signal under these circumstances, the call quality may deteriorate.

[0039] In this embodiment, the acoustic signal processing device 1 can address this problem by detecting the bandwidth of the reference signal (receiver frequency band) and performing processing according to the detection result (details will be described later).

[0040] Return to the description of FIG. 1. The acoustic signal processing apparatus 1 may be constituted by, for example, an arithmetic unit such as a CPU (Central Processing Unit) for executing information processing, a computer system including storage devices such as a RAM (Random Access Memory) and a ROM (Read Only Memory), a dedicated board mounted on various terminals (for example, in-vehicle devices, conference systems, mobile terminals), and software (tuning program). Further, the acoustic signal processing program may be prestored in an HDD as a storage medium built in a device such as a computer, a ROM in a microcomputer having a CPU, etc., and installed in the computer therefrom. Further, the acoustic signal processing program may be temporarily or permanently stored (stored) in a removable storage medium such as a semiconductor memory, a memory card, an optical disk, a magneto-optical disk, a magnetic disk, etc.

[0041] FIG. 2 is a diagram showing an outline of functional blocks of the acoustic signal processing apparatus 1. Functionally, the acoustic signal processing apparatus 1 mainly includes an echo cancellation unit 11, frequency analyzers (FFT units) 12 and 18, a band detection unit 13, a band-specific countermeasure unit 14, a double talk detection unit 15, a non-linear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. In FIG. 2, the upper signal path is a transmission-side signal path for transmitting an input signal input from the sound detection sensor 51, and the lower signal path is a reception-side signal path for transmitting a signal to the sound generation unit 52. Note that the functional components of the acoustic signal processing apparatus 1 may be further classified into more components according to the processing content, or one component may execute the processing of a plurality of components.

[0042] The echo cancellation unit 11 is, for example, a functional unit that cancels an echo using an adaptive filter. The echo cancellation unit 11 updates filter coefficients according to a given procedure, generates a pseudo echo signal from the signal transmitted through the reception-side signal path, and subtracts the pseudo echo signal from the signal transmitted through the transmission-side signal path to cancel the echo. Note that since the adaptive filter is already known, the description thereof is omitted.

[0043] In addition, in this embodiment, an adaptive filter is applied to the echo cancellation unit 11, but other known echo cancellation techniques can also be applied to the echo cancellation unit 11. Also, in this embodiment, the echo cancellation unit 11 is not essential, but since it is possible to more accurately detect the presence of near-end speech (speech of the near-end speaker A (see FIG. 1)), it is desirable to provide the echo cancellation unit 11.

[0044] The frequency analyzers (FFT units) 12 and 18 are functional units that perform fast Fourier transform (FFT, Fast Fourier Transform) on signals. The FFT unit 12 performs fast Fourier transform on the signal transmitted through the transmission-side signal path, here the signal that has passed through the echo cancellation unit 11, and the FFT unit 18 performs fast Fourier transform on the reference signal transmitted through the reception-side signal path. The FFT units 12 and 18 convert a signal arranged in time series (time domain) into a signal represented by a set of frequencies (frequency domain). Hereinafter, a signal dependent on time is indicated by...[t], and a signal dependent on frequency is indicated by...[i].

[0045] Note that the fast Fourier transform is a calculation algorithm devised for the discrete Fourier transform (DFT) to speed up the calculation, and similar results can be obtained whether using FFT or DFT. In this embodiment, FFT is used as the frequency conversion method, but DFT can also be used.

[0046] The band detection unit 13 is a functional unit that detects the frequency band of a signal (corresponding to the first frequency band of the present invention) acquired via the communication device 53 based on the reference signal transmitted through the reception-side signal path. In this embodiment, the band detection unit 13 detects the frequency band based on the reference signal (corresponding to the band detection signal of the present invention) converted into a signal in the frequency domain by the FFT unit 18. By using the reference signal converted into a signal in the frequency domain by the FFT unit 18, the band detection unit 13 can achieve a high frequency resolution with a low calculation amount, and as a result, the accuracy of band detection is improved.

[0047] For example, the bandwidth detection unit 13 detects whether the frequency band is a narrowband (corresponding to the first frequency band of the present invention) or a wideband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the narrowband. Alternatively, the bandwidth detection unit 13 may detect whether the frequency band is a narrowband wideband (corresponding to the first frequency band of the present invention) or a superwideband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the wideband, or whether the frequency band is a narrowband superwideband (corresponding to the first frequency band of the present invention) or a fullband (corresponding to the second frequency band of the present invention) which is a wider frequency band than the superwideband. Furthermore, the bandwidth detection unit 13 can combine the above embodiments to detect which of the three frequency bands—narrowband, wideband, and superwideband—is being used, or which of the three frequency bands—wideband, superwideband, and fullband—is being used. The bandwidth detection unit 13 will be described in detail later.

[0048] The bandwidth-specific countermeasures unit 14 is a functional unit that performs countermeasures according to the detected frequency band based on the detection results of the bandwidth detection unit 13. For example, if the bandwidth-specific countermeasures unit 14 detects that the reference signal is in a wider frequency band (second frequency band), it performs different processing (countermeasures for the second frequency band) than when it is detected as a narrowband (first frequency band) signal. The bandwidth-specific countermeasures unit 14 will be described in detail later.

[0049] The double talk detection unit 15 is a functional unit that detects whether or not a double talk state is in place based on the signal input from the FFT unit 12 and the reference signal input from the FFT unit 18. Here, a double talk state is a state in which both the near-end speaker A and the far-end speaker B are speaking, that is, a signal containing speech is being transmitted simultaneously to both the receiving-side signal path and the transmitting-side signal path.

[0050] The nonlinear echo suppression unit 16 performs echo suppression processing (processing to strongly suppress echoes) on the input signal. The nonlinear echo suppression unit 16 performs different processing based on the detection result of the double talk detection unit 15 (described in detail later). For example, the nonlinear echo suppression unit 16 enables echo suppression processing when it is a single talk consisting only of far-end utterances, and disables it in other cases. Also, for example, the nonlinear echo suppression unit 16 strongly suppresses echoes when it is a single talk consisting only of far-end utterances, and weakly suppresses echoes in other cases. Since echo suppression processing is already known, a detailed explanation will be omitted. The nonlinear echo suppression unit 16 receives the result of detecting whether or not a double talk state is present from the double talk detection unit 15 at unit time intervals. Therefore, the nonlinear echo suppression unit 16 switches between enabling and disabling echo suppression processing at unit time intervals.

[0051] Furthermore, the double-talk detection unit 15 and the nonlinear echo suppression unit 16 perform different processing depending on whether the signal is a narrowband signal (corresponding to the first frequency band of the present invention) or a signal with a wider frequency band (corresponding to the second frequency band of the present invention), based on the countermeasures taken by the band-specific countermeasures unit 14 (details will be described later).

[0052] The IFFT unit 17 performs an inverse FFT (IFFT) on the input signal that has passed through the nonlinear echo suppression unit 16.

[0053] The volume control unit 20 is a functional unit that adjusts the gain of the input signal. Specifically, the volume control unit 20 adjusts the degree to which the input signal is amplified (gain) to output a signal of the desired level (magnitude). The volume control unit 20 adjusts the gain according to the magnitude of the input signal on the receiving side (communication device 53). The volume control unit 20 is, for example, an AGC (Auto Gain Control) circuit, and since it is already known, a detailed explanation is omitted. However, the volume control unit 20 is not essential in this embodiment.

[0054] The compressor 21 is a functional unit that adjusts the dynamic range (sound variation range) of the signal output from the volume control unit 20. The compressor 21 also changes its compression characteristics based on noise and other environmental factors in which the terminal 50 is installed. Since the compressor 21 is already known, its description is omitted. However, the compressor 21 is not essential in this embodiment.

[0055] Next, the processing of the bandwidth detection unit 13 will be explained in detail using a specific example. The bandwidth detection unit 13 detects the frequency band of the reference signal using one of the following methods 1, 2, or 3.

[0056] <Method 1> Figure 3 is a flowchart showing the processing flow of the bandwidth detection unit 13 (Method 1). First, the bandwidth detection unit 13 associates the upper limit of each frequency band (i_Band[i]_Up) with the frequency bin (step S11). For example, the i in the narrowband upper limit i_Band[0]_Up is 109, and the i in the wideband i_Band[1]_Up is 224.

[0057] Next, the bandwidth detection unit 13 detects the largest frequency bin that exceeds the threshold T1 for the acquired reference signal (step S12). The threshold T1 may be set experimentally in advance.

[0058] Next, the bandwidth detection unit 13 compares the upper limit and frequency bin of each frequency band with the largest frequency bin that exceeds the threshold T1 to determine the frequency band of the reference signal (step S13). For example, if the largest frequency bin that exceeds the threshold T1 is less than or equal to the upper limit of an arbitrary frequency band (frequency band A), the bandwidth detection unit 13 determines that the frequency band of the reference signal is frequency band A. If the largest frequency bin that exceeds the threshold T1 is not less than or equal to the upper limit of an arbitrary frequency band (frequency band A), the unit performs a comparison and determination for the next frequency band.

[0059] The bandwidth detection unit 13 performs comparisons starting from the upper frequency bin of the narrow frequency band. That is, it first compares with the upper limit i_Band[0]_Up of the narrowband, then with i_Band[1]_Up of the wideband, then with i_Band[2]_Up of the superwideband, and finally with i_Band[3]_Up of the fullband. Note that the comparison with i_Band[3]_Up of the fullband is not mandatory.

[0060] The process in step S13 will be explained using a specific example. Figure 4 is a schematic graph showing the process of determining the frequency band. For example, the signal shown by the black line graph in Figure 4 has a maximum frequency bin i that exceeds the threshold T1 of 108. First, the band detection unit 13 compares i=108 with the upper limit of the narrowband i=109, and since the upper limit of this signal (i=108) is smaller than the upper limit of the narrowband i=109, it determines that this signal is a narrowband signal (the maximum frequency bin that exceeds the threshold T1). <i_Band[0]_Up:NB)。

[0061] For example, the signal shown by the gray line graph in Figure 3 has a maximum frequency bin i above the threshold T1 of 223. First, the bandwidth detection unit 13 compares i=223 with the narrowband upper limit i=109 and detects that this signal is in a wider frequency band than the narrowband because the upper limit of this signal (i=223) is greater than the narrowband upper limit i=109. Next, the bandwidth detection unit 13 compares i=223 with the wideband upper limit i=224, which is the next widest band after the narrowband, and determines that this signal is a wideband signal because the upper limit of this signal (i=223) is smaller than the wideband upper limit i=224 (i_Band[0]_Up ≤ maximum frequency bin above threshold T1 < i_Band[1]_Up ≤ WB).

[0062] Furthermore, if the bandwidth detection unit 13 detects that a certain signal (signal I) is a signal with a frequency band wider than the wideband, it will determine the largest frequency bin i of signal I that exceeds the threshold T1. I The frequency bin i of signal I is compared with the upper frequency bin of the superwideband, which is the next widest band after wideband. IA signal is determined to be a superwideband signal when its frequency is smaller than the upper frequency bin of the superwideband (i_Band[1]_Up ≤ the largest frequency bin above the threshold T1 < i_Band[2]_Up: SWB).

[0063] Furthermore, if the bandwidth detection unit 13 detects that a certain signal (signal II) is a signal with a frequency band wider than the wideband, it will determine the largest frequency bin i of signal II that exceeds the threshold T1. II We compare this with the upper frequency bin of the superwideband, which is the next widest band, and the frequency bin i of signal II. II If the signal is greater than the upper frequency bin of the superwideband, it is determined that this signal is a full-band signal (the largest frequency bin above the threshold T1 ≥ i_Band[2]_Up: FB).

[0064] Note that the frequency bin i of signal II II When the frequency bin i is greater than the upper frequency bin of the super wideband, the bandwidth detection unit 13 detects the frequency bin i II We compare this with the upper frequency bin of the full band, which is the next widest band after the super wideband, and frequency bin i II A signal may be determined to be a full-band signal if it is smaller than the upper frequency bin of the full band (i_Band[2]_Up ≤ maximum frequency greater than or equal to T1 < i_Band[3]_Up: FB).

[0065] Generally, the difference in frequency bands (the difference between narrowband and wideband) is mainly on the high-frequency side, with little difference on the low-frequency side. Furthermore, the reference signal is often buried in noise on the low-frequency side. Therefore, by comparing the upper limit and frequency bin of each frequency band with the largest frequency bin exceeding the threshold T1, the frequency band of the reference signal can be determined. Using only the high-frequency side allows for accurate determination while reducing processing load.

[0066] <Method 2> Figure 5 is a flowchart showing the processing flow of the bandwidth detection unit 13 (Method 2). First, the bandwidth detection unit 13 associates the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) of each frequency band with the frequency bin (step S14).

[0067] Next, the bandwidth detection unit 13 obtains the number of frequency bins that exceed the threshold T2 for each frequency band, within the range outside the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) (below the lower limit, above the upper limit) for the acquired reference signal (step S15). The threshold T2 may be set experimentally in advance.

[0068] Next, the bandwidth detection unit 13 compares the number of frequency bins acquired in step S15 with a predetermined threshold N to determine the frequency bandwidth of the reference signal (step S16). For example, the bandwidth detection unit 13 determines that the frequency band is A if the number of frequency bins exceeding the threshold T2 outside the range outside of the arbitrary frequency band (frequency band A) acquired in step S15 is less than or equal to the threshold N. If the number exceeds the threshold N, it performs a comparison over a wider frequency band than frequency band A.

[0069] The bandwidth detection unit 13 performs comparisons in order from the narrowest frequency band. Specifically, it first compares the number of frequency bins above threshold T2 with threshold N in the range outside the narrowband, then compares the number of frequency bins above threshold T2 with threshold N in the range outside the wideband, then compares the number of frequency bins above threshold T2 with threshold N in the superwideband, and finally compares the number of frequency bins above threshold T2 with threshold N in the fullband. Note that fullband comparison is not mandatory.

[0070] Figure 6 schematically illustrates the process of obtaining the number of frequency bins that exceed the threshold T2. In the case of the signal shown by the black line graph in Figure 6, the number of frequency bins n1 that exceed the threshold T2 in the range outside the lower limit (i_Band[0]_Lo=2) and upper limit (i_Band[0]_Up=109) of the narrowband (shaded area in Figure 6) is 1. Since the number n1 is smaller than the threshold N (for example, 5), it can be seen that the signal shown by the black line graph in Figure 6 is narrowband (number of bins below i_Band[0]_Lo and above i_Band[0]_Up that are above the threshold T2 < N: NB). Furthermore, it is also possible to determine that a signal is narrowband when the number of frequency bins exceeding the threshold T2 in the range between the lower limit of the narrowband and the lower limit of the wideband, and between the upper limit of the narrowband and the upper limit of the wideband, is smaller than the threshold N (number of bins greater than or equal to the threshold T2 in the range between i_Band[0]_Lo and greater than i_Band[1]_Lo and greater than or equal to i_Band[0]_Up and less than i_Band[1]_Up (out-of-band 1) < N: NB).

[0071] In the case of the signal shown by the gray line graph in Figure 6, the number of frequency bins n2 that exceed the threshold T2 in the range outside the lower and upper limits of the narrowband is 115. Since the number n2 is greater than the threshold N, it can be seen that this signal is a signal in a frequency band wider than the narrowband. Next, for the signal shown by the gray line graph, the number of frequency bins n3 that exceed the threshold T2 in the range outside the lower and upper limits of the wideband is obtained. Since the number n3 (for example, 1) is smaller than the threshold N, it can be seen that the signal shown by the gray line graph in Figure 6 is wideband (number of bins greater than or equal to the threshold T2 in the range below i_Band[1]_Lo and above i_Band[1]_Up < N: WB). Furthermore, it is also possible to determine that a signal is wideband when the number of frequency bins exceeding the threshold T2 in the range between the lower limit of the wideband and the lower limit of the superwideband, and between the upper limit of the wideband and the upper limit of the superwideband, is smaller than the threshold N (number of bins exceeding the threshold T2 in the range between i_Band[1]_Lo and i_Band[2]_Lo and between i_Band[1]_Up and i_Band[2]_Up (out-of-band 2) < N: WB).

[0072] Furthermore, if the bandwidth detection unit 13 detects that a certain signal (signal I) is a signal in a frequency band wider than wideband, it obtains the number n4 of frequency bins that exceed the threshold T2 in the range outside the lower and upper limits of the superwideband for signal I. When the number n4 is less than the threshold N, it determines that signal I is superwideband (number of bins above the threshold T2 in the i_Band[2]_Up or higher (out-of-band 3) < N: SWB), and when the number n4 is greater than or equal to the threshold N, it determines that signal I is fullband (number of bins above the threshold T2 in the i_Band[2]_Up or higher (out-of-band 3) ≥ N: FB).

[0073] The bandwidth detection unit 13 may also obtain the number n5 of frequency bins that exceed the threshold T2 in the range between the lower limit of the superwideband and the lower limit of the fullband, and between the upper limit of the superwideband and the upper limit of the fullband, for the signal I. When the number n5 is less than the threshold N, the signal I is considered to be in the superwideband (number of bins with threshold T2 or higher in the range between i_Band[2]_Lo and i_Band[3]_Lo and between i_Band[2]_Up and i_Band[3]_Up (out-of-band 3) < N: SWB), and when the number n5 is greater than or equal to the threshold N, the signal I is considered to be in the fullband (number of bins with threshold T2 or higher in the range between i_Band[2]_Lo and i_Band[3]_Lo and i_Band[2]_Up and i_Band[3]_Up (out-of-band 3) ≥ N: FB).

[0074] Generally, the difference in frequency bands is primarily in the high-frequency range, with little difference in the low-frequency range. Therefore, it is not essential to compare the number of frequency bins exceeding the threshold T2 with the threshold N in the range outside the frequency band (narrowband, etc.) (low-frequency and high-frequency ranges). It is also acceptable to compare the number of frequency bins exceeding the threshold T2 with the threshold N only in the high-frequency range of the frequency band. By using only the high-frequency range, it is possible to reduce the processing load while maintaining accuracy.

[0075] <Method 3> Figure 7 is a flowchart showing the processing flow of the bandwidth detection unit 13 (Method 3). First, the bandwidth detection unit 13 associates the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) of each frequency band with the frequency bin (step S14).

[0076] Next, the bandwidth detection unit 13 calculates the sum of volume power for each frequency band of the acquired reference signal within the range outside the lower limit (i_Band[i]_Lo) and upper limit (i_Band[i]_Up) (below the lower limit, above the upper limit) (step S17).

[0077] Next, the bandwidth detection unit 13 compares the sum of volume power calculated in step S17 with a predetermined threshold M to determine the frequency band of the reference signal (step S18). For example, if the sum of volume power in the range outside of the arbitrary frequency band (frequency band A) acquired in step S17 is less than or equal to the threshold M, it is determined that the reference signal is in frequency band A. If it exceeds the threshold M, a comparison is made with the next widest frequency band after frequency band A.

[0078] The bandwidth detection unit 13 performs comparisons in order from the narrowest frequency band. Specifically, it first compares the sum of sound power with the threshold M in the range outside the narrowband, then compares the sum of sound power with the threshold M in the range outside the wideband, then compares the sum of sound power with the threshold M in the superwideband, and finally compares the sum of sound power with the threshold M in the fullband. Note that fullband comparison is not mandatory.

[0079] Figure 8 schematically illustrates the process for calculating the sum of volume power. In the case of the signal shown by the black line graph in Figure 8, the sum of volume power S1 is calculated in the range outside the lower limit (i_Band[0]_Lo=2) and upper limit (i_Band[0]_Up=109) of the narrowband (shaded area in Figure 6). Since the sum S1 (for example, 1) is smaller than the threshold M (for example, 5), it can be seen that the signal shown by the black line graph in Figure 8 is narrowband (sum S1 < M for i_Band[0]_Lo and i_Band[0]_Up: NB). Furthermore, a signal may be determined to be narrowband if the sum of volume power S1 in the range between the lower limit of the narrowband and the lower limit of the wideband, and between the upper limit of the narrowband and the upper limit of the wideband, is less than the threshold M (sum S1 < M in the range between i_Band[0]_Lo and i_Band[1]_Lo and between i_Band[0]_Up and i_Band[1]_Up (out-of-band 1)).

[0080] In the case of the signal shown by the gray line graph in Figure 6, the sum of volume power S2 (e.g., 10) is greater than the threshold M in the range outside the lower and upper limits of the narrowband, indicating that this signal is in a frequency band wider than the narrowband. Next, for the signal shown by the gray line graph, the sum of volume power S3 is calculated in the range outside the lower and upper limits of the wideband. Since the sum S3 (e.g., 3) is less than or equal to the threshold M, the signal shown by the gray line graph in Figure 8 is wideband (sum S2 < M in the range below i_Band[1]_Lo and above i_Band[1]_Up: WB). Furthermore, a signal may be determined to be wideband if the sum S2 in the range between the lower limit of the wideband and the lower limit of the superwideband, and between the upper limit of the wideband and the upper limit of the superwideband, is less than the threshold M (sum S2 < M in the range between i_Band[1]_Lo and i_Band[2]_Lo and between i_Band[1]_Up and i_Band[2]_Up (out-of-band 2): WB).

[0081] Furthermore, if the bandwidth detection unit 13 detects that a certain signal (signal I) is a signal in a frequency band wider than wideband, it obtains the sum of volume power S3 for signal I in the range outside the lower and upper limits of the superwideband. When the sum S3 is less than the threshold M, it determines that signal I is superwideband (sum S3 < M in the range above i_Band[2]_Up (outside the band 3): SWB), and when the sum S3 is greater than or equal to the threshold M, it determines that signal I is fullband (sum S3 ≥ M in the range above i_Band[2]_Up (outside the band 3): FB).

[0082] The bandwidth detection unit 13 may also acquire the total volume power S4 for the signal I within the range between the lower limit of the super-wideband and the lower limit of the fullband, and between the upper limit of the super-wideband and the upper limit of the fullband. When the total S4 is less than the threshold M, the signal I is considered to be in the super-wideband (sum S4≧M in the range of i_Band[2]_Lo or less and greater than i_Band[3]_Lo and greater than or equal to i_Band[2]_Up and less than i_Band[3]_Up (out-of-band 3) : SWB). When the total S4 is greater than or equal to the threshold M, the signal I is considered to be in the fullband (sum S3≧M in the range of i_Band[2]_Lo or less and greater than i_Band[3]_Lo and greater than or equal to i_Band[2]_Up and less than i_Band[3]_Up (out-of-band 3) : FB).

[0083] Generally, the difference in frequency bands is primarily in the high-frequency range, with little difference in the low-frequency range. Therefore, it is not essential to compare the sum of volume power with the threshold M in the range outside the frequency band (low-frequency and high-frequency ranges); the comparison can be performed only in the high-frequency range.

[0084] Using one of the methods 1, 2, or 3 described above, the bandwidth detection unit 13 detects the frequency band of the reference signal, i.e., the signal acquired via the communication device 53. Method 1 is the simplest and requires the least computation, but there is a risk of misjudgment due to noise. Method 2 is more accurate than Method 1, but there is a risk of misjudgment if the frequency bins are coarse (low resolution). Method 3 requires the most computation but is the most accurate. In this embodiment, the bandwidth detection unit 13 detects the frequency band using Method 3.

[0085] Next, the bandwidth-specific countermeasure unit 14 will be described. The bandwidth-specific countermeasure unit 14 calculates a threshold value corresponding to the frequency band and outputs it to the double-talk detection unit 15 and the nonlinear echo suppression unit 16.

[0086] First, the processing performed by the double talk detection unit 15 and the nonlinear echo suppression unit 16 using the threshold values ​​determined by the band-specific countermeasures unit 14 will be explained using a specific example. Figure 9 schematically shows the processing performed by the double talk detection unit 15. In Figure 9, the horizontal axis represents time, and the vertical axis represents the power of the signal (suppressed signal) after the linear echo has been removed by the echo removal unit 11. The light gray bar graph in Figure 9 represents the echo component, and the dark gray represents the speech component within the closed space 90. Figure 9 also illustrates the power of the frequency bins that have exceeded the upper limit of the narrowband frequency bins when the band detection unit 13 detects that it is a narrowband.

[0087] Typically, for example, when the reference signal is a sound in the narrowband frequency range, a narrowband double-talk detection mask (corresponding to the first detection mask of the present invention, threshold T3 in Figure 9) is used to detect whether or not speech has been input (whether or not there is transmission) from the sound detection sensor 51 for signals in the narrowband frequency range among the suppressed signals.

[0088] In contrast, when the reference signal is in the narrowband frequency range, an echo signal may be input to signals outside the narrowband frequency range of the suppressed signal (for example, in the high-frequency band), even though no sound should be present (see the area enclosed by the dashed line in Figure 22(B)). In this way, in frequency bands outside the frequency band detected by the band detection unit 13, the echo signal becomes larger than it should be, which can cause the power of the suppressed signal to exceed the threshold T3 even though only an echo signal is present (near-end utterance, i.e., no transmission), potentially leading to a false detection of transmission (see the circle in Figure 9).

[0089] Therefore, when the reference signal is a sound in the narrowband frequency range, for signals in the frequency band outside the narrowband frequency range of the suppressed signal, a threshold T4 (corresponding to the second detection mask of the present invention) greater than the threshold T3, which is the double-talk detection mask in the narrowband frequency band, is used to detect whether or not speech has been input from the sound detection sensor 51. In this embodiment, threshold T4 is made greater than threshold T3 by adding an arbitrary fixed value to threshold T3.

[0090] Here, we will explain the threshold T3. The threshold T3 is determined based on a pre-determined estimated echo function. Figure 10 is a schematic diagram of the functional blocks used in the acoustic signal processing device 1 to determine the function for calculating the estimated echo. The acoustic signal processing device 1 functionally includes an estimated echo calculation unit 25.

[0091] The calculation process for estimated echoes will now be explained in detail. First, after the echo removal unit 11 has sufficiently completed learning the adaptive filter, under conditions where there is no near-end utterance and background noise is sufficiently low, a learning received signal is transmitted to the receiving-side signal path, and a far-end one-sided utterance (single talk) is repeatedly performed in which the sound generation unit 52 outputs sound based on the learning received signal. The signal transmitted to the transmitting-side signal path during single talk is then used as the learning signal. In the acoustic signal processing device 1, the signal from which the echo has been removed by the echo removal unit 11 becomes the learning signal. A sound source with a wide frequency band is used for the far-end one-sided utterance. As a result, the learning received signal and the learning signal also have a wide frequency band. For example, when the acoustic signal processing device 1 is used in narrowband and wideband modes, a wideband sound source is used for the far-end one-sided utterance.

[0092] A time-dependent learning signal (hereinafter referred to as learning signal [t]) is converted by the FFT unit 12 into a frequency-dependent learning signal (hereinafter referred to as learning signal [i]) and input to the estimated echo calculation unit 25. Similarly, a time-dependent learning received signal (hereinafter referred to as learning received signal [t]) is converted by the FFT unit 18 into a frequency-dependent learning received signal (hereinafter referred to as learning received signal [i]) and input to the estimated echo calculation unit 25.

[0093] The estimated echo calculation unit 25 calculates the power spectra of the learning signal [i] and the learning receiver signal [i] at regular intervals to obtain multiple learning power spectra. Here, a regular interval is a predetermined time domain arbitrarily defined. The power spectrum P[i] is expressed as the square of the Fourier spectrum X[i] obtained by the Fast Fourier Transform (see formula (1)). P[i]=|X[i]|²=|X[i]|×|X[i]| ... (1)

[0094] The estimated echo calculation unit 25 creates multiple scatter plots of the learning signal [i] and the learning received signal [i] based on the learning signal [i], the learning received signal [i], and the learning power spectrum. Even if the power spectrum of the learning signal is the same, the power spectrum of the learning signal, i.e., the echo, will vary. Therefore, in this embodiment, the estimated echo is calculated not only based on the power spectrum of the learning signal, but also based on multiple scatter plots with different horizontal axes.

[0095] Here, the power spectrum at each frequency of the learning signal refers to the power spectrum of the echo produced by the learning receiver signal. The total receiver power spectrum is the same as the sum of the power spectra at each frequency of the learning signal, i.e., the sum of the power spectra of the learning receiver signal [t] before passing through the FFT unit 18, and is expressed by the following formula (2).

[0096]

[0097] The total received power spectrum may also be the sum of the power spectra at each frequency within an arbitrary frequency range of the learning signal. In this case, the total received power spectrum is expressed by the following formula (3). Here, A is greater than or equal to 0, and B is less than the maximum frequency (A > 0, B < F_MAX).

[0098] When the double talk detection unit 15 performs speech detection (described in detail later), it may be more accurate to use the sum of the power spectra of an arbitrary frequency range of the learning signal (equation (3)) rather than the sum of the power spectra of all frequencies of the learning signal (equation (2)). Therefore, in such cases, it is desirable for the estimated echo calculation unit 25 to use equation (3) to obtain the total received power spectrum.

[0099] A certain relationship exists between the logarithmic and frequency information of the learning receiver signal and the power spectrum of the learning signal, i.e., the echo. In this embodiment, a sufficient amount of learning signal [i] and learning receiver signal [i] are acquired in advance, and the estimated echo amount is determined based on the certain relationship between them. The estimated echo calculation unit 25 calculates the estimated echo function using the following formula (4). The estimated echo function (estimated echo power spectrum [i]) is a frequency-dependent signal and is expressed as a function with variables the logarithm of the magnitude of the learning receiver signal at each frequency, the frequency of the learning receiver signal, the logarithm of the total receiver power spectrum of the learning receiver signal, and the logarithm of the envelope of the total receiver value of the learning receiver signal.

[0100] Estimated echo power spectrum [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum ... (4)

[0101] The calculation of the estimated echo function will be explained in detail using Figures 11 to 14. The estimated echo calculation unit 25 sequentially calculates the coefficient α of the received power spectrum [i], the coefficient β of the frequency, the coefficient γ of the total received power spectrum, and the coefficient δ of the envelope of the total received power spectrum. The coefficients α, β, γ, and δ of each variable are obtained based on data obtained by removing outliers from the learning signal [i].

[0102] Figure 11 is a scatter plot of the logarithm of the power spectrum at each frequency of the learning receiver signal (hereinafter referred to as the receiver power spectrum) and the logarithm of the power spectrum at each frequency of the learning signal (hereinafter referred to as the transmission power spectrum). In Figure 11, the measured data is plotted, and α is shown as a line.

[0103] α represents the relationship between the logarithm of the received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. α is determined by removing outliers from a scatter plot of the logarithms of the received and transmitted power spectra. α can be expressed as a linear function (without conditional branching) or a nonlinear function (with conditional branching).

[0104] As shown in Figure 11, a larger power spectrum in the receiver does not necessarily mean a larger power spectrum in the transmitter (i.e., echo); rather, the echo decreases when the power spectrum of the receiver exceeds a certain level. This is due to the characteristics of the sound generation unit 52 (there is a region where sound cannot be produced) and the fact that the echo removal unit 11 is provided before the FFT unit 12. In the example shown in Figure 5, α is given by equations (5) and (6). Thus, α is a nonlinear function.

[0105] When the logarithm of the power spectrum of the receiver is less than -1, α = 0.5 × logarithm of the power spectrum of the receiver - 0.5 ... (5)

[0106] When the logarithm of the power spectrum of the receiver is greater than or equal to -1, α = -1.0 × logarithm of the power spectrum of the receiver - 2.0 ... (6)

[0107] If the echo removal unit 11 is not provided, the peak of the line representing α will shift to the right compared to the example shown in Figure 11, and the slope of the descending line after the peak will be smaller, but α will still be a nonlinear function (with conditional branching).

[0108] Once α is calculated, the estimated echo calculation unit 25 calculates β. Figure 12 is a scatter plot of the logarithm of the frequency of the learning receiver signal (hereinafter referred to as the receiver frequency) and the logarithm of the power spectrum of the transmission. In Figure 12, the result of subtracting the α component from the measured data is plotted, and β is shown as a line.

[0109] β represents the relationship between the receiver frequency and the maximum value of the logarithm of the transmit power spectrum. β is determined by removing outliers from a scatter plot of the receiver frequency and the logarithm of the transmit power spectrum. β can be expressed as a linear or nonlinear function.

[0110] The sound generating unit 52 has the characteristic of being unable to produce low or high frequencies, so in Figure 11, the echo is small for low and high frequencies. Also, when the terminal 50 is installed inside a vehicle, as shown in Figure 12, there is a dip around 1 kHz where the echo is small due to the influence of the intermediate environment (reflection, etc.). Therefore, β is a nonlinear function.

[0111] Once β is calculated, the estimated echo calculation unit 25 calculates γ. Figure 13 is a scatter plot of the logarithm of the total received power spectrum of the learning received signal and the logarithm of the transmitted power spectrum. In Figure 13, the results obtained by subtracting the α and β components from the measured data are plotted, and γ is shown as a line.

[0112] For example, when the sound generation unit 52 outputs sounds of 100 Hz and 110 Hz, the sound generation unit 52 may also emit a sound of 105 Hz in addition to 100 Hz and 110 Hz. Therefore, in order to refer to information on whether sounds other than the intended frequency are being emitted, in this embodiment, a term with the logarithm of the total received power spectrum as a variable is added to the estimated echo function (formula (4)).

[0113] γ represents the relationship between the logarithm of the total received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. γ is determined by removing outliers from a scatter plot of the logarithms of the received frequency and transmitted power spectrum. γ can be expressed as a linear or nonlinear function. In the example shown in Figure 13, γ is a nonlinear function.

[0114] Once γ is calculated, the estimated echo calculation unit 25 calculates δ. Figure 14 is a scatter plot of the logarithm of the envelope of the total received power spectrum of the learning received signal and the logarithm of the transmitted power spectrum. In Figure 14, the results obtained by subtracting the α, β, and γ components from the measured data are plotted, and δ is shown as a line.

[0115] Because sound reflections inside the vehicle and vibrations of the sound generating unit 52 are output as sound from the sound generating unit 52, echoes can exist even without a learning received signal. Therefore, it is necessary to estimate the echo by referring not only to the total received power spectrum at the current time, but also to the learning signals over a recent period. For this reason, in this embodiment, a term in which the logarithm of the envelope of the total received power spectrum is used as a variable is added to the estimated echo function (equation (4)).

[0116] Envelope A is the maximum value over a recent period and is calculated gradually using the time constant B and the total received power spectrum C, as shown in the following formula (7). In this embodiment, the time constant B is set to 0.5 to 1.

[0117] If(A<C): A=C Else: A=B×A+(1-B)×C ・・・(7)

[0118] δ represents the relationship between the logarithm of the envelope of the total received power spectrum and the maximum value of the logarithm of the transmitted power spectrum. δ is determined by removing outliers from a scatter plot of the logarithms of the received frequency and transmitted power spectrum. δ can be expressed as a linear or nonlinear function. In the example shown in Figure 14, δ is a linear function.

[0119] Once the estimated echo function (a function representing the estimated echo power spectrum [i]) is calculated in this manner, the estimated echo calculation unit 25 stores the four coefficients α, β, γ, and δ of this function as parameters for the threshold T3 in the storage unit 14a of the band-specific countermeasure unit 14.

[0120] Furthermore, the memory unit 14a stores an arbitrary fixed value X (corresponding to the added value of the present invention) that is added to the threshold T3 when determining the threshold T4.

[0121] Figure 15 is a flowchart showing the processing flow for double talk detection and nonlinear echo suppression according to the frequency band. First, the double talk detection unit 15 determines whether or not there is a call received based on the presence or absence of a reference signal (step S21). If there is no reference signal, i.e., there is no call received (NO in step S21), the nonlinear echo suppression unit 16 does not perform suppression processing, and the series of processes in Figure 15 is terminated.

[0122] If there is a reference signal, i.e., there is a call received (YES in step S21), the double talk detection unit 15 obtains the frequency band detected by the band detection unit 13 and thresholds T3 and T4 from the band-specific countermeasures unit 14 (step S22). Thresholds T3 and T4 can be generated using formulas (8) and (9) stored in the storage unit 14a. In calculation formulas (8) and (9), the power of the reference signal (received signal) used to detect the frequency band in step S21 is applied to the received power.

[0123] Threshold T3 calculation formula [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum ... (8) Threshold T4 calculation formula [i] = α × received power spectrum [i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum + fixed value X ... (9)

[0124] Furthermore, not only the threshold T4 for wideband, but also the thresholds for superwideband and fullband can be determined in the same way. The threshold for superwideband can be determined by adding a fixed value X1 to the threshold T3, and the threshold for fullband can be determined by adding a fixed value X2 to the threshold T3.

[0125] Next, the double talk detection unit 15 calculates a threshold T3 or threshold T4 based on the calculation formula obtained in step S22, and determines whether or not there is a transmission based on the threshold T3 or threshold T4 (step S23).

[0126] For example, if the frequency band of the reference signal is determined to be narrowband by the band-specific countermeasures unit 14 in step S22, the double talk detection unit 15 compares the threshold T3 with the suppressed signal for frequency bins included in the narrowband frequency band, and compares the threshold T4 with the suppressed signal for frequency bins outside the narrowband frequency band. Also, for example, if the frequency band of the reference signal is determined to be wideband by the band-specific countermeasures unit 14 in step S22, the double talk detection unit 15 compares the threshold T3 with the suppressed signal for frequency bins included in the wideband frequency band, and compares the threshold for superwideband with the suppressed signal for frequency bins outside the wideband frequency band.

[0127] For example, the double talk detection unit 15 compares the magnitude of the suppressed signal with the magnitude of threshold T3 or threshold T4 for each frequency bin. If the number of frequency bins in which the magnitude of the suppressed signal is equal to or greater than the magnitude of threshold T3 or threshold T4 is less than or equal to a predetermined number, it determines that there is no transmission. Otherwise, it determines that there is a transmission.

[0128] If there is no transmission (NO in step S23), i.e., it is a single-talk state with only reception, the double-talk detection unit 15 outputs to the nonlinear echo suppression unit 16 that it is a single-talk state with only reception, and the nonlinear echo suppression unit 16 suppresses the nonlinear echo (strong suppression) to reduce the magnitude of the transmission signal to an acceptable value (step S24), and the series of processes shown in Figure 15 is completed.

[0129] For example, in step S24, if the band-specific countermeasures unit 14 has determined in step S22 that the frequency band of the reference signal is narrowband, the nonlinear echo suppression unit 16 suppresses nonlinear echoes for frequency bins included in the narrowband frequency band such that the suppressed signal with a magnitude of threshold T3 becomes an acceptable value (for example, near 0), and suppresses nonlinear echoes for frequency bins outside the narrowband frequency band such that the suppressed signal with a magnitude of threshold T4 (threshold T4 > threshold T3) becomes an acceptable value.

[0130] In step S24, the suppression of nonlinear echoes is strengthened for frequency bins outside the frequency band detected by the band detection unit 13 compared to frequency bins included in the frequency band detected by the band detection unit 13.

[0131] For example, when the reference signal is a sound in the narrowband frequency range, signals outside the narrowband frequency range of the suppressed signal (e.g., high-frequency bands) may receive echo signals even though they should not be present (see the area enclosed by the dashed line in Figure 22(B)). Therefore, by suppressing the echo more strongly than the original nonlinear echo suppression, it is possible to address nonlinear echoes in frequency bands where sound should not be present.

[0132] In step S24, the nonlinear echo suppression unit 16 may not suppress nonlinear echoes for frequency bins included in the narrowband frequency range, but may strongly suppress nonlinear echoes for frequency bins outside the narrowband frequency range (for example, by reducing them by 30 dB). In this case as well, the suppression of nonlinear echoes is stronger for frequency bins outside the frequency band detected by the band detection unit 13 than for frequency bins included in the frequency band detected by the band detection unit 13.

[0133] If there is a transmission (YES in step S23), that is, if there is a double talk state with both receiving and transmitting, the double talk detection unit 15 outputs that there is a double talk state to the nonlinear echo suppression unit 16, and the nonlinear echo suppression unit 16 suppresses the nonlinear echo more weakly than in the case of single talk (weak suppression) (step S25), and the series of processes shown in Figure 15 is completed.

[0134] For example, in step S25, if the frequency band of the reference signal is determined to be narrowband by the band-specific countermeasures unit 14 in step S22, the nonlinear echo suppression unit 16 will not suppress nonlinear echoes for frequency bins included in the narrowband frequency band, and will suppress nonlinear echoes for frequency bins outside the narrowband frequency band to a degree weaker than the suppression (step S24) at which the suppressed signal of a threshold T3 magnitude becomes an acceptable value (for example, 50% of strong suppression).

[0135] In step S25, the nonlinear echo suppression unit 16 may suppress nonlinear echoes for frequency bins outside the narrowband frequency range to a degree weaker than the suppression required to reduce the suppressed signal of a threshold T4 magnitude to an acceptable level (for example, 50% of strong suppression). Alternatively, if the nonlinear echo suppression unit 16 performed strong suppression in step S24 to reduce the nonlinear echo by a certain value (e.g., 30 dB) for frequency bins outside the narrowband frequency range, it may perform weak suppression to reduce the nonlinear echo by a smaller value (e.g., 10 dB) than in step S24.

[0136] In step S25, as in step S24, the suppression of nonlinear echoes is strengthened more for frequency bins outside the frequency band detected by the band detection unit 13 than for frequency bins included in the frequency band detected by the band detection unit 13.

[0137] The process shown in Figure 15 is performed each time a sample point is acquired. After the process in Figure 15 is performed, the IFFT unit 17 performs an inverse FFT on the signal output from the nonlinear echo suppression unit 16, and the signal converted to the time domain by the inverse FFT is output to the communication device 53.

[0138] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasure unit 14 performs countermeasures according to the bandwidth (changing the degree of nonlinear echo suppression depending on whether the signal input from the communication device 53 is in the frequency bin of the frequency band). As a result, good call quality can be ensured across multiple frequency bands without tuning for each frequency band.

[0139] For example, in situations where nonlinear echoes and reflections are likely to occur, such as in an in-vehicle device where the microphone and speaker are integrated, if the band-specific countermeasures unit 14 does not take countermeasures according to the frequency band, using the adjustment results for a wideband environment in a narrowband environment may result in degraded sound quality and make it difficult to use. In contrast, in this embodiment, the band-specific countermeasures unit 14 changes the degree of nonlinear echo suppression depending on whether the signal input from the communication device 53 is in the frequency bin of the band, so that call quality can be guaranteed even when the adjustment results for a wideband environment are used in a narrowband environment.

[0140] Furthermore, according to this embodiment, in the case of single talk where only receiving is possible, the nonlinear echo suppression unit 16 suppresses nonlinear echoes more strongly for frequency bins outside the frequency band detected by the frequency band detected by the frequency band detected by the frequency band detected by the frequency band detection unit 13 than for frequency bins included in the frequency band detected by the frequency band detection unit 13, thereby reliably suppressing unwanted nonlinear echoes.

[0141] Furthermore, according to this embodiment, the double talk detection unit 15 detects the presence or absence of transmission using a threshold T3 for frequency bins included in the frequency band detected by the band detection unit 13, and detects the presence or absence of transmission using a threshold T4 (threshold T4 > threshold T3) for frequency bins outside the frequency band detected by the band detection unit 13, thereby preventing false detection of the presence or absence of transmission.

[0142] Furthermore, in this embodiment, the frequency band of the reference signal (the signal input from the communication device 53) is detected based on the magnitude of the signal outside the frequency band of the frequency band to be detected, among the signals obtained by converting the reference signal transmitted through the receiver-side signal path into a frequency domain signal in the FFT unit 18. Therefore, the frequency band can be detected using only the reference signal. In particular, by detecting the frequency band based on the sum of the power of the frequency bins outside the frequency band, accurate detection can be achieved.

[0143] In this embodiment of the present invention, the threshold for double talk detection is changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 (threshold T3 for frequency bins in the frequency band detected by the band detection unit 13, and threshold T4 for frequency bins outside the frequency band detected by the band detection unit 13 (threshold T4 > threshold T3)), and the strength of suppression is changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 in the case of single talk (for frequency bins outside the frequency band detected by the band detection unit 13, nonlinear echo suppression is stronger than for frequency bins in the frequency band detected by the band detection unit 13). However, it is not essential to change the threshold depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13. For example, double talk may be detected using threshold T3 for all frequency bins, and the strength of suppression may be changed depending on whether the frequency bin is included in the frequency band detected by the band detection unit 13 in the case of single talk (receiving only).

[0144] Furthermore, in this embodiment of the present invention, a fixed value is added to the threshold T3 to obtain the threshold T4, but the method for determining the threshold T4 is not limited to this. The following describes some variations in the method for determining the threshold T4.

[0145] <Modification 1> Modification 1 is a form in which the summation value to be added to the threshold T3 is determined using the coefficient γ1 of the total received power spectrum and the coefficient δ1 of the envelope of the total received power spectrum. The summation value is determined by the estimated echo calculation unit 25. Figure 16 is a flowchart showing the flow of a series of processes for determining the summation value. Here, narrowband and wideband are explained as examples.

[0146] The estimated echo calculation unit 25 performs calibration using a wideband sound source and determines the four coefficients α, β, γ, and δ used in the threshold T3 calculation formula using the method already described (step S31). The threshold T3 calculation formula [i] is the formula (8) already described.

[0147] Next, the estimated echo calculation unit 25 performs far-end utterance using a narrowband sound source and acquires a pair value of a reference signal (learning received signal) and an echo signal (learning signal) outside the narrowband frequency band (step S32). In other words, in step SP32, even though only narrowband sound is being played, an echo signal that is outside the narrowband frequency band and within the wideband frequency band is acquired in correspondence with the reference signal from which that signal was obtained.

[0148] Next, the estimated echo calculation unit 25 subtracts the threshold values ​​(α component and β component) obtained in step S31 from the data obtained in step S32 (step S33). However, the result of the subtraction is not made to be less than or equal to 0. As a result, the result shown in Figure 13 is obtained.

[0149] Next, the estimated echo calculation unit 25 extracts the maximum value obtained by subtracting the α and β components from the logarithm of the transmitted power for each logarithm of the transmitted / received power in the results obtained in step S33 (step S34), and determines the coefficient γ1 for wideband (step S35). The method for determining the coefficient γ1 is the same as the method for determining the coefficient γ. Alternatively, the coefficient γ1 may be determined by regression analysis with the result obtained in step S33 as the dependent variable and the logarithm of the transmitted / received power as the independent variable.

[0150] Next, the estimated echo calculation unit 25 subtracts the coefficient γ1 component obtained in step S35 from the result obtained in step S33 (step S36). In the result obtained in step S36, for each logarithm of the most recent maximum value of each transmit / receive power, the maximum value obtained by subtracting the α component, β component, and γ1 component from the logarithm of the transmit power is extracted (step S37), and the coefficient δ1 for wideband is determined (step S38). The method for determining the coefficient δ1 is the same as the method for determining the coefficient δ. This completes the process of determining the parameter set for the summation value. The estimated echo calculation unit 25 stores the coefficients α, β, γ, δ and coefficients γ1 and δ1 as parameters for the summation value in the storage unit 14a of the band-specific countermeasure unit 14.

[0151] The bandwidth-specific countermeasure unit 14 determines the threshold T4 using the coefficients γ1 and δ1 stored in the memory unit 14a and formulas (10) and (11). However, a fixed value Y is not required.

[0152] Threshold T4 = Threshold T3 + Addition value ... (10) Addition value [i] = γ1 × Total received power spectrum + δ1 × Envelope of total received power spectrum + Fixed value Y ... (11)

[0153] Thus, the sum is generated by inputting a signal generated using a sound source in a narrow first frequency band (in this case, narrowband) into the receiver's signal path, and acquiring an echo signal in a frequency band wider than the first frequency band (in this case, wideband).

[0154] The estimated echo calculation unit 25 calculates not only the summation value [i] for wideband, but also for superwideband and fullband. The method for calculating the coefficients γ1' and δ1' for superwideband and the coefficients γ1'' and δ1'' for fullband is the same as for the coefficients γ1 and δ1 for wideband.

[0155] This modified method allows for the determination of more accurate summation values ​​compared to methods that simply add fixed values. Furthermore, it can be performed with a small number of parameters: a threshold T3 and three summation values.

[0156] In this modified example, by modifying the threshold T4[i] (threshold T3[i] and sum [i]) as shown in the following formula (12), the double talk detection unit 15 can perform speech detection without conditional branching between using threshold T3 and threshold T4. α × received power spectrum[i] + β × frequency + γ × total received power spectrum + δ × envelope of total received power spectrum + (γ1 × total received power spectrum + δ1 × envelope of total received power spectrum) = α × received power spectrum[i] + β × frequency + γz × total received power spectrum + δz × envelope of total received power spectrum (where γz = γ + γ1, δz = δ + δ1) ... (12)

[0157] <Modification 2> Modification 2 is a form in which threshold values ​​for each frequency band are determined separately, rather than determining the sum to be added to the threshold value T3. The threshold value T3 is as already explained (see formula (8)). The formula for calculating the wideband threshold value T4 is as shown in formula (13). Note that the method for determining the coefficients γ1 and δ1 in formula (13) is the same as in the case of formula (11).

[0158] Threshold T4 calculation formula [i] = γ1 × total received power spectrum + δ1 × envelope of total received power spectrum + fixed value Y ... (13)

[0159] Thus, the threshold for the second frequency band (in this case, wideband), which is wider than the narrow first frequency band, is generated by inputting a signal generated using a sound source in the first frequency band (in this case, narrowband) into the receiver-side signal path, and by acquiring an echo signal in the frequency band of the second frequency band.

[0160] Furthermore, the estimated echo calculation unit 25 can similarly determine thresholds for both super-wideband and full-band applications.

[0161] According to this modified example, the method for calculating the threshold T4 is the same as for the threshold T3, but the number of coefficients (parameters) can be reduced.

[0162] <Modification 3> Modification 3 is similar to Modification 2 in that the threshold values ​​for each frequency band are determined separately. Figure 17 is a flowchart showing the sequence of processes for determining the threshold values ​​T3 and T4. Here, the frequency bands are narrowband and wideband.

[0163] The estimated echo calculation unit 25 performs calibration using a wideband sound source and determines the four coefficients α, β, γ, and δ used in the calculation formula for the threshold T3 using the method already described (step S31).

[0164] Next, the estimated echo calculation unit 25 performs far-end speech using a narrowband sound source and obtains pair values ​​of a reference signal (learning received signal) and an echo signal (learning signal) outside the narrowband frequency band (step S32).

[0165] Next, the estimated echo calculation unit 25 calculates the coefficients α2, β2, γ2, and δ2 based on the data obtained in step S32 (step S37). The method for calculating the coefficients α2, β2, γ2, and δ2 is the same as for the coefficients α, β, γ, and δ. This completes the process of calculating the parameters for the thresholds T3 and T4. The estimated echo calculation unit 25 stores the coefficients α, β, γ, and δ, and the coefficients α2, β2, γ2, and δ2, as parameters for addition in the storage unit 14a of the band-specific countermeasure unit 14.

[0166] The bandwidth-specific countermeasure unit 14 uses the coefficients α2, β2, γ2, δ2 and formula (14) stored in the memory unit 14a to determine the threshold T4.

[0167] Threshold T4 calculation formula [i] = α² × received power spectrum [i] + β² × frequency + γ² × total received power spectrum + δ² × envelope of total received power spectrum ... (14)

[0168] Thus, the threshold for the second frequency band (in this case, wideband), which is wider than the narrow first frequency band, is generated by inputting a signal generated using a sound source in the first frequency band (in this case, narrowband) into the receiver-side signal path, and by acquiring an echo signal in the frequency band of the second frequency band.

[0169] Furthermore, the estimated echo calculation unit 25 can similarly determine thresholds for both super-wideband and full-band applications.

[0170] According to this modified example, the method for calculating threshold T4 is exactly the same as that for threshold T3, making the threshold adjustment process easy.

[0171] Here, we will explain why, in modified examples 2 and 3, the threshold T4 calculated separately from the threshold T3 by the calculation formula is greater than the threshold T3. First, the coefficients α, β, γ, and δ of the threshold T3 are determined using a sound source in the second frequency band (here, wideband) (see Figure 22(A)), and then the coefficients γ1 and δ1 are determined using a sound source in the first frequency band (here, narrowband) (see Figure 22(B)). When determining the threshold T3, the coefficients α, β, γ, and δ are determined in that order, so the coefficient α is dominant. However, outside the narrowband, the logarithm of the received power [i] (corresponding to α) is 0 (see Figure 22(A)), so the threshold T3 is approximately 0 (threshold T3 ≈ 0). When determining the coefficients γ1 and δ1, learning is performed with the logarithm of the received power [i] being 0 (see Figure 22(B)), so α is not considered at all and the others are predominantly calculated. Therefore, the threshold T4 is close to the echo value (>>0) (threshold T4>>0). Therefore, the threshold T4 is greater than the threshold T3.

[0172] <Second Embodiment> The following describes the acoustic signal processing device 2 according to the second embodiment. Parts identical to those in the acoustic signal processing device 1 according to the first embodiment are denoted by the same reference numerals and their descriptions are omitted.

[0173] Figure 18 is a schematic diagram of the functional blocks of the acoustic signal processing device 2. Functionally, the acoustic signal processing device 2 mainly comprises an echo removal unit 11, frequency analyzers (FFT units) 12 and 18, a band detection unit 13, a band-specific countermeasure unit 14A, a nonlinear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. Note that the functional components of the acoustic signal processing device 2 may be further classified into more components depending on the processing content, or one component may perform processing for multiple components.

[0174] The frequency band-specific countermeasure unit 14A performs countermeasures according to the frequency band detected by the frequency band detection unit 13. In this embodiment, the frequency band-specific countermeasure unit 14A controls the volume adjustment unit 20 and the compressor 21 differently depending on whether the frequency band detected by the frequency band detection unit 13 is a narrowband signal (corresponding to the first frequency band of the present invention) or a wider frequency band signal (corresponding to the second frequency band of the present invention). The frequency band-specific countermeasure unit 14A performs processing each time a sample point is acquired.

[0175] The bandwidth detection unit 13 detects the frequency band based on the reference signal, and the bandwidth-specific countermeasure unit 14A acquires the result. If the bandwidth-specific countermeasure unit 14A acquires that it is a narrowband (for example, a narrowband), the volume adjustment unit 20 and the compressor 21 perform normal processing.

[0176] If the bandwidth-specific countermeasure unit 14A detects that the frequency band is wider than a narrowband (for example, a wideband), it controls the volume adjustment unit 20 to gradually reduce the volume to a level where distortion is acceptable. The level at which distortion is acceptable can be determined in advance and stored in the memory unit 14a. Furthermore, the level at which distortion is acceptable can be set to decrease as the frequency bandwidth of the frequency band widens, and stored in the memory unit 14a.

[0177] Furthermore, if the bandwidth-specific countermeasure unit 14A detects that the frequency band is wider than a narrowband (for example, wideband), it controls the compressor 21 to gradually lower the compressor threshold until the distortion level is acceptable. The acceptable distortion level can be determined in advance and stored in the memory unit 14a. The acceptable distortion level can also be set to decrease as the bandwidth of the frequency band widens and stored in the memory unit 14a.

[0178] Thus, when the frequency band is wider than the narrowband, the volume control unit 20 and the compressor 21 reduce the overall volume in order to lower the level of the extraneous echo signal (the area enclosed by the dashed line) in the high-frequency band in Figure 22(B).

[0179] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasure unit 14A takes countermeasures according to the frequency band (if the frequency band of the signal input from the communication device 53 is wideband, the volume and compressor threshold are lowered). As a result, good call quality can be ensured across multiple frequency bands without tuning each frequency band.

[0180] In this embodiment, the volume reduction process was performed using both the volume adjustment unit 20 and the compressor 21. However, it is also possible to perform the overall volume reduction process using at least one of the volume adjustment unit 20 and the compressor 21.

[0181] Furthermore, in this embodiment, the volume adjustment unit 20 and the compressor 21 performed a process to reduce the overall volume, but it is also possible to reduce the volume for at least some of the frequency bins using the volume adjustment unit 20 and the compressor 21.

[0182] Furthermore, the process by which the frequency band-specific countermeasure unit 14A reduces the overall volume using the volume adjustment unit 20 and the compressor 21 may be performed in combination with the processing of the frequency band-specific countermeasure unit 14.

[0183] <Third Embodiment> The acoustic signal processing device 3 according to the third embodiment will be described below. Parts identical to those in the acoustic signal processing device 1 according to the first embodiment will be denoted by the same reference numerals, and their descriptions will be omitted.

[0184] Figure 19 is a schematic diagram of the functional blocks of the acoustic signal processing device 3. Functionally, the acoustic signal processing device 3 mainly comprises an echo removal unit 11, frequency analyzers (FFT units) 12 and 18, a bandwidth detection unit 13, a bandwidth-specific countermeasure unit 14B, a nonlinear echo suppression unit (echo suppressor) 16, and a restoration unit (IFFT unit) 17. Note that the functional components of the acoustic signal processing device 3 may be further classified into many components depending on the processing content, or one component may perform processing for multiple components.

[0185] The bandwidth-specific countermeasures unit 14A performs countermeasures according to the frequency band detected by the bandwidth detection unit 13. In this embodiment, the bandwidth-specific countermeasures unit 14B controls the echo removal unit 11 differently depending on whether the frequency band detected by the bandwidth detection unit 13 is a narrowband signal (corresponding to the first frequency band of the present invention) or a wider frequency band signal (corresponding to the second frequency band of the present invention). The bandwidth-specific countermeasures unit 14A performs processing each time a sample point is acquired.

[0186] The bandwidth detection unit 13 detects the frequency band based on the reference signal, and the bandwidth-specific countermeasure unit 14B acquires the result. If the bandwidth-specific countermeasure unit 14B acquires that it is a narrowband, the echo removal unit 11 performs normal processing.

[0187] The bandwidth-specific countermeasures unit 14B controls the echo removal unit 11 to increase the step size (update width) and perform learning processing of the adaptive filter when it detects that the frequency band is wider than the narrowband (for example, wideband). The step size is set to be smallest when the bandwidth is narrowband and to increase as the bandwidth widens, and is stored in the memory unit 14a.

[0188] A linear filter H(n) is expressed by the following equation (15), where μ is the step size, X(n) is the input vector, and e(n) is the error signal. H(n) = H(n-1) + μX(n)e(n) ... (15)

[0189] For example, in a normal learning process, the step size μ is μ1 (corresponding to the first step size of the present invention, e.g., 0.05), but when the frequency band is wider than a narrowband (e.g., wideband), the step size μ is increased to μ2 (corresponding to the second step size of the present invention, e.g., 0.1, μ2 > μ1). Increasing the step size brings the echo output closer to zero, so the sound quality (e.g., conversations inside a car) deteriorates somewhat, but the distortion (out-of-band noise (see the area enclosed by the dashed line in Figure 22(B))) can be brought closer to zero. As a result, call quality is greatly improved.

[0190] According to this embodiment, the bandwidth detection unit 13 detects the frequency band of the signal input from the communication device 53, and based on the detection result, the bandwidth-specific countermeasure unit 14A takes countermeasures according to the bandwidth (if the frequency band of the signal input from the communication device 53 is wideband, the step size μ is increased). As a result, good call quality can be ensured across multiple frequency bands without tuning each frequency band.

[0191] In this embodiment, the process by which the bandwidth-specific countermeasure unit 14B increases the step size μ may be performed in combination with the processing of the bandwidth-specific countermeasure unit 14 or the bandwidth-specific countermeasure unit 14A.

[0192] <Fourth Embodiment> The acoustic signal processing device 4 according to the fourth embodiment will be described below. Parts identical to those in the acoustic signal processing device 1 according to the first to third embodiments will be denoted by the same reference numerals, and their descriptions will be omitted.

[0193] Figure 20 is a schematic diagram of the functional blocks of the acoustic signal processing device 4. Functionally, the acoustic signal processing device 4 mainly comprises an echo removal unit 11, a frequency analyzer (FFT unit) 12, a bandwidth detection unit 13A, a bandwidth-specific countermeasure unit 14C, a nonlinear echo suppression unit (echo suppressor) 16, a restoration unit (IFFT unit) 17, a volume adjustment unit 20, and a compressor 21. Note that the functional components of the acoustic signal processing device 4 may be further classified into more components depending on the processing content, or one component may perform processing for multiple components.

[0194] The bandwidth detection unit 13A is a functional unit that detects the frequency band of a signal acquired via the communication device (corresponding to the first frequency band of the present invention) based on a reference signal transmitted through the receiver-side signal path. For example, the bandwidth detection unit 13A detects whether it is a narrowband (corresponding to the first frequency band of the present invention) or a wideband (corresponding to the second frequency band of the present invention) with a frequency band wider than the narrowband. Alternatively, the bandwidth detection unit 13A may detect whether it is a narrowband wideband (corresponding to the first frequency band of the present invention) or a superwideband (corresponding to the second frequency band of the present invention) with a frequency band wider than the wideband, or whether it is a narrowband superwideband (corresponding to the first frequency band of the present invention) or a fullband (corresponding to the second frequency band of the present invention) with a frequency band wider than the superwideband.

[0195] First, the bandwidth detection unit 13A applies a bandstop filter to the reference signal to isolate only the components outside the narrowband frequency range. Next, the bandwidth detection unit 13A measures the power of the signal, which now consists only of the out-of-band components of the narrowband, and determines whether this power is above a certain threshold. If the power is below the threshold, the bandwidth detection unit 13A determines that the reference signal is a narrowband signal.

[0196] If the power is not below the threshold, the bandwidth detection unit 13A determines that the reference signal is audio in a frequency band wider than narrowband. The bandwidth detection unit 13A then applies a bandstop filter to the reference signal to isolate only the components outside the wideband frequency band and measures the power of the components outside the wideband frequency band. Next, the bandwidth detection unit 13A determines whether this power is above an arbitrary threshold. If the power is below the threshold, it determines that the reference signal is a wideband signal; if the power is not below the threshold, it determines that the reference signal is audio in a frequency band wider than wideband.

[0197] If the band detection unit 13A determines that the reference signal is audio in a frequency band wider than the wideband, it applies a bandstop filter to the reference signal to isolate only the components outside the superwideband frequency band and measures the power of the components outside the superwideband frequency band. Next, the band detection unit 13A determines whether this power is above or below an arbitrary threshold. If the power is below the threshold, it determines that the reference signal is a signal in the superwideband frequency band; if the power is below the threshold, it determines that the reference signal is audio in a frequency band wider than the superwideband, i.e., the fullband frequency band.

[0198] Note that the filter used is not limited to a bandstop filter. For example, a high-pass filter may be applied to the reference signal to measure the power, leaving only the components wider than the frequency band to be detected.

[0199] The band-specific countermeasure unit 14C performs countermeasures according to the frequency band detected by the band detection unit 13A. For example, the band-specific countermeasure unit 14C controls the echo removal unit 11 to perform learning processing of the adaptive filter with a step size corresponding to the detected frequency band.

[0200] Furthermore, for example, the frequency band-specific countermeasure unit 14C controls the volume adjustment unit 20 to gradually reduce the volume to a level where distortion is acceptable, according to the detected frequency band. Also, for example, the compressor 21 controls the compressor threshold to gradually reduce the compressor threshold to a level where distortion is acceptable, according to the detected frequency band.

[0201] According to this embodiment, the frequency band of the signal input from the communication device 53 can be detected by the band detection unit 13A without converting the reference signal into a frequency domain signal. Furthermore, since the reference signal is not converted into a frequency domain signal when detecting the frequency band, low-latency frequency band detection is possible. In addition, since the band-specific countermeasures unit 14C is performed based on the detection result, good call quality can be ensured across multiple frequency bands without tuning for each frequency band.

[0202] In this embodiment, the band-specific countermeasure unit 14C controlled the echo removal unit 11, the volume adjustment unit 20, and the compressor 21 according to the frequency band detected by the band detection unit 13A. However, the band-specific countermeasure unit 14C only needs to control at least one of the echo removal unit 11, the volume adjustment unit 20, and the compressor 21.

[0203] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments, and design modifications and the like are also included within the scope of the gist of this invention. Furthermore, although power was used as an indicator of the magnitude of the signal in each of the above embodiments, the magnitude of the signal can also be expressed using amplitude (power is the square of the amplitude), and the power in the above embodiments can be replaced with amplitude.

[0204] 1, 2, 3, 4: Acoustic signal processing unit 11: Echo removal unit 12, 18: FFT unit 13, 13A: Bandwidth detection unit 14, 14A, 14B, 14C: Bandwidth-specific countermeasures unit 14a: Memory unit 15: Double talk detection unit 16: Nonlinear echo suppression unit 17: IFFT unit 20: Volume adjustment unit 21: Compressor 25: Estimated echo calculation unit 50, 54: Terminal 51: Sound detection sensor 52: Sound generation unit 53: Communication device 90: Closed space 100: Voice communication system

Claims

1. An acoustic signal processing device that transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device for voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, comprising: a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path; and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit.

2. The acoustic signal processing device according to claim 1, comprising a frequency analyzer that converts the reference signal into a frequency domain signal to obtain a band detection signal, wherein the band detection unit detects whether it is the first frequency band or the second frequency band based on the magnitude of the signal in a predetermined range of frequency bands among the band detection signals, and the predetermined range is a range higher than the upper limit of the frequency bin of the frequency band to be detected among the first frequency band and the second frequency band.

3. The acoustic signal processing device according to claim 2, characterized in that the band detection unit detects that the receiving frequency band is the first frequency band when the sum of the magnitudes of the band detection signals is less than or equal to a first threshold in a range higher than the upper limit of the frequency bin of the receiving frequency band.

4. The acoustic signal processing device according to claim 2 or 3, comprising: a double talk detection unit that sequentially detects whether or not speech has been input to the sound detection sensor based on a double talk detection mask; and a nonlinear echo suppression unit that performs processing to suppress nonlinear echoes in a signal from which linear echoes have been removed, wherein the nonlinear echo suppression unit suppresses nonlinear echoes more strongly for frequency bins not included in the first frequency band than for frequency bins included in the first frequency band in the case of single talk where only reception occurs and the reception frequency band is the first frequency band.

5. The acoustic signal processing device according to claim 4, characterized in that, when the receiving frequency band is the first frequency band, the double talk detection unit detects whether or not speech has been input from the sound detection sensor using a first detection mask for frequency bins included in the frequency band of the first frequency band, and detects whether or not speech has been input from the sound detection sensor using a second detection mask larger than the first detection mask for frequency bins not included in the frequency band of the first frequency band.

6. The acoustic signal processing apparatus according to any one of claims 1 to 5, further comprising a volume adjustment unit for adjusting the volume of a signal transmitted through the receiver-side signal path, wherein the band-specific countermeasure unit controls the volume adjustment unit to gradually reduce the volume to a predetermined value when it is detected that the receiver frequency band is the second frequency band.

7. The acoustic signal processing apparatus according to any one of claims 1 to 6, comprising a compressor for compressing a signal transmitted through the receiving-side signal path, wherein the band-specific countermeasure unit controls the compressor to gradually lower the compressor threshold at which the compressor starts compression when it is detected that the receiving frequency band is the second frequency band.

8. The acoustic signal processing apparatus according to any one of claims 1 to 7, comprising a linear echo removal unit for suppressing linear echoes generated when sound output from the sound generation unit is input to the sound detection sensor, wherein the linear echo removal unit removes linear echoes using the results of a learning process performed with a first step size when it is detected that the receiving frequency band is the first frequency band, and removes linear echoes using the results of a learning process performed with a second step size larger than the first step size when it is detected that the receiving frequency band is the second frequency band.

9. The acoustic signal processing apparatus according to claim 5, wherein the band-specific countermeasure unit generates the first detection mask and the second detection mask using an estimated echo function calculated based on a second learning receiving signal obtained by converting a learning receiving signal transmitted through the receiving-side signal path into the frequency domain, and a second learning signal obtained by converting a learning receiving signal transmitted through the transmitting-side signal path into the frequency domain when the sound output from the sound generation unit due to the learning receiving signal is input to the sound detection sensor, the estimated echo function having as variables the logarithm of the magnitude of the reference signal at each frequency, the frequency of the reference signal, the logarithm of the total received value which is the sum of the magnitudes of the reference signal or the sum of the reference signal in an arbitrary frequency range, and the logarithm of the envelope of the total received value, wherein the first detection mask is generated by inputting a signal generated using a sound source in the second frequency band into the receiving-side signal path.

10. The acoustic signal processing device according to claim 9, wherein the second detection mask is generated by adding an added value to the first detection mask, and the added value is generated by inputting a signal generated using a sound source in the first frequency band to the receiving-side signal path and acquiring an echo signal in the frequency band of the second frequency band.

11. An acoustic signal processing method comprising: transmitting a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquiring voice input via a sound detection sensor via a transmitting-side signal path and outputting it to the communication device, the method comprising: detecting, based on a reference signal transmitted through the receiving-side signal path, whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band; and taking measures according to the detected frequency band.

12. An acoustic signal processing program that transmits a signal in a predetermined frequency band, which is the receiving frequency band, acquired via a communication device that performs voice communication, to a sound generating unit via a receiving-side signal path to produce sound from the sound generating unit, and acquires voice input via a sound detection sensor via a transmitting-side signal path and outputs it to the communication device, characterized in that the computer functions as: a band detection unit that detects whether the receiving frequency band is a narrowband first frequency band or a second frequency band with a wider frequency band than the first frequency band, based on a reference signal transmitted through the receiving-side signal path; and a band-specific countermeasure unit that performs countermeasures according to the frequency band detected by the band detection unit.

Images