Echo suppression device, echo suppression method, and echo suppression program
The echo suppression device uses a nonlinear echo model to estimate and suppress nonlinear echoes, addressing the instability of conventional methods, thereby improving echo suppression performance and communication quality in devices with speaker distortion.
Patent Information
- Application Number
- JP2020177958
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-05-20
- Filing Date
- 2020-10-23
- Publication Date
- 2024-11-27
- Estimated Expiration
- 2040-10-23
AI Technical Summary
Conventional echo suppression techniques struggle to stably suppress nonlinear echo signals, particularly in systems with speaker distortion, such as notebook computers and portable video conferencing systems, due to the use of linear models that cannot represent nonlinear echoes.
An echo suppression device that employs a nonlinear echo model, such as a neural network, to estimate and suppress nonlinear echoes by using a nonlinear echo estimation unit and a second linear echo suppressor to handle residual linear echoes, improving estimation accuracy and suppression performance.
The device effectively stabilizes the suppression of nonlinear echoes, enhancing communication quality by reducing echo interference even with speakers that produce significant distortion, contributing to higher quality and cost-effective miniaturization of devices like notebook computers and web conference systems.
Smart Images

Figure 0007590850000005 
Figure 0007590850000006 
Figure 0007590850000007
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a technique for suppressing linear and nonlinear echo signals contained in an input signal acquired by a microphone. [Background technology]
[0002] In a hands-free communication system, a video conference system, or the like, when a voice communication is made using a speaker and a microphone, the voice spoken by the speaker on the transmitting side is input to the microphone on the transmitting side and transmitted as a transmission signal to the device on the receiving side via a network line. The voice amplified from the speaker on the receiving side is picked up by the microphone on the receiving side and transmitted to the device on the transmitting side via the network line. At this time, the speaker on the transmitting side reproduces the voice that has passed the time it took to pass through the network line and the time it took to propagate through the space on the receiving side. The voice propagating between the speaker on the receiving side and the microphone in this way is called an echo, and becomes a factor that interferes with the communication. For this reason, echo suppression techniques such as an echo canceller and an echo suppressor have been proposed.
[0003] For example, in the echo suppression device shown in Patent Document 1, when a received signal is reproduced on a speaker, if the level of the received signal is high and distortion may occur in the reproduced sound, a gain that provides a greater amount of suppression for each frequency than the gain that would be used if no distortion were to occur is calculated, and a value based on the picked-up signal in the frequency domain is multiplied by the gain.
[0004] Furthermore, for example, the echo suppression device shown in Patent Document 2 determines, as a second gain coefficient, a value that approaches 0 as the gain coefficient corresponding to the m-times frequency value and the surrounding frequency values, when the power of the reproduction signal for any frequency value is greater than a predetermined threshold and the frequency value is m (m=2, 3, . . . , M) times that frequency value or a frequency value in the vicinity of the m-times frequency value, and otherwise determines the gain coefficient as the second gain coefficient. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] JP 2017-191992 A [Patent Document 2] JP 2010-103875 A Summary of the Invention [Problem to be solved by the invention]
[0006] However, with the above-mentioned conventional techniques, it is difficult to stably suppress nonlinear echo signals contained in the input signal acquired by the microphone, and further improvements are required.
[0007] The present disclosure has been made to solve the above problems, and aims to provide a technique that can stably suppress a nonlinear echo signal contained in an input signal acquired by a microphone. [Means for solving the problem]
[0008] An echo suppression device according to an embodiment of the present disclosure includes: a first linear echo suppression unit that suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal included in the input signal; a nonlinear echo estimation unit that estimates a nonlinear echo signal included in an input signal from at least one of a received signal and the input signal, the nonlinear echo signal being output to a speaker, using a nonlinear echo model indicating a relationship between the nonlinear echo signal and at least one of the received signal and the input signal; a nonlinear echo suppression unit that suppresses the nonlinear echo signal from an output signal of the first linear echo suppression unit, using the nonlinear echo signal estimated by the nonlinear echo estimation unit; and a second linear echo suppression unit that suppresses the residual linear echo signal from the output signal of the nonlinear echo suppression unit by estimating an amplitude component of a residual linear echo signal not suppressed by the first linear echo suppression unit. Effect of the Invention
[0009] According to the present disclosure, a nonlinear echo signal contained in an input signal acquired by a microphone can be stably suppressed. [Brief description of the drawings]
[0010] [Figure 1] 1A and 1B are diagrams illustrating a microphone signal, an echo canceller output signal, and an echo suppressor output signal when a nonlinear echo due to speaker distortion is not included in an input signal. [Diagram 2] 1A and 1B are diagrams illustrating a microphone signal, an echo canceller output signal, and an echo suppressor output signal when a nonlinear echo due to speaker distortion is included in an input signal. [Diagram 3] 1 is a diagram illustrating a configuration of a communication device according to a first embodiment of the present disclosure. [Figure 4] FIG. 2 is a diagram showing an example of signals output from each unit of the echo suppressing device in the first embodiment. [Diagram 5] 4 is a flowchart illustrating an operation of the echo suppressing device according to the first embodiment of the present disclosure. [Figure 6] FIG. 2 is a diagram illustrating a configuration of a learning device according to the first embodiment of the present disclosure. [Figure 7] FIG. 3 is a diagram showing an example of signals output from each unit of the learning device in the first embodiment. [Figure 8] FIG. 13 is a diagram showing the amplitude spectrum of a received signal containing 1 / 3 octave band noise. [Figure 9] 9 is a diagram showing amplitude spectra of correct values and estimated values of a nonlinear echo signal contained in an input signal obtained by a microphone when the received speech signal shown in FIG. 8 is amplified. FIG. [Figure 10] FIG. 1 is a diagram showing an amplitude spectrum of a received signal containing a female voice. [Figure 11] 11 is a diagram showing amplitude spectra of correct and estimated values of a nonlinear echo signal contained in an input signal obtained by a microphone when the received speech signal shown in FIG. 10 is amplified. FIG. [Figure 12] 4 is a diagram showing the results of frequency analysis of an output signal from a conventional echo suppressor and an output signal from an echo suppressor according to the first embodiment. FIG. [Figure 13] 1 is a diagram showing a time change in amplitude of an input signal including a male voice and a time change in echo suppression amount (ERLE) for the input signal. FIG. [Figure 14] FIG. 11 is a diagram illustrating a configuration of a communication device according to a second embodiment of the present disclosure. [Figure 15] FIG. 11 is a diagram illustrating a configuration of a communication device according to a third embodiment of the present disclosure. [Figure 16] FIG. 11 is a diagram illustrating a configuration of a communication device according to a fourth embodiment of the present disclosure. [Figure 17] FIG. 13 is a diagram illustrating a configuration of a communication device according to a fifth embodiment of the present disclosure. [Figure 18] FIG. 23 is a diagram illustrating a configuration of a communication device according to a sixth embodiment of the present disclosure. [Figure 19] FIG. 23 is a diagram illustrating a configuration of a communication device according to a seventh embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] (Findings on which this disclosure is based) Echo cancellers are a technology that removes echo by estimating an echo signal using an adaptive filter and subtracting the estimated echo signal from the signal picked up by a microphone. Echo is a superposition of direct sound amplified from a speaker and reflected sound. Therefore, the transfer characteristics between a speaker and a microphone can be expressed by an FIR (Finite Impulse Response) filter. An FIR type adaptive filter learns to approximate the transfer characteristics and generates a pseudo echo signal, which is an estimate of the echo, by convolving the filter coefficient with the received signal. As learning algorithms for adaptive filters, the Least Mean Square (LMS) method, the Normalized LMS (NLMS) method, and a method based on Independent Component Analysis (ICA) have been proposed.
[0012] On the other hand, an echo suppressor is a technology that suppresses echo by estimating the power spectrum of an echo in the frequency domain and subtracting the estimated power spectrum of the echo from the power spectrum of a signal picked up by a microphone. The echo suppressor suppresses echo, for example, by the spectrum subtraction method or the Wiener filter method. The above-mentioned echo canceller takes time to learn the adaptive filter, so there is a risk of residual echo occurring immediately after turning on the power or when the echo path fluctuates. In addition, noise or a transmission signal generated by a speaker or a microphone may cause the adaptive filter to learn erroneously, which may cause an estimation error in the pseudo echo signal and increase the residual echo. For this reason, an echo suppressor is generally used in the latter stage of an echo canceller to compensate for echo suppression.
[0013] Conventional echo cancellers and conventional echo suppressors estimate echoes using a linear model, and therefore have a problem in that it is difficult to suppress nonlinear echoes that have been imparted with nonlinear noise such as speaker distortion. In devices used in notebook computers or portable videoconferencing systems, large volumes of sound are amplified from small-diameter speakers, so the effects of nonlinear echoes caused by speaker distortion are prominent, and there is a risk of making it difficult to have a comfortable conversation.
[0014] Furthermore, in the above-mentioned Patent Document 1, it is difficult to suppress nonlinear echo signals of frequency components not contained in the received signal, such as harmonic distortion.
[0015] Furthermore, in the above-mentioned Patent Document 2, it is difficult to suppress wideband distortion components, and it is also difficult to suppress distortion components occurring at frequency values other than integer multiple frequency values.
[0016] In order to solve the above problems, an echo suppression device according to an embodiment of the present disclosure includes: a first linear echo suppression unit that suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal; a nonlinear echo estimation unit that estimates the nonlinear echo signal contained in the input signal from at least one of the received signal and the input signal using a nonlinear echo model that indicates a relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to a speaker; a nonlinear echo suppression unit that suppresses the nonlinear echo signal from an output signal of the first linear echo suppression unit using the nonlinear echo signal estimated by the nonlinear echo estimation unit; and a second linear echo suppression unit that suppresses the residual linear echo signal from the output signal of the nonlinear echo suppression unit by estimating an amplitude component of the residual linear echo signal that was not suppressed by the first linear echo suppression unit.
[0017] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal and the input signal using a nonlinear echo model that indicates the relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to the speaker, and the nonlinear echo signal is suppressed from the output signal of the first linear echo suppressor using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0018] Furthermore, the second linear echo suppressor suppresses the residual linear echo signal from the output signal in which the nonlinear echo signal has been suppressed. This makes it possible to stabilize the operation of the second linear echo suppressor and improve the suppression performance of the linear echo signal.
[0019] Furthermore, in the above-described echo suppression device, the nonlinear echo model may be trained using at least one of the received signal and the input signal, and an output signal of the second linear echo suppression unit that suppresses the residual linear echo signal from an output signal of the first linear echo suppression unit that suppresses a linear echo signal from the input signal, as training data, with at least one of the received signal and the input signal as an input, and with the nonlinear echo signal as an output.
[0020] According to this configuration, the first linear echo suppression unit and the second linear echo suppression unit suppress only the linear echo signal and do not suppress the nonlinear echo signal, so that the signal in which the linear echo signal is suppressed by the first linear echo suppression unit and the second linear echo suppression unit can be used as the teacher data as a nonlinear echo signal.
[0021] Furthermore, since the nonlinear echo signal is learned using at least one of the received signal and the input signal, and the output signal of the second linear echo suppression unit as teacher data, the complex distortion caused by the speaker can be accurately modeled, and the estimation accuracy of the nonlinear echo signal can be improved.
[0022] In the above echo suppression device, the nonlinear echo model may be a neural network.
[0023] According to this configuration, a nonlinear echo model can be realized by a neural network.
[0024] In addition, in the above echo suppression device, the nonlinear echo estimation unit may estimate the nonlinear echo signal contained in the input signal from the received signal by using the nonlinear echo model indicating a relationship between the received signal and the nonlinear echo signal.
[0025] According to this configuration, the nonlinear echo signal is estimated from the received signal using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal, so that the nonlinear echo signal can be easily estimated from the received signal.
[0026] In addition, in the above-mentioned echo suppression device, the nonlinear echo estimation unit may estimate the nonlinear echo signal contained in the input signal from the received signal and the input signal using the nonlinear echo model indicating the relationship between the received signal and the input signal and the nonlinear echo signal.
[0027] According to this configuration, since the nonlinear echo signal is estimated not only from the received signal but also from the received signal and the input signal, the estimation accuracy of the nonlinear echo signal can be improved.
[0028] In addition, in the above echo suppression device, the nonlinear echo estimation unit may estimate the nonlinear echo signal contained in the input signal from the received signal and the output signal of the first linear echo suppression unit, using the nonlinear echo model indicating a relationship between the received signal, the output signal of the first linear echo suppression unit, and the nonlinear echo signal.
[0029] According to this configuration, since the nonlinear echo signal is estimated not only from the received signal but also from the received signal and the output signal of the first linear echo suppressor, it is possible to improve the estimation accuracy of the nonlinear echo signal.
[0030] Also, in the above echo suppression device, the first linear echo suppression unit may include an adaptive filter that generates a pseudo-linear echo signal indicating a component of the received signal included in the input signal by convolving a filter coefficient and the received signal, and a subtraction unit that subtracts the pseudo-linear echo signal from the input signal, and the nonlinear echo estimation unit may estimate the nonlinear echo signal included in the input signal from the received signal and the pseudo-linear echo signal from the adaptive filter using the nonlinear echo model that indicates a relationship between the pseudo-linear echo signal from the received signal and the adaptive filter, and the nonlinear echo signal.
[0031] According to this configuration, the nonlinear echo signal is estimated not only from the received signal but also from the received signal and the pseudo-linear echo signal from the adaptive filter of the first linear echo suppressor, so that the estimation accuracy of the nonlinear echo signal can be improved.
[0032] In addition, in the above echo suppression device, the nonlinear echo estimation unit may estimate the nonlinear echo signal contained in the input signal from the input signal using the nonlinear echo model indicating a relationship between the input signal and the nonlinear echo signal.
[0033] According to this configuration, the nonlinear echo signal is estimated from the input signal using a nonlinear echo model that indicates the relationship between the input signal and the nonlinear echo signal, so that the nonlinear echo signal can be easily estimated from the input signal.
[0034] In addition, the above-mentioned echo suppression device may further include a correction unit that calculates a variable gain for minimizing either the output signal of the nonlinear echo suppression unit or the output signal of the second linear echo suppression unit, and corrects the nonlinear echo signal estimated by the nonlinear echo estimation unit using the calculated variable gain.
[0035] According to this configuration, it is possible to correct the estimation error of the nonlinear echo signal, and it is possible to improve the performance of suppressing the nonlinear echo signal.
[0036] An echo suppression device according to another aspect of the present disclosure includes a first linear echo suppression unit that suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal, a nonlinear echo estimation unit that estimates the nonlinear echo signal contained in the input signal from at least one of a received signal output to a speaker and the input signal, a nonlinear echo suppression unit that suppresses the nonlinear echo signal from the input signal using the nonlinear echo signal estimated by the nonlinear echo estimation unit, and a second linear echo suppression unit that suppresses the residual linear echo signal by estimating an amplitude component of the residual linear echo signal that was not suppressed by the first linear echo suppression unit.
[0037] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal output to the speaker and the input signal, and the nonlinear echo signal is suppressed from the input signal using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0038] Furthermore, the residual linear echo signal is suppressed by the second linear echo suppressor, so that the operation of the second linear echo suppressor can be stabilized and the suppression performance of the linear echo signal can be improved.
[0039] In an echo suppression method according to another aspect of the present disclosure, a first linear echo suppression unit suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal, a nonlinear echo estimation unit estimates the nonlinear echo signal contained in the input signal from at least one of a received signal and the input signal using a nonlinear echo model indicating a relationship between the nonlinear echo signal and at least one of a received signal and the input signal output to a speaker, a nonlinear echo suppression unit suppresses the nonlinear echo signal from an output signal of the first linear echo suppression unit using the nonlinear echo signal estimated by the nonlinear echo estimation unit, and a second linear echo suppression unit suppresses the residual linear echo signal from the output signal of the nonlinear echo suppression unit by estimating an amplitude component of the residual linear echo signal that was not suppressed by the first linear echo suppression unit.
[0040] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal and the input signal using a nonlinear echo model that indicates the relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to the speaker, and the nonlinear echo signal is suppressed from the output signal of the first linear echo suppressor using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0041] Furthermore, the second linear echo suppressor suppresses the residual linear echo signal from the output signal in which the nonlinear echo signal has been suppressed. This makes it possible to stabilize the operation of the second linear echo suppressor and improve the suppression performance of the linear echo signal.
[0042] In an echo suppression method according to another aspect of the present disclosure, a first linear echo suppression unit suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal, a nonlinear echo estimation unit estimates the nonlinear echo signal contained in the input signal from at least one of a received signal output to a speaker and the input signal, a nonlinear echo suppression unit suppresses the nonlinear echo signal from the input signal using the nonlinear echo signal estimated by the nonlinear echo estimation unit, and a second linear echo suppression unit suppresses the residual linear echo signal by estimating an amplitude component of the residual linear echo signal that was not suppressed by the first linear echo suppression unit.
[0043] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal output to the speaker and the input signal, and the nonlinear echo signal is suppressed from the input signal using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0044] Furthermore, the residual linear echo signal is suppressed by the second linear echo suppressor, so that the operation of the second linear echo suppressor can be stabilized and the suppression performance of the linear echo signal can be improved.
[0045] An echo suppression program according to another aspect of the present disclosure causes a computer to function as a first linear echo suppression unit that suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal; a nonlinear echo estimation unit that estimates a nonlinear echo signal contained in an input signal from at least one of a received signal and the input signal, the nonlinear echo signal being output to a speaker, using a nonlinear echo model indicating a relationship between the nonlinear echo signal and at least one of the received signal and the input signal; a nonlinear echo suppression unit that suppresses the nonlinear echo signal from an output signal of the first linear echo suppression unit, using the nonlinear echo signal estimated by the nonlinear echo estimation unit; and a second linear echo suppression unit that suppresses the residual linear echo signal from the output signal of the nonlinear echo suppression unit by estimating an amplitude component of a residual linear echo signal not suppressed by the first linear echo suppression unit.
[0046] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal and the input signal using a nonlinear echo model that indicates the relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to the speaker, and the nonlinear echo signal is suppressed from the output signal of the first linear echo suppressor using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0047] Furthermore, the second linear echo suppressor suppresses the residual linear echo signal from the output signal in which the nonlinear echo signal has been suppressed. This makes it possible to stabilize the operation of the second linear echo suppressor and improve the suppression performance of the linear echo signal.
[0048] An echo suppression program according to another aspect of the present disclosure causes a computer to function as a first linear echo suppression unit that suppresses a linear echo signal from an input signal acquired by a microphone by estimating an amplitude component and a phase component of a linear echo signal contained in the input signal, a nonlinear echo estimation unit that estimates the nonlinear echo signal contained in the input signal from at least one of a received signal output to a speaker and the input signal, a nonlinear echo suppression unit that suppresses the nonlinear echo signal from the input signal using the nonlinear echo signal estimated by the nonlinear echo estimation unit, and a second linear echo suppression unit that suppresses the residual linear echo signal by estimating an amplitude component of the residual linear echo signal that was not suppressed by the first linear echo suppression unit.
[0049] According to this configuration, a nonlinear echo signal included in the input signal is estimated from at least one of the received signal output to the speaker and the input signal, and the nonlinear echo signal is suppressed from the input signal using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone can be stably suppressed.
[0050] Furthermore, the residual linear echo signal is suppressed by the second linear echo suppressor, so that the operation of the second linear echo suppressor can be stabilized and the suppression performance of the linear echo signal can be improved.
[0051] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the following embodiments are examples of the present disclosure and do not limit the technical scope of the present disclosure.
[0052] (Embodiment 1) First, the cause of the nonlinear echo will be described.
[0053] Nonlinear distortion is a general term for distortion that occurs when the input and output of a system are not proportional. For example, when a two-tone sine wave with frequencies f1 and f2 is input to a system with input / output characteristics in which the output amplitude clips as the input amplitude increases, nonlinear distortion occurs in frequency components that do not exist in the input signal in the amplitude spectrum of the output waveform. Nonlinear distortion can be broadly divided into harmonic distortion that occurs at integer multiples of the input signal, such as 2f1 and 2f2, and intermodulation distortion that occurs at the sum and difference frequencies of the input signals, such as f1+f2 and f2-f1.
[0054] In an actual system, nonlinear distortion of the loudspeaker's amplified sound is the cause of nonlinear echo. In electrodynamic speakers, which are widely used, the displacement of the diaphragm increases in the frequency band near the lowest resonance frequency f0. Nonlinear distortion occurs due to the nonlinearity of the driving force caused by the voice coil moving beyond the range of the magnetic flux created by the permanent magnet, or the mechanical nonlinearity of the support system such as the cone edge or damper. Furthermore, in small-diameter speakers, the sound pressure near the lowest resonance frequency f0 may be boosted by pre-processing to compensate for the decrease in the sound pressure level in the low range. In this case, the displacement of the diaphragm increases, which causes further nonlinear distortion.
[0055] Next, the effect of nonlinear echoes on conventional echo suppression techniques will be described, with reference to a system equipped with an echo canceller and an echo suppressor.
[0056] The echo canceller uses an adaptive filter to calculate an estimated echo, i.e., a pseudo echo signal, and then subtracts the calculated pseudo echo signal from the microphone signal to remove the echo. In other words, the received signal is x(k), and the coefficients of the adaptive filter are w n (k) and the number of taps of the adaptive filter is N, the pseudo echo y(k) is expressed by the following equation (1).
[0057]
number
[0058] The above formula (1) means that the pseudo echo is expressed by a linear sum in which the phase and amplitude of the received signal are changed, and it cannot express a nonlinear echo regardless of the adaptive algorithm used for coefficient learning.
[0059] The echo suppressor is provided after the echo canceller. The echo suppressor suppresses the residual echo by estimating the power spectrum of the residual echo that was not suppressed by the echo canceller. In the echo suppressor based on the Wiener filter method, which is widely used, the short-time spectrum X(ω) of the received signal and the short-time spectrum Y of the residual echo are EC Amount of acoustic coupling between (ω) and E (ω) is estimated, and the Wiener filter G wiener (ω) is calculated.
[0060]
number
[0061] And the echo suppressor is the Wiener filter G wiener (ω) is expressed as the short-time spectrum Y of the residual echo as shown in the following equation (3). EC By multiplying (ω), the echo-suppressed signal Y ES (ω) is obtained.
[0062] Y ES (ω)=G wiener (ω)Y EC (ω) (3) In other words, the echo suppressor uses the acoustic coupling amount A estimated for each frequency component. E The residual echo is estimated based on the received signal X(ω) and the received signal X(ω). Therefore, the echo suppressor cannot estimate frequency components that do not exist in the received signal, such as nonlinear echoes.
[0063] To support the above, the inventors conducted an experiment to confirm the evaluation of the influence of nonlinear echo. A conventional echo suppression device was used in the evaluation experiment. The conventional echo suppression device includes a speaker that amplifies a received signal, a microphone, an echo canceller that suppresses an echo signal from an input signal acquired by the microphone, and an echo suppressor that suppresses an echo signal from an output signal from the echo canceller. In addition, 1 / 3 octave band noise with a center frequency of 400 Hz, which is near the lowest resonance frequency f0 of the speaker used for amplifying the sound, was used in the evaluation experiment.
[0064] FIG. 1 is a diagram showing a microphone signal, an echo canceller output signal, and an echo suppressor output signal when the input signal does not contain a nonlinear echo due to speaker distortion, and FIG. 2 is a diagram showing a microphone signal, an echo canceller output signal, and an echo suppressor output signal when the input signal contains a nonlinear echo due to speaker distortion.
[0065] 1 and 2, the solid line represents the microphone signal (input signal) output from the microphone, the dashed line represents the echo canceller output signal, and the dashed line represents the echo suppressor output signal. In Fig. 1 and Fig. 2, the horizontal axis represents frequency, and the vertical axis represents amplitude level.
[0066] In Fig. 2, the second to fourth harmonics of the input signal appear, and as mentioned above, it is shown that the conventional echo canceller and echo suppressor cannot suppress the nonlinear echo at all. Furthermore, when focusing on the fundamental tone around 400 Hz in Fig. 1 and Fig. 2, the echo canceller suppresses the echo by about 35 dB when the nonlinear echo is not included, whereas the suppression amount of the echo canceller deteriorates to about 20 dB when the nonlinear echo is included. This is thought to be because the adaptive filter continues to forcibly update the filter coefficients in an attempt to simulate the nonlinear echo that should not be able to be expressed, which causes erroneous learning and results in an error in the echo estimation.
[0067] The essential problem with conventional echo suppression technology is that it is not possible to express a nonlinear echo because the echo is estimated using a linear model. Therefore, the echo suppression device in the first embodiment estimates the nonlinear echo using a neural network that can approximate any nonlinear function. There are two possible methods for introducing a neural network: a method for estimating the amplitude and phase of the nonlinear echo and applying it to an echo canceller, and a method for estimating only the amplitude of the nonlinear echo and applying it to an echo suppressor. The former requires higher estimation accuracy than the latter, and has the problem of an increased amount of calculation. Therefore, the echo suppression device in the first embodiment realizes the suppression of the nonlinear echo using an echo suppressor method that can be implemented with low power consumption, low cost, and a small amount of calculation.
[0068] Fig. 3 is a diagram showing a configuration of a communication device according to the first embodiment of the present disclosure. Fig. 4 is a diagram showing an example of signals output from each unit of the echo suppression device according to the first embodiment. The communication device is used in a loudspeaker type hands-free communication system, a loudspeaker type two-way communication conference system, an intercom system, etc.
[0069] The communication device shown in FIG. 3 includes an echo suppressing device 1, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22.
[0070] The input terminal 11 outputs to the echo suppressing device 1 a received signal x(k) received from a receiving communication device (not shown).
[0071] The speaker 12 outputs the input reception signal x(k) to the outside. Here, when the voice output from the speaker 12 is picked up by the microphone 13, the voice uttered by the speaker on the reception side is reproduced from the speaker on the reception side with a delay, and so-called acoustic echo occurs. Therefore, the echo suppression device 1 reproduces the input signal x(k) output from the microphone 13 with a delay. mic (k) suppresses the acoustic echo signals, where the acoustic echo signals include linear echo signals and nonlinear echo signals.
[0072] The microphone 13 is disposed in a space where the speaker is present, and picks up the speaker's voice. The microphone 13 outputs an input signal x mic (k) is output to the echo suppressor 1.
[0073] The output terminal 22 receives the input signal y 1 in which the linear echo signal and the nonlinear echo signal have been suppressed by the echo suppression device 1. ES Output (k).
[0074] The input terminal 11 and the output terminal 22 are connected to a communication unit (not shown). The communication unit transmits an input signal y to a call device (not shown) on the receiving side via a network. ES The terminal 10 transmits a signal x(k) to a call device (not shown) on the receiving side and receives a received signal x(k) from the call device (not shown) on the receiving side via a network, for example the Internet.
[0075] The echo suppression device 1 includes an echo canceller 14, fast Fourier transform units 15 and 16, a nonlinear echo model storage unit 17, a nonlinear echo estimation unit 18, a nonlinear echo suppression unit 19, an echo suppressor 20, and an inverse fast Fourier transform unit 21.
[0076] An input terminal 11 outputs a received signal x(k) to a speaker 12 , an echo canceller 14 , and a fast Fourier transform unit 15 .
[0077] The echo canceller 14 receives the input signal x mic The amplitude and phase components of the linear echo signal contained in (k) are estimated to obtain the input signal x mic The echo canceller 14 suppresses a linear echo signal from the input signal x(k) output from the microphone 13. The echo canceller 14 is an example of a first linear echo suppressor. As shown in FIG. mic Only the linear echo signal contained in (k) is suppressed.
[0078] The echo canceller 14 includes an adaptive filter and a subtraction unit, neither of which is shown.
[0079] The adaptive filter convolves the filter coefficients with the received signal to obtain the input signal x mic A pseudo echo signal indicating the component of the received signal contained in (k) is generated.
[0080] The subtractor receives the input signal x from the microphone 13. mic The adaptive filter calculates an error signal between (k) and a pseudo echo signal from an adaptive filter, and outputs the calculated error signal to the adaptive filter. The adaptive filter modifies the filter coefficient based on the input error signal, and generates a pseudo echo signal by convolving the modified filter coefficient with the received signal. The adaptive filter modifies the filter coefficient using an adaptive algorithm so that the error signal is minimized. As the adaptive algorithm, for example, a learning identification method (NLMS (Normarized Least Mean Square) method), an affine projection method, or a recursive least squares method (RLS (Recursive Least Square) method) is used.
[0081] The subtractor also subtracts the input signal x from the microphone 13. mic By subtracting the pseudo echo signal from the adaptive filter from (k), the input signal x mic (k) is used to suppress the linear echo signal. Then, the subtraction unit outputs the input signal y EC (k) is output to the fast Fourier transform unit 15.
[0082] The fast Fourier transform unit 15 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 15 converts the time domain input signal y EC (k) is the frequency domain input signal Y EC The fast Fourier transform unit 15 converts the input signal Y in the frequency domain, in which only the linear echo signal has been suppressed by the echo canceller 14, into EC (ω) is output to the nonlinear echo suppressor 19.
[0083] The fast Fourier transform unit 16 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 16 converts a time domain received signal x(k) input to the nonlinear echo estimation unit 18 into a frequency domain received signal X(ω). The fast Fourier transform unit 16 outputs the frequency domain received signal X(ω) to the nonlinear echo estimation unit 18 and the echo suppressor 20.
[0084] The nonlinear echo model storage unit 17 stores in advance a nonlinear echo model indicating a relationship between at least one of the received signal output to the speaker 12 and the input signal acquired by the microphone 13 and the nonlinear echo signal. The nonlinear echo model storage unit 17 in the first embodiment stores in advance a nonlinear echo model indicating a relationship between the received signal and the nonlinear echo signal. The nonlinear echo model is, for example, a neural network.
[0085] The nonlinear echo model uses at least one of the received signal and the input signal, and the output signal of the echo suppressor that suppresses a linear echo signal from the output signal of the echo canceller that suppresses a linear echo signal from the input signal as training data, and is trained with at least one of the received signal and the input signal as the nonlinear echo signal. The nonlinear echo model in the first embodiment uses at least one of the received signal and the output signal of the echo suppressor that suppresses a linear echo signal from the output signal of the echo canceller that suppresses a linear echo signal from the input signal as training data, and is trained with the received signal as the input and the nonlinear echo signal as the nonlinear echo signal.
[0086] The nonlinear echo estimation unit 18 estimates the received signal X(ω) output to the speaker 12 and the input signal x mic (k) from at least one of the input signals Y EC The nonlinear echo signal X contained in (ω) NN More specifically, the nonlinear echo estimation unit 18 estimates the received signal X(ω) output to the speaker 12 and the input signal x mic Using a nonlinear echo model showing the relationship between at least one of (k) and the nonlinear echo signal, the received signal X(ω) and the input signal x mic(k) from at least one of the input signals Y EC The nonlinear echo signal X contained in (ω) NN The nonlinear echo estimation unit 18 in the first embodiment estimates the nonlinear echo signal X(ω) contained in the input signal from the received signal X(ω) using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. NN Estimate (ω).
[0087] The nonlinear echo estimation unit 18 reads out the nonlinear echo model from the nonlinear echo model storage unit 17. The nonlinear echo estimation unit 18 inputs the received signal X(ω) output from the fast Fourier transform unit 16 to the nonlinear echo model, thereby deriving a nonlinear echo signal X(ω) from the nonlinear echo model. NN The nonlinear echo estimation unit 18 obtains a nonlinear echo signal X(ω) estimated using the received signal X(ω). NN (ω) is output to the nonlinear echo suppressor 19.
[0088] The nonlinear echo suppressor 19 suppresses the nonlinear echo signal X estimated by the nonlinear echo estimator 18. NN (ω) is used to calculate the input signal Y EC (ω) to the nonlinear echo signal X NN More specifically, the nonlinear echo suppressor 19 suppresses the nonlinear echo signal X (ω) estimated by the nonlinear echo estimator 18. NN (ω) is used to obtain a nonlinear echo signal X from the output signal of the echo canceller 14. NN Suppress (ω).
[0089] The nonlinear echo suppressor 19 estimates the nonlinear echo signal X based on the following equation (4): NN (ω) and the input signal Y from the echo canceller 14 EC (ω) and the Wiener filter G NN Calculate (ω).
[0090]
number
[0091] The nonlinear echo suppressor 19 is a Wiener filter G NN (ω) is expressed as the input signal Y EC The input signal Y with the nonlinear echo signal suppressed by multiplying it by (ω) NL-ES (ω) is obtained.
[0092] Y NL-ES (ω)=G NN (ω)Y EC (ω) (5)
[0093] The nonlinear echo suppressor 19 suppresses the nonlinear echo signal X NN Input signal Y with only (ω) suppressed NL-ES (ω) is output to the echo suppressor 20.
[0094] The echo suppressor 20 suppresses the residual linear echo signal by estimating the amplitude component of the residual linear echo signal that has not been suppressed by the echo canceller 14. More specifically, the echo suppressor 20 estimates the amplitude component of the residual linear echo signal that has not been suppressed by the echo canceller 14, thereby obtaining the output signal Y NL-ES The echo suppressor 20 suppresses the residual linear echo signal from (ω) and is an example of a second linear echo suppressor.
[0095] The echo suppressor 20 suppresses the residual linear echo signal by the spectral subtraction method or the Wiener filter method. The echo suppressor 20 estimates the acoustic coupling amount for each frequency by using the spatial or coherence function of only the echo signal. The echo suppressor 20 calculates the estimated acoustic coupling amount and the output signal Y NL-ES The echo suppressor 20 multiplies the output signal of the nonlinear echo suppressor 19 by the calculated suppression gain, thereby suppressing the residual linear echo signal that was not suppressed by the echo canceller 14. The echo suppressor 20 multiplies the output signal of the nonlinear echo suppressor 19 by the calculated suppression gain. NL-ES The input signal Y with only the residual linear echo signal suppressed from (ω) ES(ω) is output to the inverse fast Fourier transform unit 21.
[0096] The inverse fast Fourier transform unit 21 performs an inverse discrete Fourier transform at high speed. The inverse fast Fourier transform unit 21 converts the frequency domain input signal Y ES (ω) is the time domain input signal y ES (k). The inverse fast Fourier transform unit 21 converts the input signal y ES (k) is output to output terminal 22.
[0097] Next, the operation of the echo suppressing device 1 according to the first embodiment of the present disclosure will be described.
[0098] FIG. 5 is a flowchart illustrating the operation of the echo suppressing device according to the first embodiment of the present disclosure.
[0099] First, in step S1, the echo canceller 14 receives an input signal x mic The amplitude and phase components of the linear echo signal contained in (k) are estimated to obtain the input signal x mic The linear echo signal is suppressed from (k).
[0100] Next, in step S2, the nonlinear echo estimation unit 18 derives the nonlinear echo signal X(ω) contained in the input signal from the received signal X(ω) using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. NN Estimate (ω).
[0101] Next, in step S3, the nonlinear echo suppressor 19 outputs the input signal Y EC (ω), the nonlinear echo signal X estimated by the nonlinear echo estimation unit 18 is NN Suppress (ω).
[0102] Next, in step S4, the echo suppressor 20 estimates the amplitude component of the residual linear echo signal that was not suppressed by the echo canceller 14, thereby obtaining the input signal Y NL-ES (ω). The echo suppressor 20 suppresses the residual linear echo signal from the input signal Y NL-ES The input signal Y with only the residual linear echo signal suppressed from (ω) ES (ω) to the inverse fast Fourier transform unit 21. The inverse fast Fourier transform unit 21 converts the time domain input signal y ES (k) is output to output terminal 22.
[0103] As described above, the nonlinear echo signal included in the input signal is estimated from at least one of the received signal and the input signal using a nonlinear echo model that indicates the relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to the speaker 12, and the nonlinear echo signal is suppressed from the output signal of the echo canceller 14 using the estimated nonlinear echo signal. Therefore, the nonlinear echo signal included in the input signal acquired by the microphone 13 can be stably suppressed.
[0104] Furthermore, the residual linear echo signal is suppressed from the output signal in which the nonlinear echo signal has been suppressed by the echo suppressor 20. Therefore, the operation of the echo suppressor 20 can be stabilized, and the performance of suppressing the linear echo signal can be improved.
[0105] Next, a method for learning the nonlinear echo model in the first embodiment will be described.
[0106] Fig. 6 is a diagram showing a configuration of the learning device according to the first embodiment of the present disclosure, Fig. 7 is a diagram showing an example of signals output from each unit of the learning device according to the first embodiment.
[0107] The learning device shown in FIG. 6 includes a nonlinear echo model creating device 2, an input terminal 31, a speaker 32, a microphone 33, and an output terminal 39.
[0108] The input terminal 31 outputs to the echo suppressing device 1 a received signal x(k) received from a receiving communication device (not shown).
[0109] The speaker 32 outputs the received speech signal x(k) to the outside.
[0110] The microphone 33 is disposed in a space where the speaker is present, and picks up the speaker's voice. The microphone 33 outputs an input signal x mic (k) is output to the nonlinear echo model creation device 2.
[0111] The output terminal 39 is the input signal y , in which the linear echo signal has been suppressed by the nonlinear echo model creation device 2. ES Output (k).
[0112] The configurations of the input terminal 31, the speaker 32, the microphone 33, and the output terminal 39 are the same as the configurations of the input terminal 11, the speaker 12, the microphone 13, and the output terminal 22 in FIG.
[0113] The nonlinear echo model creating device 2 includes an echo canceller 34 , fast Fourier transform units 35 and 36 , an echo suppressor 37 , an inverse fast Fourier transform unit 38 , a nonlinear echo model learning unit 40 , and a nonlinear echo model storage unit 41 .
[0114] The echo canceller 34 receives the input signal x mic The amplitude and phase components of the linear echo signal contained in (k) are estimated to obtain the input signal x mic The echo canceller 34 suppresses the linear echo signal from the input signal y(k). The configuration of the echo canceller 34 is the same as the configuration of the echo canceller 14 shown in FIG. 3. The echo canceller 34 suppresses the linear echo signal from the input signal y(k). EC (k) is output to the fast Fourier transform unit 35.
[0115] The fast Fourier transform unit 35 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 35 converts the time domain input signal y EC (k) is the frequency domain input signal Y EC The fast Fourier transform unit 35 converts the input signal Y in the frequency domain, in which only the linear echo signal has been suppressed by the echo canceller 34, into EC (ω) is output to the echo suppressor 37.
[0116] The fast Fourier transform unit 36 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 36 converts the time domain received signal x(k) input to the echo suppressor 37 into a frequency domain received signal X(ω). The fast Fourier transform unit 36 outputs the frequency domain received signal X(ω) to the echo suppressor 37 and the nonlinear echo model learning unit 40.
[0117] The echo suppressor 37 estimates the amplitude component of the residual linear echo signal that was not suppressed by the echo canceller 34, thereby obtaining the input signal Y EC (ω). The echo suppressor 37 suppresses the residual linear echo signal from the input signal Y EC The input signal Y with only the residual linear echo signal suppressed from (ω) ES (ω) is output to the inverse fast Fourier transform unit 21 and the nonlinear echo model learning unit 40.
[0118] The inverse fast Fourier transform unit 38 performs an inverse discrete Fourier transform at high speed. The inverse fast Fourier transform unit 38 converts the frequency domain input signal Y ES (ω) is the time domain input signal y ES (k). The inverse fast Fourier transform unit 38 converts the input signal y ES (k) is output to output terminal 39.
[0119] The nonlinear echo model learning unit 40 learns the received signal X(ω) and the input signal x mic At least one of (k) and an input signal x micThe output signal Y of the echo canceller 34 that suppresses the linear echo signal from (k) EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) as training data, and the input is the received signal X(ω) and the input signal x mic The nonlinear echo model training unit 40 in the first embodiment trains a nonlinear echo model that outputs a nonlinear echo signal by using at least one of the received signal X(ω) and the input signal x mic The output signal Y of the echo canceller 34 that suppresses the linear echo signal from (k) EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) are used as training data, and a nonlinear echo model is trained in which the received signal X(ω) is used as input and the nonlinear echo signal is used as output.
[0120] The nonlinear echo model is a signal amplitude spectrum X(ω) of the received signal and a residual echo amplitude spectrum Y(ω) of the echo canceller 34 and the echo suppressor 37. ES (ω) as teacher data. The echo canceller 34 and the echo suppressor 37 can suppress only linear echo signals. Therefore, the output signals (residual echo signals) of the echo canceller 34 and the echo suppressor 37 are almost equal to the nonlinear echo signal. In this way, the nonlinear echo model learning unit 40 can model the relationship between the amplitude spectrum of the received signal and the amplitude spectrum of the nonlinear echo signal.
[0121] Machine learning includes, for example, supervised learning, which learns the relationship between input and output using training data in which labels (output information) are assigned to the input information, unsupervised learning, which builds a data structure only from unlabeled input, semi-supervised learning, which handles both labeled and unlabeled data, and reinforcement learning, which learns actions that maximize rewards by trial and error. Specific machine learning techniques include neural networks (including deep learning using multi-layered neural networks), as well as genetic programming, decision trees, Bayesian networks, and support vector machines (SVMs). In machine learning of the nonlinear echo model, any of the specific examples listed above may be used.
[0122] The nonlinear echo model learning unit 40 stores the learned nonlinear echo model in the nonlinear echo model storage unit 41 .
[0123] The nonlinear echo model storage unit 41 stores the nonlinear echo model learned by the nonlinear echo model learning unit 40 .
[0124] 3 may include a nonlinear echo model learning unit 40. In this case, the echo suppressing device 1 may further include a mode switching unit that switches between a learning mode and an echo suppression mode. When the mode switching unit switches to the learning mode, the echo canceller 14 outputs an output signal to the echo suppressor 20. The nonlinear echo model learning unit 40 receives the input signal Y , in which the linear echo signal has been suppressed by the echo canceller 14 and the echo suppressor 20, and outputs the input signal Y . ES A nonlinear echo model may be trained using the received signal X(ω) and the received signal X(ω) as training data.
[0125] Furthermore, the nonlinear echo model learned by the learning device may be stored in advance in the nonlinear echo model storage unit 17 of the echo suppressing device 1. Furthermore, the echo suppressing device 1 may receive the nonlinear echo model learned by the learning device and update the nonlinear echo model stored in the nonlinear echo model storage unit 17.
[0126] Next, a simulation result comparing the echo suppression amount of the echo suppressor 1 in the first embodiment with the echo suppressor amount of a conventional echo suppressor will be described.
[0127] First, the neural network (nonlinear echo model) used in the simulation had the amplitude spectrum of the short-time Fourier transform as the input / output feature.
[0128] Fig. 8 to Fig. 11 are diagrams showing examples of estimating the amplitude spectrum of a nonlinear echo signal using a neural network. Fig. 8 is a diagram showing the amplitude spectrum of a received signal including 1 / 3 octave band noise, and Fig. 9 is a diagram showing the amplitude spectrum of a correct value and an estimated value of a nonlinear echo signal included in an input signal obtained by a microphone when the received signal shown in Fig. 8 is amplified. Fig. 10 is a diagram showing the amplitude spectrum of a received signal including a female voice, and Fig. 11 is a diagram showing the amplitude spectrum of a correct value and an estimated value of a nonlinear echo signal included in an input signal obtained by a microphone when the received signal shown in Fig. 10 is amplified.
[0129] 8 to 11, the horizontal axis represents frequency and the vertical axis represents amplitude level. In Fig. 9 and Fig. 11, the solid lines represent correct values of the nonlinear echo signal, and the dashed lines represent estimated values of the nonlinear echo signal.
[0130] As shown in Figs. 9 and 11, it is understood that the neural network can accurately estimate the nonlinear echo signal indicated by the solid line.
[0131] Next, a simulation result of the echo suppression device 1 of the first embodiment using a trained neural network and a conventional echo suppression device will be described. Note that the conventional echo suppression device includes only an echo canceller and an echo suppressor, and suppresses only linear echo signals by the echo canceller and the echo suppressor.
[0132] Fig. 12 is a diagram showing the results of frequency analysis of an output signal from a conventional echo suppression device and an output signal from the echo suppression device of the first embodiment. In Fig. 12, the horizontal axis represents frequency, and the vertical axis represents amplitude level. In Fig. 12, the solid line represents the input signal from the microphone 13, the dashed line represents the output signal from the conventional echo suppression device, and the dashed-dotted line represents the output signal from the echo suppression device 1 of the first embodiment. The received signal is 1 / 3 octave band noise with a center frequency of 315 Hz.
[0133] As shown in Fig. 12, the echo suppression device 1 of the first embodiment achieves a suppression effect of 15 dB to 20 dB, which exceeds the target value, for harmonic distortion, which is a nonlinear echo signal. Furthermore, the echo suppression device 1 of the first embodiment achieves a suppression effect that is about 15 dB higher for a linear echo signal of 315 Hz, as compared with the conventional echo suppression device. This is believed to be because the nonlinear echo signal is suppressed by the nonlinear echo suppressor 19 of the first embodiment, and thus the estimation of the acoustic coupling amount in the subsequent echo suppressor 20 becomes stable.
[0134] Next, an evaluation result of the echo suppression amount by the echo suppressor 1 of the first embodiment and the conventional echo suppressor for an input signal having a complex frequency structure such as a human voice will be described. Note that ERLE (Echo Return Loss Enhancement), which means the echo suppression amount, was used as the evaluation index. ERLE is calculated by the following formula (6).
[0135]
number
[0136] Fig. 13 is a diagram showing a change over time in the amplitude of an input signal containing a male voice, and a change over time in the echo suppression amount (ERLE) for the input signal. In the upper part of Fig. 13, the horizontal axis represents time, and the vertical axis represents amplitude. In the lower part of Fig. 13, the horizontal axis represents time, and the vertical axis represents the echo suppression amount. In the lower part of Fig. 13, the solid line represents the echo suppression amount by the echo suppression device 1 of the first embodiment, and the dashed line represents the echo suppression amount by a conventional echo suppression device.
[0137] The echo suppressor 1 of the first embodiment has a suppression effect about 10 dB higher than that of the conventional echo suppressor. Therefore, it has been shown that the echo suppressor 1 of the first embodiment is sufficiently effective for an input signal having a complex frequency structure such as a human voice.
[0138] In this way, the echo suppressing device 1 of the first embodiment enables comfortable conversation even with speakers that produce a lot of distortion, and can contribute to higher quality, smaller size, and lower cost of notebook computers, web conference systems, mobile phones, and the like.
[0139] (Embodiment 2) The nonlinear echo estimating unit 18 in the above-mentioned embodiment 1 estimates the nonlinear echo signal included in the input signal from the received signal using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. In contrast, the nonlinear echo estimating unit in the embodiment 2 estimates the nonlinear echo signal included in the input signal from the received signal and the input signal using a nonlinear echo model that indicates the relationship between the received signal, the input signal, and the nonlinear echo signal.
[0140] FIG. 14 is a diagram illustrating a configuration of a communication device according to the second embodiment of the present disclosure.
[0141] 14 includes an echo suppressing device 1A, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the second embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.
[0142] The echo suppression device 1A includes an echo canceller 14, fast Fourier transform units 15, 16, and 23, a nonlinear echo model storage unit 171, a nonlinear echo estimation unit 181, a nonlinear echo suppression unit 19, an echo suppressor 20, and an inverse fast Fourier transform unit .
[0143] Microphone 13 receives the input signal x mic The signal (k) is output to the echo canceller 14 and also to the nonlinear echo estimation unit 181 via the fast Fourier transform unit 23 .
[0144] The fast Fourier transform unit 23 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 23 converts the time domain input signal x mic (k) is the frequency domain input signal X mic The fast Fourier transform unit 23 converts the input signal X in the frequency domain into mic (ω) is output to the nonlinear echo estimation unit 181.
[0145] The nonlinear echo model storage unit 171 pre-stores a nonlinear echo model that indicates the relationship between the received signal output to the speaker 12 and the input signal acquired by the microphone 13, and the nonlinear echo signal. The nonlinear echo model is, for example, a neural network.
[0146] The nonlinear echo model in the second embodiment is trained using a received signal, an input signal, and an output signal of an echo suppressor that suppresses a residual linear echo signal from an output signal of an echo canceller that suppresses a linear echo signal from the input signal as training data, with the input being the received signal and the input signal, and the output being a nonlinear echo signal.
[0147] In the method for learning a nonlinear echo model in the second embodiment, a received signal X(ω) and a frequency domain input signal X are input to a nonlinear echo model learning unit 40 shown in FIG. mic The nonlinear echo model learning unit 40 in the second embodiment receives the received signal X(ω) and the input signal X mic (ω) and the input signal xmic The output signal Y of the echo canceller 34 that suppresses the linear echo signal from (k) EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) are used as training data, and the input is the received signal X(ω) and the input signal X mic (ω) and a nonlinear echo model whose output is a nonlinear echo signal is trained.
[0148] The nonlinear echo estimation unit 181 estimates the received signal X(ω) and the input signal X(ω) by using a nonlinear echo model that indicates the relationship between the received signal, the input signal, and the nonlinear echo signal. mic (ω) to the nonlinear echo signal X contained in the input signal NN Estimate (ω).
[0149] The nonlinear echo estimation unit 181 reads out a nonlinear echo model from the nonlinear echo model storage unit 171. The nonlinear echo estimation unit 181 receives the received signal X(ω) output from the fast Fourier transform unit 16 and the input signal X(ω) output from the fast Fourier transform unit 23. mic By inputting (ω) to the nonlinear echo model, the nonlinear echo signal X NN The nonlinear echo estimation unit 181 obtains the received signal X(ω) and the input signal X mic The nonlinear echo signal X estimated using (ω) NN (ω) is output to the nonlinear echo suppressor 19.
[0150] The operation of the echo suppressing device 1A in the second embodiment differs only in the process of step S2 shown in Fig. 5. That is, in the second embodiment, the nonlinear echo estimating unit 181 estimates the received signal X(ω) and the input signal X(ω) by using a nonlinear echo model indicating the relationship between the received signal, the input signal, and the nonlinear echo signal. mic (ω) to the nonlinear echo signal X NN Estimate (ω).
[0151] In the second embodiment, since the nonlinear echo signal is estimated from the received signal and the input signal, the estimation accuracy of the nonlinear echo signal can be further improved.
[0152] (Embodiment 3) The nonlinear echo estimating unit 18 in the above-mentioned embodiment 1 estimates the nonlinear echo signal included in the input signal from the received signal using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. In contrast, the nonlinear echo estimating unit in the embodiment 3 estimates the nonlinear echo signal included in the input signal from the received signal and the output signal of the echo canceller 14 using a nonlinear echo model that indicates the relationship between the received signal, the output signal of the echo canceller, and the nonlinear echo signal.
[0153] FIG. 15 is a diagram illustrating a configuration of a communication device according to the third embodiment of the present disclosure.
[0154] 15 includes an echo suppressing device 1B, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the third embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.
[0155] The echo suppression device 1B includes an echo canceller 14, fast Fourier transform units 15 and 16, a nonlinear echo model storage unit 172, a nonlinear echo estimation unit 182, a nonlinear echo suppression unit 19, an echo suppressor 20, and an inverse fast Fourier transform unit .
[0156] The fast Fourier transform unit 15 converts the input signal Y in the frequency domain, in which only the linear echo signal has been suppressed by the echo canceller 14, into EC (ω) is output to the nonlinear echo suppressor 19 and the nonlinear echo estimator 182 .
[0157] The nonlinear echo model storage unit 172 stores in advance a nonlinear echo model that indicates the relationship between the received signal and the output signal of the echo canceller that are output to the speaker 12, and the nonlinear echo signal. The nonlinear echo model is, for example, a neural network.
[0158] The nonlinear echo model in the third embodiment uses the received signal, the output signal of the echo canceller, and the output signal of the echo suppressor that suppresses the residual linear echo signal from the output signal of the echo canceller that suppresses the linear echo signal from the input signal as training data, and is trained with the received signal and the output signal of the echo canceller as inputs and the nonlinear echo signal as output.
[0159] In the method for learning the nonlinear echo model in the third embodiment, the received signal X(ω) and the frequency domain output signal Y of the echo canceller 34 are input to the nonlinear echo model learning unit 40 shown in FIG. EC The nonlinear echo model learning unit 40 in the third embodiment receives the received signal X(ω) and the frequency domain output signal Y EC (ω) and the input signal x mic (k) is the frequency domain output signal Y of the echo canceller 34 that suppresses the linear echo signal. EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) as training data, and the input is the received signal X(ω) and the frequency domain output signal Y EC (ω) and a nonlinear echo model whose output is a nonlinear echo signal is trained.
[0160] The nonlinear echo estimation unit 182 estimates the received signal X(ω) and the frequency domain output signal Y of the echo canceller 14 using a nonlinear echo model that indicates the relationship between the received signal, the output signal of the echo canceller, and the nonlinear echo signal. EC (ω) to the nonlinear echo signal X contained in the input signal NN Estimate (ω).
[0161] The nonlinear echo estimation unit 182 reads out the nonlinear echo model from the nonlinear echo model storage unit 172. The nonlinear echo estimation unit 182 estimates the received signal X(ω) output from the fast Fourier transform unit 16 and the input signal Y ECBy inputting (ω) to the nonlinear echo model, the nonlinear echo signal X NN The nonlinear echo estimation unit 182 obtains the received signal X(ω) and the input signal Y EC The nonlinear echo signal X estimated using (ω) NN (ω) is output to the nonlinear echo suppressor 19.
[0162] 5. In other words, in the third embodiment, the nonlinear echo estimation unit 182 estimates the received signal X(ω) and the frequency domain output signal Y of the echo canceller 14 by using a nonlinear echo model that indicates the relationship between the received signal, the output signal of the echo canceller 14, and the nonlinear echo signal. EC (ω) to the nonlinear echo signal X NN Estimate (ω).
[0163] In the third embodiment, since the nonlinear echo signal is estimated from the received signal and the output signal of the echo canceller, the estimation accuracy of the nonlinear echo signal can be further improved.
[0164] (Embodiment 4) The nonlinear echo estimating unit 18 in the above-mentioned embodiment 1 estimates the nonlinear echo signal included in the input signal from the received signal using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. In contrast, the nonlinear echo estimating unit in the embodiment 4 estimates the nonlinear echo signal included in the input signal from the received signal and the pseudo-linear echo signal from the adaptive filter of the echo canceller using a nonlinear echo model that indicates the relationship between the received signal, the pseudo-linear echo signal from the adaptive filter of the echo canceller, and the nonlinear echo signal.
[0165] FIG. 16 is a diagram illustrating a configuration of a communication device according to the fourth embodiment of the present disclosure.
[0166] 16 includes an echo suppressing device 1C, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the fourth embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.
[0167] The echo suppression device 1C includes an echo canceller 14, fast Fourier transform units 15, 16, and 24, a nonlinear echo model storage unit 173, a nonlinear echo estimation unit 183, a nonlinear echo suppression unit 19, an echo suppressor 20, and an inverse fast Fourier transform unit .
[0168] The echo canceller 14 includes an adaptive filter 141 and a subtraction unit 142. The adaptive filter 141 generates a pseudo-linear echo signal indicating the component of the received signal contained in the input signal by convolving a filter coefficient and the received signal. The subtraction unit 142 subtracts the pseudo-linear echo signal from the input signal.
[0169] The fast Fourier transform unit 24 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 24 converts the time-domain pseudo-linear echo signal input to the nonlinear echo estimation unit 183 into a frequency-domain pseudo-linear echo signal. The fast Fourier transform unit 24 outputs the frequency-domain pseudo-linear echo signal to the nonlinear echo estimation unit 183.
[0170] The nonlinear echo model storage unit 173 stores in advance a nonlinear echo model that indicates the relationship between the received signal output to the speaker 12, the pseudo-linear echo signal from the adaptive filter of the echo canceller, and the nonlinear echo signal. The nonlinear echo model is, for example, a neural network.
[0171] The nonlinear echo model in the fourth embodiment uses the received signal, the pseudo-linear echo signal from the adaptive filter of the echo canceller, and the output signal of the echo suppressor that suppresses the residual linear echo signal from the output signal of the echo canceller that suppresses the linear echo signal from the input signal as training data, and is trained with the received signal and the pseudo-linear echo signal as inputs and the nonlinear echo signal as output.
[0172] In the learning method of the nonlinear echo model in the fourth embodiment, the received signal X(ω) and the pseudo-linear echo signal from the adaptive filter of the echo canceller 34 are input to the nonlinear echo model learning unit 40 shown in Fig. 6. The nonlinear echo model learning unit 40 in the fourth embodiment receives the received signal X(ω), the pseudo-linear echo signal from the adaptive filter of the echo canceller 34, and the input signal x mic The output signal Y of the echo canceller 34 that suppresses the linear echo signal from (k) EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) are used as training data, and a nonlinear echo model is trained in which the received signal X(ω) and a pseudo-linear echo signal are used as inputs and the nonlinear echo signal is used as output.
[0173] The nonlinear echo estimation unit 183 derives the nonlinear echo signal X(ω) contained in the input signal from the received signal X(ω) and the pseudo-linear echo signal from the adaptive filter 141 using a nonlinear echo model that indicates the relationship between the received signal and the pseudo-linear echo signal from the adaptive filter, and the nonlinear echo signal. NN Estimate (ω).
[0174] The nonlinear echo estimation unit 183 reads out the nonlinear echo model from the nonlinear echo model storage unit 173. The nonlinear echo estimation unit 183 inputs the received signal X(ω) output from the fast Fourier transform unit 16 and the pseudo-linear echo signal output from the fast Fourier transform unit 24 to the nonlinear echo model, and derives a nonlinear echo signal X(ω) from the nonlinear echo model. NN The nonlinear echo estimation unit 183 obtains a nonlinear echo signal X(ω) estimated using the received signal X(ω) and the pseudo-linear echo signal. NN (ω) is output to the nonlinear echo suppressor 19.
[0175] The operation of the echo suppressing device 1C in the fourth embodiment differs only in the process of step S2 shown in Fig. 5. That is, in the fourth embodiment, the nonlinear echo estimating unit 183 derives a nonlinear echo signal X(ω) from the received signal X(ω) and the pseudo-linear echo signal from the adaptive filter 141 of the echo canceller 14 by using a nonlinear echo model indicating the relationship between the received signal and the pseudo-linear echo signal from the adaptive filter of the echo canceller, and the nonlinear echo signal. NN Estimate (ω).
[0176] In the fourth embodiment, since the nonlinear echo signal is estimated from the received signal and the pseudo-linear echo signal from the adaptive filter 141 of the echo canceller 14, the estimation accuracy of the nonlinear echo signal can be further improved.
[0177] (Embodiment 5) The nonlinear echo estimating unit 18 in the above-mentioned embodiment 1 estimates the nonlinear echo signal included in the input signal from the received signal using a nonlinear echo model that indicates the relationship between the received signal and the nonlinear echo signal. In contrast, the nonlinear echo estimating unit in the embodiment 5 estimates the nonlinear echo signal included in the input signal from the input signal using a nonlinear echo model that indicates the relationship between the input signal and the nonlinear echo signal.
[0178] FIG. 17 is a diagram illustrating a configuration of a communication device according to a fifth embodiment of the present disclosure.
[0179] 17 includes an echo suppressing device 1D, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the fifth embodiment, the same components as those in the first and second embodiments are given the same reference numerals, and the description thereof will be omitted.
[0180] The echo suppression device 1D includes an echo canceller 14, fast Fourier transform units 15, 16, and 23, a nonlinear echo model storage unit 174, a nonlinear echo estimation unit 184, a nonlinear echo suppression unit 19, an echo suppressor 20, and an inverse fast Fourier transform unit .
[0181] Microphone 13 receives the input signal x mic The signal (k) is output to the echo canceller 14 and also to the nonlinear echo estimation unit 184 via the fast Fourier transform unit 23 .
[0182] The fast Fourier transform unit 23 performs a discrete Fourier transform at high speed. The fast Fourier transform unit 23 converts the time domain input signal x mic (k) is the frequency domain input signal X mic The fast Fourier transform unit 23 converts the input signal X in the frequency domain into mic (ω) is output to the nonlinear echo estimation unit 184.
[0183] The nonlinear echo model storage unit 174 stores in advance a nonlinear echo model that indicates the relationship between the input signal acquired by the microphone 13 and the nonlinear echo signal. The nonlinear echo model is, for example, a neural network.
[0184] The nonlinear echo model in the fifth embodiment is trained using an input signal acquired by a microphone and an output signal of an echo suppressor that suppresses a residual linear echo signal from an output signal of an echo canceller that suppresses a linear echo signal from the input signal as training data, with the input being the input signal and the output being the nonlinear echo signal.
[0185] In the method for learning a nonlinear echo model in the fifth embodiment, a frequency domain input signal X mic The nonlinear echo model learning unit 40 in the fifth embodiment receives the input signal X mic (ω) and the input signal x mic The output signal Y of the echo canceller 34 that suppresses the linear echo signal from (k) EC The output signal Y of the echo suppressor 37 which suppresses the residual linear echo signal from (ω) ES (ω) and are used as training data, and the input is the input signal X mic (ω) and a nonlinear echo model whose output is a nonlinear echo signal is trained.
[0186] The nonlinear echo estimation unit 184 estimates the input signal X by using a nonlinear echo model that indicates the relationship between the input signal and the nonlinear echo signal. mic (ω) to the nonlinear echo signal X contained in the input signal NN Estimate (ω).
[0187] The nonlinear echo estimation unit 184 reads out the nonlinear echo model from the nonlinear echo model storage unit 174. The nonlinear echo estimation unit 184 estimates the input signal X mic By inputting (ω) to the nonlinear echo model, the nonlinear echo signal X NN (ω). The nonlinear echo estimation unit 184 obtains the input signal X mic The nonlinear echo signal X estimated using (ω) NN (ω) is output to the nonlinear echo suppressor 19.
[0188] The operation of the echo suppressing device 1D in the fifth embodiment differs only in the process of step S2 shown in Fig. 5. That is, in the fifth embodiment, the nonlinear echo estimating unit 184 estimates the input signal X by using a nonlinear echo model that indicates the relationship between the input signal and the nonlinear echo signal. mic (ω) to the nonlinear echo signal X NN Estimate (ω).
[0189] In the fifth embodiment, a nonlinear echo signal can be estimated only from the input signal acquired by the microphone 13.
[0190] (Embodiment 6) In the above-mentioned first embodiment, the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is output to the nonlinear echo suppression unit 19. In contrast, in the sixth embodiment, the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is corrected using the output signal of the nonlinear echo suppression unit 19.
[0191] FIG. 18 is a diagram illustrating a configuration of a communication device according to the sixth embodiment of the present disclosure.
[0192] 18 includes an echo suppressing device 1E, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the sixth embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.
[0193] The echo suppression device 1E includes an echo canceller 14, fast Fourier transform units 15 and 16, a nonlinear echo model storage unit 17, a nonlinear echo estimation unit 18, a nonlinear echo suppression unit 19, an echo suppressor 20, an inverse fast Fourier transform unit 21, and a correction unit 25.
[0194] The correction unit 25 calculates a variable gain for minimizing the output signal of the nonlinear echo suppression unit 19, and corrects the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 using the calculated variable gain. At this time, the correction unit 25 calculates the variable gain so that the output signal of the nonlinear echo suppression unit 19 approaches 0. Then, the correction unit 25 multiplies the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 by the calculated variable gain. In this way, the correction unit 25 corrects the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18.
[0195] In the operation of the echo suppressing device 1E in the sixth embodiment, a new process is added between step S2 and step S3 shown in Fig. 5. That is, in the sixth embodiment, after the process of step S2, the correcting unit 25 calculates a variable gain for minimizing the output signal of the nonlinear echo suppressing unit 19, and corrects the nonlinear echo signal estimated by the nonlinear echo estimating unit 18 using the calculated variable gain.
[0196] In the sixth embodiment, the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is corrected using the output signal of the nonlinear echo suppression unit 19, so that the estimation accuracy of the nonlinear echo signal can be improved, and the echo suppression performance can be improved. In particular, the sixth embodiment is effective when the nonlinear echo model is a fixed value.
[0197] The echo suppressing devices 1A to 1D in the above-mentioned second to fifth embodiments may include the correction unit 25 in the sixth embodiment.
[0198] (Embodiment 7) In the above-mentioned first embodiment, the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is output to the nonlinear echo suppression unit 19. In contrast, in the seventh embodiment, the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is corrected using the output signal of the echo suppressor 20.
[0199] FIG. 19 is a diagram illustrating a configuration of a communication device according to a seventh embodiment of the present disclosure.
[0200] 19 includes an echo suppressing device 1F, an input terminal 11, a speaker 12, a microphone 13, and an output terminal 22. In the seventh embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted.
[0201] The echo suppression device 1F includes an echo canceller 14, fast Fourier transform units 15 and 16, a nonlinear echo model storage unit 17, a nonlinear echo estimation unit 18, a nonlinear echo suppression unit 19, an echo suppressor 20, an inverse fast Fourier transform unit 21 and a correction unit 251.
[0202] The correction unit 251 calculates a variable gain for minimizing the output signal of the echo suppressor 20, and corrects the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 using the calculated variable gain. At this time, the correction unit 25 calculates the variable gain so that the output signal of the echo suppressor 20 approaches zero. Then, the correction unit 251 multiplies the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 by the calculated variable gain. In this way, the correction unit 251 corrects the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18.
[0203] In the operation of the echo suppressing device 1F in the seventh embodiment, a new process is added between step S2 and step S3 shown in Fig. 5. That is, in the seventh embodiment, after the process of step S2, the correction unit 251 calculates a variable gain for minimizing the output signal of the echo suppressor 20, and corrects the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 using the calculated variable gain.
[0204] In the seventh embodiment, the estimation error of the nonlinear echo signal estimated by the nonlinear echo estimation unit 18 is corrected using the output signal of the echo suppressor 20, so that the estimation accuracy of the nonlinear echo signal can be improved, and the echo suppression performance can be improved. In particular, the seventh embodiment is effective when the nonlinear echo model is a fixed value.
[0205] The echo suppressing devices 1A to 1D in the above-mentioned second to fifth embodiments may include the correction unit 251 in the seventh embodiment.
[0206] In each of the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.
[0207] A part or all of the functions of the device according to the embodiment of the present disclosure are typically realized as an LSI (Large Scale Integration), which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip that includes some or all of the functions. The integrated circuit is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells inside the LSI, may be used.
[0208] Furthermore, some or all of the functions of the device according to the embodiment of the present disclosure may be realized by a processor such as a CPU executing a program.
[0209] Furthermore, all the numbers used above are merely examples for the purpose of specifically explaining the present disclosure, and the present disclosure is not limited to the numbers exemplified.
[0210] In addition, the order in which each step is performed shown in the above flowchart is merely an example for specifically explaining the present disclosure, and an order other than the above may be used as long as the same effect is obtained. In addition, some of the steps may be performed simultaneously (in parallel) with other steps. [Industrial Applicability]
[0211] The technology disclosed herein can stably suppress nonlinear echo signals contained in an input signal acquired by a microphone, and is therefore useful as a technology for suppressing linear echo signals and nonlinear echo signals contained in an input signal acquired by a microphone. [Explanation of symbols]
[0212] 1,1A,1B,1C,1D,1E,1F Echo suppression device 2. Nonlinear echo model creation device 11,31 Input terminal 12,32 Speaker 13,33 Microphone 14,34 Echo Canceller 15,16,23,24,35,36 Fast Fourier transform section 17,171,172,173,174 Nonlinear echo model memory 18,181,182,183,184 Nonlinear echo estimator 19 Nonlinear echo suppression section 20,37 Echo Suppressor 21,38 Inverse Fast Fourier Transform 22,39 Output terminal 25,251 Correction section 40 Nonlinear echo model learning section 41 Nonlinear echo model memory section 141 Adaptive Filter 142 Subtraction section
Claims
1. a first linear echo suppressor configured to estimate an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing the linear echo signal from the input signal; a nonlinear echo estimating unit that estimates the nonlinear echo signal included in the input signal from at least one of the received signal and the input signal using a nonlinear echo model that indicates a relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to a speaker; a nonlinear echo suppressor that suppresses the nonlinear echo signal from the output signal of the first linear echo suppressor by using the nonlinear echo signal estimated by the nonlinear echo estimation unit; a second linear echo suppressor that estimates an amplitude component of a residual linear echo signal that has not been suppressed by the first linear echo suppressor, and suppresses the residual linear echo signal from the output signal of the nonlinear echo suppressor; An echo suppression device comprising:
2. The nonlinear echo model is trained by using at least one of the received signal and the input signal, and an output signal of the second linear echo suppressor that suppresses the residual linear echo signal from an output signal of the first linear echo suppressor that suppresses a linear echo signal from the input signal as teacher data, with an input being at least one of the received signal and the input signal, and an output being the nonlinear echo signal.
2. The echo suppression device according to claim 1.
3. The nonlinear echo model is a neural network.
3. An echo suppression device according to claim 1 or 2.
4. the nonlinear echo estimation unit estimates the nonlinear echo signal included in the input signal from the received signal by using the nonlinear echo model indicating a relationship between the received signal and the nonlinear echo signal; 4. The echo suppression device according to claim 1.
5. the nonlinear echo estimation unit estimates the nonlinear echo signal included in the input signal from the received signal and the input signal by using the nonlinear echo model indicating a relationship between the received signal, the input signal, and the nonlinear echo signal; 4. The echo suppression device according to claim 1.
6. the nonlinear echo estimation unit estimates the nonlinear echo signal included in the input signal from the received signal and the output signal of the first linear echo suppression unit using the nonlinear echo model indicating a relationship between the received signal, the output signal of the first linear echo suppression unit, and the nonlinear echo signal; 4. The echo suppression device according to claim 1.
7. the first linear echo suppression unit includes an adaptive filter that generates a pseudo-linear echo signal indicating a component of the received signal included in the input signal by convolving a filter coefficient and the received signal, and a subtraction unit that subtracts the pseudo-linear echo signal from the input signal; the nonlinear echo estimation unit estimates the nonlinear echo signal included in the input signal from the received signal and the pseudo-linear echo signal from the adaptive filter using the nonlinear echo model indicating a relationship between the received signal and the pseudo-linear echo signal from the adaptive filter and the nonlinear echo signal; 4. The echo suppression device according to claim 1.
8. the nonlinear echo estimation unit estimates the nonlinear echo signal contained in the input signal from the input signal by using the nonlinear echo model indicating a relationship between the input signal and the nonlinear echo signal; 4. The echo suppression device according to claim 1.
9. a correction unit that calculates a variable gain for minimizing either the output signal of the nonlinear echo suppression unit or the output signal of the second linear echo suppression unit, and corrects the nonlinear echo signal estimated by the nonlinear echo estimation unit using the calculated variable gain.
9. An echo suppression device according to claim 1.
10. a first linear echo suppressor configured to estimate an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing the linear echo signal from the input signal; a nonlinear echo estimator that estimates a nonlinear echo signal included in the input signal from at least one of a received signal output to a speaker and the input signal; a nonlinear echo suppressor that suppresses the nonlinear echo signal from the input signal by using the nonlinear echo signal estimated by the nonlinear echo estimator; a second linear echo suppressor that estimates an amplitude component of a residual linear echo signal that has not been suppressed by the first linear echo suppressor, and suppresses the residual linear echo signal from the output signal of the nonlinear echo suppressor; An echo suppression device comprising:
11. a first linear echo suppressor estimating an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing a linear echo signal from the input signal; a nonlinear echo estimation unit estimating the nonlinear echo signal included in the input signal from at least one of the received signal and the input signal using a nonlinear echo model indicating a relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to a speaker; a nonlinear echo suppressor suppresses the nonlinear echo signal from the output signal of the first linear echo suppressor by using the nonlinear echo signal estimated by the nonlinear echo estimation unit; a second linear echo suppressor estimating an amplitude component of the residual linear echo signal that has not been suppressed by the first linear echo suppressor, thereby suppressing the residual linear echo signal from the output signal of the nonlinear echo suppressor; Echo suppression method.
12. a first linear echo suppressor estimating an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing a linear echo signal from the input signal; a nonlinear echo estimating unit estimating a nonlinear echo signal included in the input signal from at least one of a received signal output to a speaker and the input signal; a nonlinear echo suppressor suppresses the nonlinear echo signal from the input signal by using the nonlinear echo signal estimated by the nonlinear echo estimator; a second linear echo suppressor estimating an amplitude component of the residual linear echo signal that has not been suppressed by the first linear echo suppressor, thereby suppressing the residual linear echo signal from the output signal of the nonlinear echo suppressor; Echo suppression method.
13. a first linear echo suppressor configured to estimate an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing the linear echo signal from the input signal; a nonlinear echo estimating unit that estimates the nonlinear echo signal included in the input signal from at least one of the received signal and the input signal using a nonlinear echo model that indicates a relationship between the nonlinear echo signal and at least one of the received signal and the input signal output to a speaker; a nonlinear echo suppressor that suppresses the nonlinear echo signal from the output signal of the first linear echo suppressor by using the nonlinear echo signal estimated by the nonlinear echo estimation unit; by estimating an amplitude component of a residual linear echo signal that has not been suppressed by the first linear echo suppressor, the computer is caused to function as a second linear echo suppressor that suppresses the residual linear echo signal from the output signal of the nonlinear echo suppressor. Echo suppression program.
14. a first linear echo suppressor configured to estimate an amplitude component and a phase component of a linear echo signal included in an input signal acquired by a microphone, thereby suppressing the linear echo signal from the input signal; a nonlinear echo estimator that estimates a nonlinear echo signal included in the input signal from at least one of a received signal output to a speaker and the input signal; a nonlinear echo suppressor that suppresses the nonlinear echo signal from the input signal by using the nonlinear echo signal estimated by the nonlinear echo estimator; by estimating an amplitude component of a residual linear echo signal that has not been suppressed by the first linear echo suppressor, the computer is caused to function as a second linear echo suppressor that suppresses the residual linear echo signal from the output signal of the nonlinear echo suppressor. Echo suppression program.
Citation Information
Patent Citations
Echo suppression apparatus, echo suppression method, echo suppression program, and recording medium
JP2010103875A
Adaptive equalizer, echo canceler and active noise control apparatus
JP2011205533A
Echo cancellation device, echo cancellation program and echo cancellation method
JP2016076860A
Echo suppressor, method therefor, program, and recording medium
JP2017191992A
Audio processing device, audio processing method, and recording medium on which audio processing program is recorded
WO2012157788A1
