Sound pickup devices, sound pickup methods, and computer program products

Abstract

The pickup device (1) includes: an adaptive filter (141) that generates a speculative noise signal from a reference signal representing the noise signal component contained in the input signal acquired by the microphone (11); a noise removal signal generation unit (15) that generates a noise removal signal after subtracting the speculative noise signal from the input signal; a filter coefficient update unit (142) that updates the filter coefficients of the adaptive filter (141) using the noise removal signal; and a sample position determination unit (162) that determines at least one signal sample position from the signal sample position with the largest absolute value of the noise removal signal up to the predetermined largest signal sample position, wherein the filter coefficient update unit (142) updates the filter coefficients at the at least one signal sample position determined by the sample position determination unit (162).

Description

Technical Field

[0001] The present invention relates to a pickup device, pickup method, and pickup program for removing noise signals contained in an input signal acquired through a microphone. Background Technology

[0002] Conventional two-way communication systems using microphones and speakers have existed. In such systems, the speaker's voice is input to the microphone on the speaking side, transmitted as a speech signal through a communication line to the receiving side's device, and reproduced by the receiving side's speaker. The sound reproduced by the receiving side's speaker propagates through the space on the receiving side and is input to the receiving side's microphone, then transmitted to the speaking side. At this point, the speaker on the speaking side reproduces its own voice after the time taken to travel through the communication line and the time taken to propagate through the space on the receiving side. As mentioned above, the sound propagating between the speaker and microphone on the receiving side is called acoustic echo, which degrades call quality.

[0003] For example, the noise cancellation device shown in Patent Document 1 generates a first radio signal corresponding to the sound and noise input to the sound input terminal, and generates a second radio signal corresponding to the noise input to the reference input terminal. It uses an adaptive filter to generate a pseudo-noise signal from the second radio signal, subtracts the pseudo-noise signal from the first radio signal to generate a noise suppression signal, updates the filter coefficients of the adaptive filter using the noise suppression signal, derives the peak tap position representing the peak of the filter coefficient based on the relative position relationship between the noise source and the reference input terminal and the sound input terminal, and classifies the intervals of the tap positions corresponding to the filter coefficients using the peak tap positions. For each classified interval, it controls the update frequency of the filter coefficients corresponding to the interval.

[0004] However, in the aforementioned prior art, further improvements are needed to reduce the amount of computation required for noise removal.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Publication No. 5205935 Summary of the Invention

[0008] This invention was made to solve the above-mentioned problems, and its purpose is to provide a technique that can reduce the amount of computation required for noise removal.

[0009] One aspect of the present invention relates to a sound pickup device comprising: an adaptive filter that generates a speculative noise signal representing a component of a noise signal contained in an input signal acquired by a microphone from a reference signal; a signal generation unit that generates a noise-removed signal obtained by subtracting the speculative noise signal from the input signal; a coefficient update unit that updates the filter coefficients of the adaptive filter using the noise-removed signal; and a determination unit that determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, wherein the coefficient update unit updates the filter coefficients at the at least one signal sample position determined by the determination unit.

[0010] According to the present invention, the amount of computation required for noise removal can be reduced. Attached Figure Description

[0011] Figure 1 This is a diagram showing the configuration of the communication device in Embodiment 1 of the present invention.

[0012] Figure 2 This is a flowchart illustrating the operation of the pickup device in Embodiment 1 of the present invention.

[0013] Figure 3 This is a diagram showing the configuration of the communication device in Embodiment 2 of the present invention.

[0014] Figure 4 This is a diagram showing the configuration of the communication device in Embodiment 3 of the present invention. Detailed Implementation

[0015] (Basic knowledge of this invention)

[0016] In the aforementioned prior art, based on the relative positions of the noise source, the reference input terminal, and the audio input terminal, a peak tap position representing the peak value of the filter coefficient is derived. The intervals corresponding to the tap positions of the filter coefficients are then classified using these peak tap positions. For each classified interval, the update frequency of the filter coefficients corresponding to that interval is controlled. That is, in the prior art, a first interval near the peak tap position is determined, and the update frequency of the filter coefficients in intervals outside the first interval is lower than the update frequency of the filter coefficients in the first interval. Thus, in the prior art, the computational load is suppressed by reducing the update frequency of the filter coefficients in intervals where the filter coefficients change little due to the device or environment.

[0017] The aforementioned conventional techniques calculate the peak tap position of the tap representing the peak value of the filter coefficient based on the relative positions of the noise source, the reference input terminal, and the audio input terminal. However, they do not calculate the peak value of the difference signal obtained by subtracting the pseudo-noise signal generated by the adaptive filter from the signal generated from the audio input terminal. Therefore, it is believed that the aforementioned conventional techniques can be further improved to further reduce the computational load used for noise removal.

[0018] To address the above problems, one aspect of the present invention relates to a sound pickup device comprising: an adaptive filter that generates a speculative noise signal representing a component of a noise signal contained in an input signal acquired by a microphone from a reference signal; a signal generation unit that generates a noise-removed signal obtained by subtracting the speculative noise signal from the input signal; a coefficient update unit that updates the filter coefficients of the adaptive filter using the noise-removed signal; and a determination unit that determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, wherein the coefficient update unit updates the filter coefficients at the at least one signal sample position determined by the determination unit.

[0019] Conventionally, filter coefficients are updated at all signal sample positions at a specified sampling frequency. In contrast, according to this configuration, at least one signal sample position is determined from among multiple signal sample positions up to a predetermined large signal sample position, and the filter coefficients of the adaptive filter are updated at this determined at least one signal sample position. Therefore, the number of update operations for the adaptive filter coefficients is reduced, thereby reducing the computational load used for noise removal.

[0020] Furthermore, in the aforementioned pickup device, the at least one signal sample position can also be the signal sample position where the absolute value of the noise removal signal is the largest.

[0021] According to this configuration, the filter coefficients of the adaptive filter are updated at the signal sample position where the absolute value of the noise removal signal is the largest. Therefore, the filter coefficients are updated once per frame, which can further reduce the amount of computation used for noise removal.

[0022] Furthermore, in the aforementioned pickup device, the reference signal can also be a regenerated signal output to the loudspeaker.

[0023] Based on this configuration, it is possible to remove acoustic echo components contained in the input signal acquired through the microphone.

[0024] Furthermore, in the aforementioned pickup device, the coefficient update unit can also adjust the update speed of the filter coefficients based on the magnitude of the absolute value of the noise removal signal at the at least one signal sample location.

[0025] According to this configuration, for example, the noise removal signal can be roughly converged by increasing the update speed, and the noise removal signal can be accurately converged by decreasing the update speed.

[0026] Furthermore, in the aforementioned pickup device, the coefficient update unit may also make the update speed faster than currently when the absolute value of the noise removal signal at at least one signal sample position is greater than a threshold, and make the update speed slower than currently when the absolute value of the noise removal signal at at least one signal sample position is less than the threshold.

[0027] According to this configuration, when the absolute value of the noise-removed signal at at least one signal sample location is greater than a threshold, the update speed is set to be faster than the current speed, thus enabling coarse convergence of the noise-removed signal. Furthermore, when the absolute value of the noise-removed signal at at least one signal sample location is less than the threshold, the update speed is set to be slower than the current speed, thus enabling precise convergence of the noise-removed signal. As a result, both improved computational speed and improved computational accuracy can be achieved.

[0028] Furthermore, the aforementioned pickup device may also include: a first frequency band segmentation unit for segmenting the input signal into multiple frequency bands; a second frequency band segmentation unit for segmenting the reference signal into the multiple frequency bands; and a frequency band synthesis unit for synthesizing the noise-removed signal generated for each of the multiple frequency bands, wherein the adaptive filter includes multiple adaptive filters for generating the speculative noise signal for each of the multiple frequency bands, the signal generation unit includes multiple signal generation units for generating the noise-removed signal for each of the multiple frequency bands, the determination unit includes multiple determination units for determining the position of the at least one signal sample for each of the multiple frequency bands, and the coefficient update unit includes multiple coefficient update units for updating the filter coefficients at the position of the at least one signal sample for each of the multiple frequency bands.

[0029] According to this configuration, the input signal and the reference signal are divided into multiple frequency bands, and for each of the multiple frequency bands, the following processes are performed: noise signal generation, noise removal signal generation, signal sample location determination, and filter coefficient update. Therefore, the computation time can be shortened and the amount of computation per unit time can be reduced.

[0030] Furthermore, in the aforementioned sound pickup device, the adaptive filter may include: a first adaptive filter that generates from a reference signal a first speculative noise signal representing a component of the noise signal contained in a first input signal acquired through a first microphone; and a second adaptive filter that generates from the reference signal a second speculative noise signal representing a component of the noise signal contained in a second input signal acquired through a second microphone different from the first microphone. The signal generation unit includes: a first signal generation unit that generates a first noise-removed signal after subtracting the first speculative noise signal from the first input signal; and a second signal generation unit that generates a second noise-removed signal after subtracting the second speculative noise signal from the second input signal. The signal, wherein the coefficient update unit comprises: a first coefficient update unit that updates the filter coefficients of the first adaptive filter using the first noise removal signal; and a second coefficient update unit that updates the filter coefficients of the second adaptive filter using the second noise removal signal, and the determining unit comprises: a first determining unit that determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position from the signal sample position where the absolute value of the first noise removal signal is the largest; and a second determining unit that determines at least one signal sample position from the signal sample position where the absolute value of the second noise removal signal is the largest up to the predetermined large signal sample position.

[0031] According to this configuration, whenever one microphone is added, one more adaptive filter, one signal generation unit, one coefficient update unit, and one determination unit are each added. However, even with the increase in microphones, the computational load of each process is reduced compared to the past, thus reducing the overall computational load of the process.

[0032] Another aspect of the present invention relates to a sound pickup method comprising a sound pickup device having an adaptive filter, a signal generation unit, a coefficient update unit, and a determination unit, comprising the following steps: the adaptive filter generates a speculative noise signal representing a component of a noise signal contained in an input signal acquired by a microphone from a reference signal; the signal generation unit generates a noise removal signal obtained by subtracting the speculative noise signal from the input signal; the determination unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, where the absolute value of the noise removal signal is the largest; and the coefficient update unit updates the filter coefficients of the adaptive filter at the determined at least one signal sample position using the noise removal signal.

[0033] Conventionally, filter coefficients are updated at all signal sample positions at a specified sampling frequency. In contrast, according to this configuration, at least one signal sample position is determined from among multiple signal sample positions up to a predetermined large signal sample position, and the filter coefficients of the adaptive filter are updated at this determined at least one signal sample position. Therefore, the number of update operations for the adaptive filter coefficients is reduced, thereby reducing the computational load used for noise removal.

[0034] Another aspect of the present invention relates to a sound pickup program that allows a computer to function as the following units: an adaptive filter that generates a speculative noise signal from a reference signal representing a component of noise signal contained in an input signal acquired through a microphone; a signal generation unit that generates a noise-removed signal by subtracting the speculative noise signal from the input signal; a coefficient update unit that updates the filter coefficients of the adaptive filter using the noise-removed signal; and a determination unit that determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, wherein the coefficient update unit updates the filter coefficients at the at least one signal sample position determined by the determination unit.

[0035] Conventionally, filter coefficients are updated at all signal sample positions at a specified sampling frequency. In contrast, according to this configuration, at least one signal sample position is determined from among multiple signal sample positions up to a predetermined large signal sample position, and the filter coefficients of the adaptive filter are updated at this determined at least one signal sample position. Therefore, the number of update operations for the adaptive filter coefficients is reduced, thereby reducing the computational load used for noise removal.

[0036] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Furthermore, the following embodiments are merely examples embodying the present invention and do not limit the scope of the invention.

[0037] (Implementation Method 1)

[0038] Figure 1 This diagram illustrates the configuration of the communication device in Embodiment 1 of the present invention. Furthermore, the communication device is used in amplified hands-free communication system, amplified two-way communication conferencing system, and walkie-talkie system mounted in automobiles, etc.

[0039] Figure 1 The communication device shown includes a sound pick-up device 1, a microphone 11, an input terminal 12, a speaker 13, and an output terminal 17.

[0040] Microphone 11 is positioned within the space where the speaker is located to pick up the speaker's voice. Microphone 11 outputs an input signal representing the picked-up sound to the pickup device 1.

[0041] The input terminal 12 outputs the regenerated signal received from the call device (not shown) on the receiving side to the pickup device 1 and the speaker 13.

[0042] The speaker 13 outputs the input regenerated signal to the outside. Here, when the sound output from the speaker 13 is picked up by the microphone 11, the speaker on the receiving side regenerates the sound emitted by the speaker on the receiving side with a delay, resulting in a so-called acoustic echo.

[0043] The pickup device 1 includes an echo canceller 14, a noise removal signal generation unit 15, and a coefficient update judgment unit 16.

[0044] The echo canceller 14 includes an adaptive filter 141 and a filter coefficient update unit 142.

[0045] The adaptive filter 141 generates a speculative noise signal representing the noise components contained in the input signal acquired through the microphone 11, from a reference signal. The reference signal is, for example, the regenerated signal output to the speaker 13. The noise signal is, for example, an acoustic echo signal. The adaptive filter 141 generates the speculative noise signal representing the noise components contained in the input signal by convolving the filter coefficients with the reference signal. This speculative noise signal is also referred to as a pseudo-echo signal.

[0046] The noise removal signal generation unit 15 generates a noise removal signal obtained by subtracting the predicted noise signal from the input signal. The noise removal signal generation unit 15 generates the noise removal signal by subtracting the predicted noise signal from the input signal. The noise removal signal generation unit 15 outputs the generated noise removal signal to the coefficient update determination unit 16 and the output terminal 17.

[0047] The coefficient update judgment unit 16 includes an absolute value calculation unit 161 and a sample position determination unit 162.

[0048] The absolute value calculation unit 161 calculates the absolute value of the noise removal signal generated by the noise removal signal generation unit 15.

[0049] The sample position determination unit 162 determines at least one signal sample position from a plurality of signal sample positions up to the position of the signal sample with the largest absolute value of the noise-removed signal and up to the position of the predetermined largest signal sample.

[0050] The filter coefficient update unit 142 updates the filter coefficients of the adaptive filter 141 using the noise removal signal at a predetermined sampling period. The adaptive filter 141 generates a speculative noise signal by convolving the filter coefficients updated by the filter coefficient update unit 142 with a reference signal. The filter coefficient update unit 142 updates the filter coefficients at at least one signal sample position determined by the sample position determination unit 162. The at least one signal sample position is the signal sample position with the largest absolute value of the noise removal signal.

[0051] The filter coefficient update unit 142 uses an adaptive algorithm to update the filter coefficients in a way that minimizes the noise removal signal. As an adaptive algorithm, for example, a learning recognition method (NLMS (Normalized Least Mean Square) method), an affine projection method, or a recursive least squares method (RLS (Recursive Least Square) method) is used.

[0052] Output terminal 17 outputs a noise removal signal that has had its noise components (acoustic echo components) eliminated from the input signal by the pickup device 1. Output terminal 17 also outputs a noise removal signal generated by the noise removal signal generation unit 15.

[0053] Additionally, input terminal 12 and output terminal 17 are connected to a communication unit (not shown). The communication unit sends a noise reduction signal to the receiving end of the call (not shown) via a network, and receives a regenerated signal from the receiving end of the call (not shown) via the network. The network is, for example, the Internet.

[0054] Here, the filter coefficient update in this embodiment 1 will be explained.

[0055] Previously, filter coefficients were calculated using the following equation (1).

[0056] In equation (1) above, pfCoef[n] represents the filter coefficient at tap position n, μ represents the step gain, pfSpk[n+k] represents the reference signal, err[n+k] represents the noise removal signal, and k represents the signal sample position in one frame. Each frame contains L signal samples, and the signal sample position k can take values ​​from 0 to L-1. Additionally, there are M taps, and the tap position n can take values ​​from 0 to M-1.

[0057] As shown in equation (1) above, since there are L signal samples per frame, the filter coefficients are updated L times per frame. Furthermore, since there are M taps, the number of operations per frame for the filter coefficients is L*M times.

[0058] On the other hand, the filter coefficients in this embodiment 1 are calculated using the following equation (2).

[0059] In equation (2) above, pfCoef[n] represents the filter coefficient at tap position n, μ represents the step gain, pfSpk[n+k] represents the reference signal, err_max represents the maximum absolute value of the noise-removed signal in one frame, and t represents the signal sample position where the absolute value of the noise-removed signal in one frame becomes the largest. Each frame contains L signal samples, and the signal sample position k can take values ​​from 0 to L-1. Furthermore, there are M taps, and the tap position n can take values ​​from 0 to M-1.

[0060] As shown in equation (2) above, since the signal sample position t with the largest absolute value of the noise removal signal is determined from L signal sample positions, the filter coefficients of this embodiment 1 are updated only once per frame. Furthermore, since there are M taps, the number of operations on the filter coefficients of this embodiment 1 per frame is 1*M.

[0061] As described above, the filter coefficient update unit 142 in this embodiment 1 can significantly reduce the amount of computation required to update the filter coefficients compared to the conventional version.

[0062] Furthermore, the step gain μ in equations (1) and (2) above is also called the step size, which is a positive constant that determines the update speed of the filter coefficients.

[0063] The filter coefficient update unit 142 can also adjust the update speed of the filter coefficients based on the absolute value of the noise removal signal at at least one signal sample location. That is, the filter coefficient update unit 142 can also adjust the update speed based on the absolute value of the noise removal signal at the signal sample location with the largest absolute value of the noise removal signal. When the absolute value of the noise removal signal at at least one signal sample location is greater than a threshold, the filter coefficient update unit 142 adjusts the update speed faster than currently; when the absolute value of the noise removal signal at at least one signal sample location is less than the threshold, the update speed is slower than currently.

[0064] The filter coefficient update unit 142 can change the update speed of the filter coefficients by adjusting the step gain μ in equation (2) above. Specifically, when the absolute value of the noise-removed signal at the position of the signal sample with the largest absolute value of the noise-removed signal is greater than a threshold, the filter coefficient update unit 142 increases the step gain μ. As a result, the update speed of the filter coefficients is faster than currently. On the other hand, when the absolute value of the noise-removed signal at the position of the signal sample with the largest absolute value of the noise-removed signal is less than a threshold, the filter coefficient update unit 142 decreases the step gain μ. As a result, the update speed of the filter coefficients is slower than currently.

[0065] As described above, when the absolute value of the noise-removed signal at at least one signal sample location is greater than the threshold, the update speed is set to be faster than the current speed, thus enabling coarse convergence of the noise-removed signal. Furthermore, when the absolute value of the noise-removed signal at at least one signal sample location is less than the threshold, the update speed is set to be slower than the current speed, thus enabling precise convergence of the noise-removed signal. As a result, both improved computational speed and improved computational accuracy can be achieved.

[0066] Next, the operation of the pickup device 1 in Embodiment 1 of the present invention will be explained.

[0067] Figure 2 This is a flowchart illustrating the operation of the pickup device in Embodiment 1 of the present invention.

[0068] First, in step S1, the noise removal signal generation unit 15 acquires the input signal from the microphone 11. At this time, the microphone 11 outputs the input signal to the noise removal signal generation unit 15.

[0069] Next, in step S2, the adaptive filter 141 of the echo canceller 14 acquires a reference signal from the input terminal 12. At this time, the input terminal 12 outputs the regenerated signal received from the call device (not shown) on the receiving side to the pickup device 1 and the speaker 13. The adaptive filter 141 acquires the regenerated signal output from the input terminal 12 to the speaker 13 as a reference signal.

[0070] Next, in step S3, the adaptive filter 141 generates a speculative noise signal representing the components of the noise signal contained in the input signal by convolving the filter coefficients with the reference signal.

[0071] Next, in step S4, the noise removal signal generation unit 15 generates a noise removal signal by subtracting the inferred noise signal from the input signal. The noise removal signal generation unit 15 outputs the generated noise removal signal to the coefficient update determination unit 16 and the output terminal 17.

[0072] Next, in step S5, the output terminal 17 outputs the noise removal signal generated by the noise removal signal generation unit 15.

[0073] Next, in step S6, the absolute value calculation unit 161 of the coefficient update judgment unit 16 acquires the noise removal signal generated by the noise removal signal generation unit 15.

[0074] Next, in step S7, the absolute value calculation unit 161 determines whether a noise removal signal of 1 frame has been acquired.

[0075] Additionally, the pickup device 1 includes a memory (not shown). The noise removal signal generated by the noise removal signal generation unit 15 is stored in the memory. The absolute value calculation unit 161 can also determine whether a frame of noise removal signal has been acquired by judging whether a frame of noise removal signal is stored in the memory.

[0076] Here, if it is determined that no noise removal signal of 1 frame has been acquired (step S7 is no), the process returns to step S1.

[0077] On the other hand, if it is determined that a noise removal signal of 1 frame has been obtained (yes in step S7), in step S8, the absolute value calculation unit 161 calculates the absolute value of each noise removal signal obtained at the signal sample position of each frame.

[0078] Next, in step S9, the sample position determination unit 162 determines the position of the signal sample with the largest absolute value of the noise removal signal.

[0079] Next, in step S10, the filter coefficient update unit 142 updates the filter coefficients at the signal sample position with the largest absolute value of the noise removal signal determined by the sample position determination unit 162.

[0080] Conventionally, filter coefficients are updated at all signal sample positions at a specified sampling frequency. In contrast, according to Embodiment 1, at least one signal sample position is determined from among a plurality of signal sample positions up to a predetermined large signal sample position, and the filter coefficients of the adaptive filter 141 are updated at this determined at least one signal sample position. Therefore, the number of update operations for the filter coefficients of the adaptive filter 141 is reduced, thereby reducing the computational load for noise removal.

[0081] Furthermore, in this embodiment 1, the filter coefficient update unit 142 updates the filter coefficients at the signal sample position with the largest absolute value of the noise-removed signal, but the present invention is not particularly limited to this. The filter coefficient update unit 142 may update the filter coefficients at the signal sample position with the second largest absolute value of the noise-removed signal, or it may update the filter coefficients at the signal sample position with the third largest absolute value of the noise-removed signal. That is, the filter coefficient update unit 142 may update the filter coefficients at at least one signal sample position from the signal sample position with the largest absolute value of the noise-removed signal up to the predetermined largest signal sample position.

[0082] Furthermore, the filter coefficient update unit 142 can update the filter coefficients at the signal sample position with the largest absolute value of the noise-removed signal and at the signal sample position with the second largest absolute value of the noise-removed signal. That is, the filter coefficient update unit 142 can update the filter coefficients at at least two of the multiple signal sample positions from the signal sample position with the largest absolute value of the noise-removed signal to the predetermined largest signal sample position.

[0083] Furthermore, in this embodiment 1, the regenerated signal output to the speaker 13 is input to the adaptive filter 141 as a reference signal to generate a speculative noise signal representing the acoustic echo signal components contained in the input signal; however, the present invention is not particularly limited to this. Other microphones, different from the microphone 11, may also be arranged within the space where the microphone 11 is located. These other microphones can acquire noise in the space other than the sound acquired by the microphone 11, and output the acquired noise signal as a reference signal to the adaptive filter 141.

[0084] Furthermore, in this embodiment 1, the echo canceller 14 is input with a regenerated signal in the time domain, and the noise removal signal generation unit 15 is input with an input signal in the time domain. However, the present invention is not particularly limited to this, and the echo canceller 14 may also be input with a regenerated signal in the frequency domain, and the noise removal signal generation unit 15 may also be input with an input signal in the frequency domain.

[0085] In this case, a Fast Fourier Transform (FFT) unit that converts the time-domain regenerated signal input to the echo canceller 14 into a frequency-domain regenerated signal can be provided between the input terminal 12 and the speaker 13. Furthermore, a FFT unit that converts the time-domain input signal input to the noise removal signal generation unit 15 into a frequency-domain input signal can be provided between the microphone 11 and the noise removal signal generation unit 15. Additionally, an Inverse Fast Fourier Transform (IFT) unit that converts the frequency-domain noise removal signal input from the noise removal signal generation unit 15 to the output terminal 17 into a time-domain noise removal signal can be provided between the noise removal signal generation unit 15 and the output terminal 17.

[0086] (Implementation Method 2)

[0087] The communication device in Embodiment 1 has one microphone 11, while the communication device in Embodiment 2 has multiple microphones.

[0088] Figure 3 This is a diagram showing the configuration of the communication device in Embodiment 2 of the present invention.

[0089] Figure 3 The communication device shown includes a pickup device 1A, a first microphone 11A, a second microphone 11B, an input terminal 12, a speaker 13, a first output terminal 17A, and a second output terminal 17B. Furthermore, in Embodiment 2, the same reference numerals are used for components identical to those in Embodiment 1, and descriptions are omitted.

[0090] Microphone 11A and microphone 11B are positioned within the space where the speaker is located to pick up the speaker's voice. Microphone 11A outputs a first input signal representing the picked-up sound to microphone 1A. Microphone 11B outputs a second input signal representing the picked-up sound to microphone 1A.

[0091] The first output terminal 17A outputs a first noise removal signal, which eliminates the noise component (acoustic echo component) from the first input signal via the pickup device 1A. The second output terminal 17B outputs a second noise removal signal, which eliminates the noise component (acoustic echo component) from the second input signal via the pickup device 1A.

[0092] Additionally, input terminal 12, first output terminal 17A, and second output terminal 17B are connected to the communication unit (not shown). The communication unit sends a noise removal signal to the receiving end of the call (not shown) via a network, and receives a regenerated signal from the receiving end of the call (not shown) via a network.

[0093] The pickup device 1A includes a first echo canceller 14A, a first noise removal signal generation unit 15A, a first coefficient update judgment unit 16A, a second echo canceller 14B, a second noise removal signal generation unit 15B, and a second coefficient update judgment unit 16B.

[0094] The first echo canceller 14A includes a first adaptive filter 141A and a first filter coefficient update unit 142A. The second echo canceller 14B includes a second adaptive filter 141B and a second filter coefficient update unit 142B.

[0095] The first adaptive filter 141A generates a first speculative noise signal from the reference signal, representing the components of the noise signal contained in the first input signal acquired by the first microphone 11A.

[0096] The second adaptive filter 141B generates a second speculative noise signal from the reference signal, representing the components of the noise signal contained in the second input signal obtained by the second microphone 11B, which is different from the first microphone 11A.

[0097] The first noise removal signal generation unit 15A generates a first noise removal signal obtained by subtracting the first speculative noise signal from the first input signal. The first noise removal signal generation unit 15A generates the first noise removal signal by subtracting the first speculative noise signal from the first input signal. The first noise removal signal generation unit 15A outputs the generated first noise removal signal to the first coefficient update determination unit 16A and the first output terminal 17A.

[0098] The second noise removal signal generation unit 15B generates a second noise removal signal after subtracting the second speculative noise signal from the second input signal. The second noise removal signal generation unit 15B generates the second noise removal signal by subtracting the second speculative noise signal from the second input signal. The second noise removal signal generation unit 15B outputs the generated second noise removal signal to the second coefficient update determination unit 16B and the second output terminal 17B.

[0099] The first coefficient update judgment unit 16A includes a first absolute value calculation unit 161A and a first sample position determination unit 162A. The second coefficient update judgment unit 16B includes a second absolute value calculation unit 161B and a second sample position determination unit 162B.

[0100] The first absolute value calculation unit 161A calculates the absolute value of the first noise removal signal generated by the first noise removal signal generation unit 15A.

[0101] The first sample position determination unit 162A determines at least one signal sample position from a plurality of signal sample positions up to the first predetermined large signal sample position, from the signal sample position with the largest absolute value of the first noise removal signal.

[0102] The second absolute value calculation unit 161B calculates the absolute value of the second noise removal signal generated by the second noise removal signal generation unit 15B.

[0103] The second sample position determination unit 162B determines at least one signal sample position from a plurality of signal sample positions up to the position of the signal sample with the largest absolute value of the second noise removal signal.

[0104] The first filter coefficient update unit 142A updates the filter coefficients of the first adaptive filter 141A using the first noise removal signal. The first adaptive filter 141A generates a first speculative noise signal by convolving the filter coefficients updated by the first filter coefficient update unit 142A with a reference signal. The first filter coefficient update unit 142A updates the filter coefficients at at least one signal sample position determined by the first sample position determination unit 162A. The at least one signal sample position is the signal sample position where the absolute value of the first noise removal signal is the largest.

[0105] The second filter coefficient update unit 142B updates the filter coefficients of the second adaptive filter 141B using the second noise removal signal. The second adaptive filter 141B generates a second speculative noise signal by convolving the filter coefficients updated by the second filter coefficient update unit 142B with a reference signal. The second filter coefficient update unit 142B updates the filter coefficients at at least one signal sample position determined by the second sample position determination unit 162B. The at least one signal sample position is the signal sample position where the absolute value of the second noise removal signal is the largest.

[0106] Furthermore, the filter coefficient update process in this embodiment 2 is the same as the filter coefficient update process in embodiment 1.

[0107] Furthermore, in this embodiment 2, the communication device has two microphones, but the present invention is not particularly limited to this, and the communication device may also have three or more microphones.

[0108] As described above, each time one microphone is added, one echo canceller and one coefficient update determination unit are also added. However, even with the increase in microphones, the computational load of each process is reduced compared to the past, thus reducing the overall computational load of the process.

[0109] (Implementation Method 3)

[0110] In embodiment 3, the pickup device divides the input signal into multiple frequency bands, divides the reference signal into multiple frequency bands, generates a noise removal signal for each of the multiple frequency bands, and synthesizes the noise removal signal generated for each of the multiple frequency bands.

[0111] Figure 4 This is a diagram showing the configuration of the communication device in Embodiment 3 of the present invention.

[0112] Figure 4 The communication device shown includes a pickup device 1B, a microphone 11, an input terminal 12, a speaker 13, and an output terminal 17. Furthermore, in Embodiment 3, the same reference numerals are used for components identical to those in Embodiment 1, and descriptions are omitted.

[0113] The pickup device 1B includes a first echo canceller 14A, a first noise removal signal generation unit 15A, a first coefficient update judgment unit 16A, a second echo canceller 14B, a second noise removal signal generation unit 15B, a second coefficient update judgment unit 16B, a third echo canceller 14C, a third noise removal signal generation unit 15C, a third coefficient update judgment unit 16C, a fourth echo canceller 14D, a fourth noise removal signal generation unit 15D, a fourth coefficient update judgment unit 16D, a first frequency band segmentation unit 21, a second frequency band segmentation unit 22, and a frequency band synthesis unit 23.

[0114] The first frequency band segmentation unit 21 divides the input signal output from the microphone 11 into multiple frequency bands. In this embodiment 3, the input signal is divided into 4 frequency bands. The first frequency band segmentation unit 21 divides the full-band input signal into 4 sub-frequency bands with different frequency bands. The first frequency band segmentation unit 21 outputs the 4 sub-frequency band input signals to the first noise removal signal generation unit 15A, the second noise removal signal generation unit 15B, the third noise removal signal generation unit 15C, and the fourth noise removal signal generation unit 15D, respectively.

[0115] The second frequency band segmentation unit 22 divides the reference signal output from the input terminal 12 into multiple frequency bands. In this embodiment 3, the reference signal is divided into 4 frequency bands. The second frequency band segmentation unit 22 divides the full-band reference signal into 4 sub-band reference signals with different frequency bands. The second frequency band segmentation unit 22 outputs the 4 sub-band reference signals to the first echo canceller 14A, the second echo canceller 14B, the third echo canceller 14C, and the fourth echo canceller 14D, respectively.

[0116] The structures of the first echo canceller 14A, the second echo canceller 14B, the third echo canceller 14C, and the fourth echo canceller 14D are the same as those of the echo canceller 14 in Embodiment 1. That is, each of the first echo canceller 14A, the second echo canceller 14B, the third echo canceller 14C, and the fourth echo canceller 14D includes an adaptive filter 141 and a filter coefficient update unit 142.

[0117] The structures of the first coefficient update judgment unit 16A, the second coefficient update judgment unit 16B, the third coefficient update judgment unit 16C, and the fourth coefficient update judgment unit 16D are the same as those of the coefficient update judgment unit 16 in Embodiment 1. That is, each of the first coefficient update judgment unit 16A, the second coefficient update judgment unit 16B, the third coefficient update judgment unit 16C, and the fourth coefficient update judgment unit 16D has an absolute value calculation unit 161 and a sample position determination unit 162.

[0118] Multiple adaptive filters 141 generate speculative noise signals for each of multiple frequency bands.

[0119] The first noise removal signal generation unit 15A, the second noise removal signal generation unit 15B, the third noise removal signal generation unit 15C, and the fourth noise removal signal generation unit 15D generate noise removal signals for each of the multiple frequency bands.

[0120] The multiple sample location determination unit 162 determines at least one signal sample location for each of the multiple frequency bands.

[0121] Multiple filter coefficient update units 142 update the filter coefficients at at least one signal sample location for each of the multiple frequency bands. Furthermore, the filter coefficient update process in this embodiment 3 is the same as the filter coefficient update process in embodiment 1.

[0122] The band combining unit 23 combines the noise removal signals generated for each of the multiple frequency bands. The band combining unit 23 combines the noise removal signals generated by the first noise removal signal generation unit 15A, the second noise removal signal generation unit 15B, the third noise removal signal generation unit 15C, and the fourth noise removal signal generation unit 15D. The band combining unit 23 combines the noise removal signals of the four sub-bands into a full-band noise removal signal. The band combining unit 23 outputs the full-band noise removal signal to the output terminal 17.

[0123] In addition, in this embodiment 3, the input signal and the reference signal are divided into 4 frequency bands, but the present invention is not particularly limited to this. The input signal and the reference signal can be divided into 2 frequency bands, 3 frequency bands, or 5 or more frequency bands.

[0124] As described above, the input signal and the reference signal are divided into multiple frequency bands, and for each of the multiple frequency bands, the following processes are performed: noise signal generation, noise removal signal generation, signal sample location determination, and filter coefficient update. Therefore, the computation time can be shortened and the amount of computation per unit time can be reduced.

[0125] Furthermore, in the above embodiments, each component can be constructed using dedicated hardware or implemented by executing software programs suitable for each component. Each component can be implemented by reading and executing a software program stored on a recording medium such as a hard disk or semiconductor memory by a program execution unit such as a CPU or processor. Alternatively, the program can be implemented through a separate computer system by recording the program on a recording medium and transferring it, or by transferring the program via a network.

[0126] The functionality of the apparatus described in the embodiments of the present invention is typically implemented, in whole or in part, as an integrated circuit, i.e., a Large Scale Integration (LSI). These functions can be integrated into a single chip individually or in a manner that includes some or all of the functions. Furthermore, integrated circuitry is not limited to LSIs; it can also be implemented using dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconstruct the connections or settings of the circuitry elements within the LSI, can be utilized.

[0127] Furthermore, the functions of the apparatus involved in the embodiments of the present invention can be partially or entirely implemented by a program executed by a processor such as a CPU.

[0128] Furthermore, all the numbers used above are illustrative for the purpose of illustrating the present invention, and the present invention is not limited to the illustrative numbers.

[0129] Furthermore, the order in which the steps shown in the flowchart above are executed is an illustrative order for the purpose of specifically illustrating the present invention, and other orders may be used to achieve the same effect. Additionally, some of the above steps may be executed simultaneously (in parallel) with other steps.

[0130] Industrial availability

[0131] The technology involved in this invention can reduce the amount of computation required for noise removal, and therefore has practical value in removing noise signals contained in input signals acquired through a microphone.

Claims

1. A sound pickup device, characterized in that... include: An adaptive filter generates a speculative noise signal from a reference signal, representing the components of the noise signal contained in the input signal obtained through a microphone. The signal generation unit generates a noise-removed signal obtained by subtracting the speculative noise signal from the input signal; The coefficient update unit updates the filter coefficients of the adaptive filter using the noise removal signal; as well as, The determining unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, wherein... The coefficient update unit updates the filter coefficients at the position of the at least one signal sample determined by the determination unit. The coefficient update unit adjusts the update rate of the filter coefficients based on the magnitude of the absolute value of the noise-removed signal at the at least one signal sample location. The coefficient update unit adjusts the update speed faster than currently if the absolute value of the noise-removed signal at at least one signal sample location is greater than a threshold, and adjusts the update speed slower than currently if the absolute value of the noise-removed signal at at least one signal sample location is less than the threshold. The at least one signal sample location is the signal sample location where the absolute value of the noise-removed signal is the largest.

2. The pickup device according to claim 1, characterized in that, The reference signal is the regenerated signal output to the loudspeaker.

3. The pickup device according to claim 1, characterized in that... Also includes: The first frequency band segmentation unit divides the input signal into multiple frequency bands; The second frequency band segmentation unit divides the reference signal into the plurality of frequency bands; as well as, The frequency band synthesis unit synthesizes the noise-removed signal generated for each of the plurality of frequency bands, wherein... The adaptive filter comprises multiple adaptive filters that generate the inferred noise signal for each of the multiple frequency bands. The signal generation unit includes multiple signal generation units that generate the noise removal signal for each of the multiple frequency bands. The determining unit includes multiple determining units that determine the position of the at least one signal sample for each of the multiple frequency bands. The coefficient update unit includes multiple coefficient update units that update the filter coefficients at at least one signal sample position for each of the multiple frequency bands.

4. The pickup device according to claim 1, characterized in that, The adaptive filter includes: The first adaptive filter generates a first inferred noise signal from the reference signal, representing the components of the noise signal contained in the first input signal acquired through the first microphone; and, The second adaptive filter generates a second inferred noise signal from the reference signal, representing the components of the noise signal contained in the second input signal obtained through a second microphone different from the first microphone. The signal generation unit includes: The first signal generation unit generates a first noise removal signal obtained by subtracting the first speculative noise signal from the first input signal; and... The second signal generation unit generates a second noise-removed signal, which is the result of subtracting the second hypothesized noise signal from the second input signal. The coefficient update unit includes: The first coefficient update unit updates the filter coefficients of the first adaptive filter using the first noise removal signal; and, The second coefficient update unit updates the filter coefficients of the second adaptive filter using the second noise removal signal. The determining unit includes: The first determining unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, starting from the signal sample position where the absolute value of the first noise-removed signal is the largest; and, The second determining unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, from the signal sample position where the absolute value of the second noise removal signal is the largest.

5. A sound pickup method, comprising a sound pickup device including an adaptive filter, a signal generation unit, a coefficient update unit, and a determination unit, characterized in that... Includes the following steps: The adaptive filter generates a speculative noise signal from the reference signal, representing the components of the noise signal contained in the input signal obtained through the microphone; The signal generation unit generates a noise-removed signal obtained by subtracting the speculative noise signal from the input signal; The determining unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, starting from the signal sample position where the absolute value of the noise-removed signal is the largest; and, The coefficient update unit updates the filter coefficients of the adaptive filter using the noise removal signal at the determined at least one signal sample location. The coefficient update unit adjusts the update rate of the filter coefficients based on the magnitude of the absolute value of the noise-removed signal at the at least one signal sample location. The coefficient update unit adjusts the update speed faster than currently if the absolute value of the noise-removed signal at at least one signal sample location is greater than a threshold, and adjusts the update speed slower than currently if the absolute value of the noise-removed signal at at least one signal sample location is less than the threshold. The at least one signal sample location is the signal sample location where the absolute value of the noise-removed signal is the largest.

6. A computer program product comprising a pickup program that enables a computer to function as: An adaptive filter generates a speculative noise signal from a reference signal, representing the components of the noise signal contained in the input signal obtained through a microphone. The signal generation unit generates a noise-removed signal obtained by subtracting the speculative noise signal from the input signal; The coefficient update unit updates the filter coefficients of the adaptive filter using the noise removal signal; and, The determining unit determines at least one signal sample position from a plurality of signal sample positions up to a predetermined large signal sample position, wherein... The coefficient update unit updates the filter coefficients at the position of the at least one signal sample determined by the determination unit. The coefficient update unit adjusts the update rate of the filter coefficients based on the magnitude of the absolute value of the noise-removed signal at the at least one signal sample location. The coefficient update unit adjusts the update speed faster than currently if the absolute value of the noise-removed signal at at least one signal sample location is greater than a threshold, and adjusts the update speed slower than currently if the absolute value of the noise-removed signal at at least one signal sample location is less than the threshold. The at least one signal sample location is the signal sample location where the absolute value of the noise-removed signal is the largest.

Images