Noise elimination device and noise elimination method

Abstract

Noise reduction device, comprehensive: a target noise vector selection unit (103) for selecting from pre-acquired control vectors indicating the arrival direction of noises with respect to a sensor arrangement (200) comprising two or more acoustic sensors (200a, 200b, 200c), of a target noise control vector indicating an arrival direction of a target noise; an interference noise vector selection unit (104) for selecting, from the pre-acquired control vectors, an interference noise control vector specifying an arrival direction of interference noise other than the target noise; and a signal processing unit (105) for acquiring, based on two or more observation signals received from the sensor arrangement, the target noise control vector selected by the target noise vector selection unit and the interference noise control vector selected by the interference noise vector selection unit, a signal obtained by eliminating the interference noise from the observation signals, wherein the signal processing unit acquires a signal by time-frequency masking using a mask to block a time-frequency spectrum of the interference noise, which is obtained by eliminating the interference noise from the observation signals, and where, in time-frequency masking, a control vector for each time frequency is estimated from the two or more observation signals, and a similarity is calculated between a control vector of the estimated observation signal and the target noise control vector and the interference noise control vector, and if the control vector, which exhibits the maximum calculated similarity, is the target noise control vector, a time-frequency spectrum of the observation signal is passed through, and if the control vector exhibiting the maximum calculated similarity is not the target noise control vector, a time-frequency spectrum of the observation signal is blocked, where the mask is given by B ( ω , τ ) = { 1 ( ‖ a trg ( ω ) − a ^ ( ω , τ ) ‖ < ‖ a dst ( ω ) − a ^ ( ω , τ ) ‖ ) 0 (otherwise) where B(ω, τ) denotes the mask, ω represents a discrete frequency, τ represents a discrete frame number, a trg(ω) denotes the target noise control vector, â(ω, τ) denotes the estimated control vector of the observation signal, and a dst (ω) denotes the interference noise control vector.

Description

TECHNICAL FIELD

[0001] The present invention relates to a technique for eliminating noise other than voice coming from a desired direction. STATE OF THE ART

[0002] Conventionally, there is a noise elimination technique to amplify speech coming from a desired direction and eliminate noise other than speech by using a sensor array consisting of a plurality of acoustic sensors (for example, microphones) and performing predetermined signal processing on an observation signal obtained from each of the sensors.

[0003] For example, the noise elimination technique described above makes it possible to clarify voices that are difficult to detect due to noise generated by equipment such as air conditioning equipment, or to extract only the voice of a desired speaker when multiple speakers are speaking simultaneously. In this way, the noise elimination technique can not only make speech easier for people to hear, but also improve the robustness of speech recognition processing against noise by eliminating noise as a preprocessing step in speech recognition.

[0004] Various techniques for forming directivity through signal processing using a sensor array have been conventionally disclosed. For example, Non-Patent Literature 1 discloses a technique for eliminating noise other than the target noise by statistically calculating a linear filter coefficient that minimizes the average gain of an output signal and thus performing linear beamforming, using a steering vector indicating a pre-measured or generated arrival direction of the target noise, and under a condition that does not change the gain of the voice coming from the arrival direction of the target noise.

[0005] However, in the above-described technique described in Non-Patent Literature 1, the linear filter coefficient is calculated to adequately eliminate the interference noise, so an observation signal of interference noise requires a certain length. Since the information about a position of an interference noise source is not given in advance, it is necessary to estimate the position of the interference noise source from the observation signal. As a result, the technique described in Non-Patent Literature 1 has the problem that sufficient noise elimination processing performance cannot be obtained immediately after the start of noise elimination processing.

[0006] To solve this problem, in a noise signal processing device described in Patent Literature 1, noise is eliminated by generating a control vector indicating an arrival direction of the target noise in advance, calculating a similarity in a phase difference between sensors calculated from an observation signal for each time-frequency and phase difference between sensors calculated from the control vector in the arrival direction of the target noise, and applying time-frequency masking that passes only a time-frequency spectrum with a high similarity to the observation signal.

[0007] Document WO 2016 / 167141 A1 describes a method for noise suppression. In the method, a control vector of a desired sound and a control vector of a noise are determined from an observed signal. Furthermore, based on the determined control vectors, the method determines a filter that is used to suppress the noise in the observed signal.

[0008] Document US 2012 / 0 183 149 A1 describes a device for audio signal processing. LIST OF CITED PATENTS

[0009] Patent Literature 1: JP 2012-234150 A NON-PATENT LITERATURE

[0010] Non-Patent Literature 1: Futoshi Asano, “Sound Array Signal Processing Sound Source Localization / Tracking and Separation,” Corona Publishing Co., Ltd., 2011, pp. 86-88 SUMMARY OF THE INVENTIONTECHNICAL PROBLEM

[0011] Since in the noise signal processing device described above in Patent Literature 1, an output signal is determined only by the observation signal at that moment without using statistical calculation, stable noise elimination performance can be achieved immediately after the start of noise elimination processing.

[0012] However, since the noise signal processing device described in Patent Literature 1 uses only the arrival direction of the target sound as information about the arrival direction of a noise source to extract the target sound, a position where an interference noise source exists with respect to the target sound source is not taken into account. Therefore, the noise signal processing device described in Patent Literature 1 has a problem that the noise elimination performance is reduced when the arrival direction of the target sound and the arrival direction of the interference noise are close to each other, when a difference between the phase difference between the target sound and the interference noise observed by a sensor is small, or the like.

[0013] This is because in time-frequency masking in a low frequency range where the occurrence of the phase difference between the target noise and the interference noise is unlikely, there is a high probability that a time-frequency spectrum of the interference noise will be falsely passed, and it is difficult to obtain a high-quality output signal.

[0014] The present invention has been made to solve the above-mentioned problems, and its objects are to achieve good noise elimination performance even when an arrival direction of the target noise and an arrival direction of the interference noise are close to each other, and to achieve stable noise elimination performance immediately after the start of the noise elimination processing. SOLUTION TO THE PROBLEM

[0015] This object is achieved by subject matter having the features according to the independent claims. Advantageous embodiments of the invention are the subject of the figures, the description, and the dependent claims. A noise elimination device according to the present invention comprises: a target noise vector selection unit for selecting, from control vectors acquired in advance and indicating arrival directions of noises with respect to a sensor comprising two or more acoustic sensors, a target noise control vector indicating an arrival direction of a target noise; an interference noise vector selection unit for selecting, from the pre-acquired control vectors, an interference noise control vector indicating an arrival direction of an interference noise other than the target noise;and a signal processing unit for acquiring, based on two or more observation signals obtained from the sensor array, the target noise control vector selected by the target noise vector selecting unit and the interference noise control vector selected by the interference noise vector selecting unit, a signal obtained by eliminating the interference noise from the observation signals.; ADVANTAGEOUS EFFECTS OF THE INVENTION

[0016] According to the present invention, even when an arrival direction of the target sound and an arrival direction of the interference sound are close to each other, good noise elimination performance can be achieved, and stable noise elimination performance can be achieved immediately after the start of noise processing. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing a configuration of a noise eliminating device according to a first embodiment. Fig. 2A and Fig. 2B are diagrams illustrating a hardware configuration example of the noise eliminating device according to the first embodiment. Fig. 3 is a flowchart showing an operation of a signal processing unit of the noise elimination device according to the first embodiment. Fig. 4 is a flowchart showing an operation of a signal processing unit of the noise elimination device according to a second embodiment. Fig. 5 is a diagram showing an application example of the noise eliminating device according to the first embodiment or the second embodiment. Fig. 6 is a diagram showing an application example of the noise eliminating device according to the first embodiment or the second embodiment. DESCRIPTION OF THE EMBODIMENTS

[0017] Hereinafter, in order to explain the present invention in more detail, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

[0018] Furthermore, in the embodiments for implementing the present invention, an omnidirectional microphone is used as a concrete example of an acoustic sensor, and a sensor arrangement using a microphone arrangement is described. Note that the acoustic sensor is not limited to the omnidirectional microphone and is also applicable to, for example, a directional microphone or an ultrasonic sensor. First embodiment

[0019] Fig. Figure 1 is a block diagram showing a configuration of a noise canceling device 100 according to a first embodiment. The first embodiment is an example not falling within the scope of the claims.

[0020] The noise elimination device 100 includes an observation signal acquisition unit 101, a vector storage unit 102, a target noise vector selection unit 103, an interference noise vector selection unit 104, and a signal processing unit 105.

[0021] Furthermore, a microphone arrangement 200 comprising a plurality of microphones 200a, 200b, 200c, ... and an external device 300 are connected to the noise elimination device 100.

[0022] In the noise elimination device 100, the signal processing unit 105 generates an output signal obtained by eliminating noise from the observation signals based on observation signals observed by the microphone array 200 and control vectors selected and outputted by the target noise vector selection unit 103 and the interference noise vector selection unit 104 from among the control vectors stored in the vector storage unit 102, and outputs the output signal to the external device 300.

[0023] The observation signal acquisition unit 101 performs A / D conversion of the observation signals observed by the microphone array 200 and converts them into digital signals. The observation signal acquisition unit 101 outputs the observation signals converted into digital signals to the signal processing unit 105.

[0024] The vector storage unit 102 is a storage area for storing a plurality of control vectors measured or generated in advance. The control vector is a vector corresponding to a sound arrival direction as viewed from the microphone array 200. The control vector stored in the vector storage unit 102 is a spectrum in which frequency spectra obtained by discrete Fourier transform of impulse responses in specific directions measured in advance by the microphone array 200 are divided by a frequency spectrum of an arbitrary microphone and normalized. In other words, when the number of microphones constituting the microphone array 200 is M, a complex vector a(ω) shown in the following equation (1) is generated by using the frequency spectra S1(ω) to S M(ω), obtained by discrete Fourier transform of the impulse responses measured by the M microphones, is set as a control vector. In equation (1), ω represents a discrete frequency and T represents a vector transposition. a(ω)=(1S2(ω)S1(ω)⋯SM(ω)S1(ω))T

[0025] It should be noted that the control vector does not necessarily have to be obtained by the same method as Equation (1) described above. For example, in Equation (1) above, normalization is performed by the frequency spectrum S1(ω) corresponding to the first of the M microphones, but normalization may be performed by a frequency spectrum corresponding to a microphone other than the first microphone. Furthermore, the frequency spectra of the impulse responses can be used as control vectors without normalization. However, in the following description, it is assumed that the control vector is normalized by the frequency spectrum corresponding to the first microphone, as shown in Equation (1).

[0026] The target sound vector selection unit 103 selects a control vector indicating a direction in which the desired voice arrives (hereinafter referred to as a target sound control vector) from the control vectors stored in the vector storage unit 102. The target sound vector selection unit 103 outputs the selected target sound control vector to the signal processing unit 105. The direction in which the target sound vector selection unit 103 selects the target sound control vector is determined based on, for example, a direction in which the desired voice determined based on a user input arrives.

[0027] The interference noise vector selection unit 104 selects, from the control vectors stored in the vector storage unit 102, a control vector indicating a direction in which interference noise to be eliminated arrives (hereinafter referred to as an interference noise control vector). The interference noise vector selection unit 104 outputs the selected interference noise control vector to the signal processing unit 105. The direction in which the interference noise vector selection unit 104 selects the interference noise control vector is determined based on, for example, a direction in which interference noise to be eliminated, determined based on a user input, arrives.

[0028] However, in a situation where a positional relationship between a target sound source and an interference sound source does not change, the target sound vector selecting unit 103 may continue to output a control vector in an arrival direction of a single target sound, and the interference sound vector selecting unit 104 may continue to output a control vector in an arrival direction of a single interference sound.

[0029] When a plurality of target noise sources and interference noise sources exist, the target noise vector selection unit 103 can output a plurality of target noise control vectors, and the interference noise vector selection unit 104 can output a plurality of interference noise control vectors. Since a plurality of target noise sources exist in this case, the noise elimination device 100 can output a plurality of target noises obtained by eliminating interference noise as a plurality of output signals.

[0030] However, for the sake of simplicity of description, it is assumed below that the target noise vector selection unit 103 and the interference noise vector selection unit 104 each select and output a single target noise control vector and a single interference noise control vector. In other words, the output signal of the signal processing unit 105 is a target noise signal obtained by eliminating a single interference noise. In the following, the target noise control vector selected and output by the target noise vector selection unit 103 is also referred to as a target noise control vector a. trg (ω). Hereinafter, the interference noise control vector selected and output by the interference noise vector selection unit 104 is also referred to as an interference noise control vector a dst (ω) is described.

[0031] Using the observation signals obtained by the observation signal acquisition unit 101, the target noise control vector obtained by the target noise vector selection unit 103, and the interference noise control vector obtained by the interference noise vector selection unit 104, the signal processing unit 105 outputs a signal obtained by eliminating noise other than the target noise as an output signal. Here, a mounting method by linear beamforming will be described as an example of the signal processing unit 105.

[0032] Subsequently, the signal processing unit 105 performs a discrete Fourier transformation on the signals observed by the M microphones to obtain the time-frequency spectra X1(ω, τ) to X M(ω, τ). Here, τ represents a discrete frame number. The signal processing unit 105 obtains a time-frequency spectrum Y(ω, τ) of an output signal by linear beamforming based on the following equation (2). x(ω, τ) in equation (2) is a complex vector in which the time-frequency spectra X1(ω, τ) to X M (ω, τ) are arranged as shown in equation (3). Furthermore, w(ω) in equation (2) is a complex vector in which linear filter coefficients are arranged in linear beamforming. Furthermore, H in equation (2) represents a complex conjugate transpose of a vector or matrix. Y(ω,τ)=w(ω)H×(ω,τ) x(ω,τ)=(X1(ω,τ),⋯,XM(ω,τ))

[0033] When the linear filter coefficient w(ω) is appropriately given in equation (2) described above, the signal processing unit 105 acquires the time-frequency spectrum Y(ω, τ) obtained by eliminating noise. Here, a condition that the linear filter coefficient w(ω) must satisfy is a condition for ensuring amplification of the target noise and setting amplification of the interference noise to zero. In other words, after forming the directivity in the arrival direction of the target noise, the linear filter coefficient w(ω) forms a blind spot in the arrival direction of the interference noise. This corresponds to the linear filter coefficient w(ω) satisfying the following equations (4) and (5). w(ω)Hatrg(ω)=1 w(ω)Hadst(ω)=0

[0034] Equations (4) and (5) described above can be expressed using a matrix as equation (6). Note that A in equation (6) is a complex matrix represented by the following equation (7), and r in equation (6) is a vector represented by the following equation (8). AHw(ω)=r A=(atrg(ω)adst(ω)) r=(1 0)T

[0035] The linear filter coefficient w(ω) that satisfies equation (6) described above is given by the following equation (9). w(ω)=A+r

[0036] A +In the above equation (9), is a pseudo-inverse Moore-Penrose matrix of the matrix A. The signal processing unit 105 calculates the above-described equation (2) using the linear filter coefficient w(ω) obtained by the above-described equation (9). As a result, the signal processing unit 105 acquires the time-frequency spectrum Y(ω, τ) obtained by eliminating the noise. The signal processing unit 105 performs a discrete inverse Fourier transform on the acquired time-frequency spectrum Y(ω, τ), reconstructs a time waveform, and outputs it as a final output signal.

[0037] The external device 300 is a device that is implemented, for example, with a speaker unit, a storage medium such as a hard disk or RAM, and outputs the output signal output by the signal processing unit 105. If the external device 300 is implemented with a speaker unit, the output signal is output as a noise wave from the speaker unit. If the external device 300 is implemented with a storage medium such as a hard disk or RAM, the storage medium stores the output signal as digital data on the hard disk or RAM.

[0038] Next, a hardware configuration example of the noise elimination device 100 will be described.

[0039] Fig. 2A and Fig. 2B are diagrams illustrating the hardware configuration examples of the noise canceling device 100.

[0040] The vector storage unit 102 in the noise elimination device 100 is implemented by a mass storage device 100a. Furthermore, the functions of the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 in the noise elimination device 100 are implemented by a processing circuit. In other words, the noise elimination device 100 includes the processing circuit for realizing the above-mentioned functions. The processing circuit may be a processing circuit 100b, which is dedicated hardware, as shown in Fig. 2A, or a processor 100c for executing a program stored in a memory 100d, as shown in Fig. 2B.

[0041] As in Fig. As shown in FIG. 2A, when the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 are dedicated hardware, the processing circuit 100b corresponds, for example, to a single circuit, a compound circuit, a programmed processor, a parallel programmed processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. Furthermore, the respective functions of the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 may be implemented by the processing circuit, or may be implemented by a single processing circuit by combining the functions of the units.

[0042] As in Fig. As shown in Figure 2B, when the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 are the processor 100c, the functions of the units are implemented by software, firmware, or a combination of software and firmware. The software or firmware is each described as a program and stored in the memory 100d. The processor 100c implements the functions of the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 by reading and executing the program stored in the memory 100d.In other words, when the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 are equipped with the working memory 100d for storing a program, the information shown in . Fig. 3 below are executed as a result when the program is executed by the processor 100c. Furthermore, these programs cause a computer to execute operations or processes of the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105.

[0043] Here, the processor 100c may be a central processing unit (CPU), a processing device, an arithmetic device, a microprocessor, a microcomputer, a digital signal processor (DSP), or the like.

[0044] Meanwhile, the memory 100d may be, for example, a non-volatile or volatile semiconductor memory such as random access memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), or electronically programmable read-only memory (EEPROM). This may be a hard disk, a magnetic disk such as a flexible disk, or an optical disk such as a minidisk, a compact disc (CD), or a digital versatile disc (DVD).

[0045] It should be noted that some of the functions of the observation signal acquisition unit 101, the target noise vector selection unit 103, the interference noise vector selection unit 104, and the signal processing unit 105 may be implemented by dedicated hardware, and some of them may be implemented by software or firmware. As described above, the processing circuit 100b in the noise elimination device 100 may implement the above-described functions by hardware, software, firmware, or a combination thereof.

[0046] Next, an operation of the noise elimination device 100 will be described.

[0047] Fig. 3 is a flowchart showing an operation of the signal processing unit 105 of the noise elimination device 100 according to the first embodiment.

[0048] In the flowchart of Fig. 3, it is assumed that the positions of a target noise source and an interference noise source do not change while the noise elimination device 100 performs the noise elimination processing, and is explained. In other words, it is assumed that a target noise control vector and an interference noise control vector do not change during the noise elimination process.

[0049] The signal processing unit 105 obtains a linear filter coefficient w(ω) from the target noise control vector selected by the target noise vector selection unit 103 and the interference noise control vector selected by the interference noise vector selection unit 104 (step ST1). The signal processing unit 105 accumulates observation signals input from the observation signal acquisition unit 101 in a buffer area (not shown) (step ST2).

[0050] The signal processing unit 105 determines whether or not the accumulated observation signals have a predetermined length (step ST3). If the accumulated observation signals do not have the predetermined length (step ST3; NO), the process returns to step ST2. On the other hand, if the accumulated observation signals have the predetermined length (step ST3; YES), the signal processing unit 105 performs a discrete Fourier transform on the accumulated observation signals to obtain an observation signal vector x(w, τ) (step ST4).

[0051] The signal processing unit 105 obtains a time-frequency spectrum Y(ω, τ) from the linear filter coefficient w(ω) obtained in step ST1 and the observation signal vector x(w, τ) obtained in step ST4 (step ST5). The signal processing unit 105 performs a discrete inverse Fourier transform on the time-frequency spectrum Y(ω, τ) obtained in step ST5 to obtain a time waveform (step ST6). The signal processing unit 105 outputs the time waveform obtained in step ST6 as an output signal to the external device 300 (step ST7), and the process ends.

[0052] As described above, according to the first embodiment, there are provided: a target sound vector selection unit 103 for selecting, from control vectors acquired in advance and indicating arrival directions of noise with respect to a sensor array including two or more acoustic sensors, a target sound control vector indicative of an arrival direction of target sound; an interference noise vector selection unit 104 for selecting, from the control vectors acquired in advance, an interference noise control vector indicative of an arrival direction of interference noise other than the target sound; and a signal processing unit 105 for acquiring, based on two or more observation signals obtained from the microphone array 200, the selected target sound control vector and the selected interference noise control vector, a signal obtained by eliminating the interference noise from the observation signals.Therefore, by using both the control vector in the arrival direction of the target sound and the control vector in the arrival direction of the interference noise, speech amplification in the arrival direction of the target sound can be ensured and amplification in the arrival direction of the interference noise can be reduced. Compared with noise elimination processing that uses only the control vector in the arrival direction of the target sound, noise elimination performance can be improved when the arrival direction of the target sound and the arrival direction of the interference noise are close to each other, and a high-quality output signal can be obtained.In addition, since the control vector in the arrival direction of the target noise and the control vector in the arrival direction of the interference noise are given, it is not necessary to estimate the position of a noise source from the observation signals, and a stable noise elimination performance can be achieved immediately after the start of noise elimination.

[0053] In addition, according to the first embodiment, since the signal processing unit 105 acquires the signal obtained by eliminating the interference noise from the observation signals by linear beamforming having a linear filter coefficient with the arrival direction of the target noise as a directivity forming direction and the arrival direction of the interference noise as a blind spot forming direction, an output signal with low distortion can be obtained by linear beamforming, and a high-quality output signal can be obtained. Second embodiment

[0054] In the first embodiment described above, the configuration in which the signal processing unit 105 is implemented using a method based on linear beamforming was described, but in this second embodiment, a configuration in which the signal processing unit 105 is implemented using a method based on nonlinear processing is described. The nonlinear processing here is, for example, time-frequency masking.

[0055] Since a block diagram showing a configuration of a noise canceling device 100 according to the second embodiment is the same as that of the first embodiment, a description thereof will be omitted. Furthermore, the components of the noise canceling device 100 according to the second embodiment are described with the same reference numerals as those used in the first embodiment.

[0056] The following describes a configuration in which the signal processing unit 105 performs signal processing using time-frequency masking based on the similarity between an observation signal input from an observation signal acquisition unit 101 and a control vector stored in a pre-measured vector storage unit 102.

[0057] In the same manner as the linear beamforming processing described in the first embodiment, the signal processing unit 105 sets time-frequency spectra obtained by performing discrete Fourier transform on the observation signals observed by M microphones to X1(ω, τ) to X M (ω, 1). If speech sparsity is detected at this time, as shown in the following equation (10), the signal processing unit 105 obtains an estimated value â(ω, τ) of a control vector of an observation signal by dividing and normalizing the observation signals by a time-frequency spectrum corresponding to the first microphone. a^(ω,τ)=(1x2(ω,τ)X1(ω,τ)⋯XM(ω,τ)X1(ω,τ))T

[0058] In an ideal environment where speech sparsity is fully detected, when a spectrum of the observation signal in a time-frequency is target noise, the estimated value â(ω, τ) of the control vector of the observation signal obtained based on the above equation (10) agrees with a target noise control vector a trg (ω), and in case of interference noise, the estimate â(ω, τ) agrees with an interference noise control vector a dst (ω). This is because the target noise control vector a trg (ω) and the interference noise control vector a dst (ω) are normalized by equation (1) described above in the same way as the observation signals in equation (10) described above.

[0059] Therefore, the signal processing unit 105 can, based on the estimated value â(ω, τ) of the control vector of the observation signal and a control vector a trg(ω) and the interference noise control vector a dst (ω) generate an optimal time-frequency mask.

[0060] In practice, however, an error is included in the estimated value â(ω, τ) of the control vector of the observation signal. Accordingly, the signal processing unit 105 can achieve stable noise elimination performance by generating a time-frequency mask based on a similarity between the estimated value â(ω, τ) of the control vector of the observation signal and a value derived from the target noise control vector a. trg (ω) and the interference noise control vector a dst (ω). In the signal processing unit 105, the estimated value â(ω, τ) of the control vector of the observation signal calculates a similarity between the target noise control vector a trg (ω) and the interference noise control vector a dst (ω). If a control vector having the maximum calculated similarity, the target noise control vector a trg(ω), the signal processing unit 105 passes a time-frequency spectrum of the observation signal. On the other hand, if the control vector having the maximum calculated similarity is the interference noise control vector a dst (ω), the signal processing unit 105 blocks the time-frequency spectrum of the observation signal.

[0061] Specifically, when a time-frequency mask that passes only the target noise is B(ω, τ), the signal processing unit 105 generates a time-frequency mask B(ω, τ) based on a distance between the control vectors, as shown in the following equation (11). B(ω,τ)={1(‖atrg(ω)−a^(ω,τ)‖<‖adst(ω)−a^(ω,τ)‖)0(otherwise)

[0062] According to equation (11), the time-frequency mask B(ω, t) only passes a time-frequency spectrum of the target noise and blocks a time-frequency spectrum other than the target noise.

[0063] Using the time-frequency mask B(ω, τ), the signal processing unit 105 obtains a time-frequency spectrum Y(ω, τ) of an output signal based on the following equation (12). Y(ω,τ)=B(ω,τ)X1(ω,τ)

[0064] The signal processing unit 105 performs a discrete inverse Fourier transform on the obtained time-frequency spectrum Y (ω, τ), reconstructs a time waveform, and generates an output signal. The signal processing unit 105 outputs the generated output signal to an external device 300.

[0065] Fig. 4 is a flowchart showing an operation of a signal processing unit 105 of the noise elimination device 100 according to the second embodiment.

[0066] As a prerequisite for carrying out the tasks described in the flow chart of Fig. 4, it is assumed that a target noise control vector and an interference noise control vector do not change while the noise elimination device 100 performs noise elimination processing.

[0067] It should be noted that hereinafter, the same steps as those of the noise elimination device 100 according to the first embodiment are denoted by the same reference numerals as in Fig. 3 and the description of these steps is omitted or simplified.

[0068] The signal processing unit 105 accumulates observation signals input from the observation signal acquisition unit 101 in a buffer area (not shown) (step ST2). The signal processing unit 105 determines whether or not the accumulated observation signals have a predetermined length (step ST3). If the accumulated observation signals do not have the predetermined length (step ST3; NO), the process returns to step ST2. On the other hand, if the accumulated observation signals have the predetermined length (step ST3; YES), the signal processing unit 105 performs a discrete Fourier transform on the accumulated observation signals to obtain time-frequency spectra X1(ω, τ) to X M (ω, τ) of the observation signals (step ST11). The signal processing unit 105 obtains an estimated value â(ω, τ) of a control vector of an observation signal from the time-frequency spectra X1(ω, τ) to XM (ω, τ) of the observation signals obtained in step ST11 (ST12).

[0069] The signal processing unit 105 generates a mask based on a distance between the estimated value â(ω, τ) of the control vector of the observation signal obtained in step ST12 and a target noise control vector a trg (ω) and a distance between the estimated value â(ω, τ) of the control vector of the observation signal and an interference noise control vector a dst (ω) (step ST13). In the detailed description of the processing in step ST13, the signal processing unit 105 generates a time-frequency mask B(ω, τ) which becomes "1" at a time-frequency in which the distance between the estimated value â(ω, τ) of the control vector of the observation signal and the target noise control vector a trg (ω) is smaller than the distance between the estimated value â(ω, τ) of the control vector of the observation signal and the interference noise control vector a dst(ω) and generates a time-frequency mask B(ω, τ) which becomes “0” in the other time-frequency.

[0070] The signal processing unit 105 obtains a time-frequency spectrum Y(ω, τ) of an output signal from the time-frequency spectrum X1(ω, τ) of the observation signal obtained in step ST11 and the mask generated in step ST13 (step ST14). The signal processing unit 105 performs a discrete inverse Fourier transform on the time-frequency spectrum Y(ω, τ) obtained in step ST14 to obtain a time waveform (step ST6). The signal processing unit 105 outputs the time waveform obtained in step ST6 as an output signal to the external device 300 (step ST7), and the process ends.

[0071] As described above, according to the second embodiment, since the signal processing unit 105 acquires a signal obtained by eliminating interference noise from the observation signals through time-frequency masking using a mask that blocks a time-frequency spectrum of the interference noise, there is no restriction that the number of control vectors to be simultaneously extracted or eliminated must be equal to or smaller than the number of microphones, and can be applied in a wide range of situations. Furthermore, higher noise elimination performance can be achieved than with linear beamforming.

[0072] Furthermore, according to the second embodiment, in time-frequency masking, a control vector is estimated for each time-frequency from the two or more observation signals, and a similarity between the estimated control vector of the observation signal and the target noise control vector and the interference noise control vector is calculated. When the control vector with the maximum calculated similarity is the target noise control vector, a time-frequency spectrum of the observation signal is passed through, and when the control vector with the maximum calculated similarity is not the target noise control vector, a time-frequency spectrum of the observation signal is blocked. Therefore, since not only a time difference of the speech observed by the microphone array but also an amplitude difference are simultaneously considered, it is possible to generate a more accurate time-frequency mask. As a result, high noise elimination performance can be obtained.

[0073] The noise elimination device 100 described in the first embodiment or the second embodiment can be applied to a recording system, a hands-free system, a voice recognition system, or the like.

[0074] First, a case will be described where the noise eliminating device 100 described in the first or second embodiment is applied to a recording system.

[0075] Fig. 5 is a diagram showing an application example of the noise eliminating device 100 according to the first embodiment or the second embodiment. Fig. 5 shows a case where the noise canceling device 100 is applied to a recording system that records, for example, speech in a conference.

[0076] As in Fig. As shown in Figure 5, the noise canceler 100 is placed on a conference table 400. The conference participants sit on a plurality of chairs 500 arranged around the conference table 400. It is assumed that the vector storage unit 102 of the noise canceler 100 stores in advance a result obtained by measuring a control vector corresponding to an arrangement direction of each chair 500 as viewed from the microphone array 200 connected to the noise canceler 100.

[0077] When extracting utterances from each conference participant individually, the target sound vector selection unit 103 selects the control vector corresponding to the arrangement direction of each chair 500 as a target sound control vector. On the other hand, the interference sound vector selection unit 104 selects a control vector corresponding to a different direction than the above-described chair 500 as an interference sound control vector.

[0078] When the conference starts, with the conference participants seated on chairs 500, the microphone array 200 collects the conference participants' voices and outputs them as observation signals to the noise canceler 100. The observation signal acquisition unit 101 of the noise canceler 100 converts the input observation signals into digital signals and outputs the digital signals to the signal processing unit 105. Using the observation signals input from the observation signal acquisition unit 101, the target noise control vector selected by the target noise vector selection unit 103, and the interference noise control vector selected by the interference noise vector selection unit 104, the signal processing unit 105 extracts the individual utterances of the conference participants.The external device 300 records the voice signals of the individual utterances of the conference participants extracted by the signal processing unit 105. For example, minutes can easily be created using the recording system.

[0079] On the other hand, when only the utterance of a specific conference participant is extracted, the target sound vector selection unit 103 selects a control vector corresponding to a disposition direction of the chair 500 of the conference participant whose utterance is being extracted as the target sound control vector. On the other hand, the interference noise vector selection unit 104 selects a control vector corresponding to a direction other than the direction of the conference participant described above as the interference noise control vector.

[0080] When the conference participants are seated in the chairs 500 and the conference is started, the microphone array 200 collects utterances from the conference participants and outputs them as observation signals to the noise elimination device 100. The observation signal acquisition unit 101 of the noise elimination device 100 converts the input observation signals into digital signals and outputs the digital signals to the signal processing unit 105. Using the observation signals input from the observation signal acquisition unit 101, the target noise control vector selected by the target noise vector selection unit 103, and the interference noise control vector selected by the interference noise vector selection unit 104, the signal processing unit 105 extracts only the utterance of the specific conference participant.The external device 300 records a voice signal of the utterance of the specific conference participant extracted by the signal processing unit 105.

[0081] As described above, under the premise that the speaker units are seated on the chairs 500, by measuring in advance the control vectors corresponding to the directions of the chairs 500, the utterances of the speaker units placed on the chairs 500 can be extracted or eliminated with high accuracy.

[0082] Next, a case will be described where the noise canceling device 100 shown in the first embodiment or the second embodiment is applied to a hands-free system or a voice recognition system.

[0083] Fig. 6 is a diagram showing an application example of the noise eliminating device 100 according to the first embodiment or the second embodiment. Fig.6 shows a case where the noise canceling device 100 is applied to a hands-free system or a voice recognition system in a vehicle. The noise canceling device 100 is arranged, for example, in front of a vehicle 600, that is, in front of the vehicle 600 with respect to a driver's seat 601 and a passenger seat 602.

[0084] A driver 601a of the vehicle 600 sits in the driver's seat 601. The other passengers 602a, 603a, and 603b of the vehicle 600 sit in the front passenger seat 602 and the rear seats 603. The noise elimination device 100 collects utterances of the driver 601a sitting in the driver's seat 601 and performs noise elimination processing for hands-free calling or noise elimination processing for voice recognition. In order for the driver 601a to conduct a hands-free telephone call or to perform voice recognition of the voice of the driver 601a, it is necessary to eliminate various noises mixed with the utterance of the driver 601a. For example, when the driver 601a speaks, speech uttered by the occupant 602a sitting in the passenger seat 602 becomes a noise to be eliminated.

[0085] It is assumed that the vector storage unit 102 of the noise canceler 100 stores in advance the results obtained by measuring control vectors corresponding to the directions of the driver's seat 601 and the passenger's seat 602, as viewed by the microphone array 200 connected to the noise canceler 100. Next, when only the utterance of the driver 601a sitting in the driver's seat 601 is extracted, the target noise vector selection unit 103 selects the control vector corresponding to the direction of the driver's seat 601 as a target noise control vector. On the other hand, the interference noise vector selection unit 104 selects the control vector corresponding to the direction of the passenger's seat 602 as an interference noise control vector.

[0086] When the driver 601a and the passenger 602a speak, the microphone assembly 200 collects the speech of the driver 601a and outputs it as an observation signal to the noise canceler 100. The observation signal acquisition unit 101 of the noise canceler 100 converts the input observation signal into a digital signal and outputs the digital signal to the signal processing unit 105. Using the observation signal input from the observation signal acquisition unit 101, the target noise control vector selected by the target noise vector selection unit 103, and the interference noise control vector selected by the interference noise vector selection unit 104, the signal processing unit 105 extracts the individual utterances of the driver 601a. The external device 300 accumulates voice signals of the individual utterances of the driver 601a extracted by the signal processing unit 105.The hands-free system or the voice recognition system performs voice call processing or voice recognition processing using the voice signals accumulated in the external device 300. As a result, voice call processing or voice recognition processing can be performed by eliminating the voice uttered by the occupant 602a sitting in the passenger seat 602 and extracting only the utterances of the driver 601a with high accuracy.

[0087] Note that in the above description, the speech uttered by the occupant 602a sitting in the front passenger seat 602 was described as an example of noise to be eliminated when the driver 601a speaks. However, in addition to the front passenger seat 602, speech uttered by the occupants 603a, 603b sitting in the rear seats 603 may be eliminated as noise.

[0088] As described above, by measuring in advance the control vectors corresponding to the directions of the driver's seat 601, the passenger seat 602, and the rear seats 603 of the vehicle 600, the utterance of the driver 601a sitting in the driver's seat 601 can be accurately extracted. This can improve the call sound quality in the hands-free system. Furthermore, in the voice recognition system, the driver's utterance can be recognized with high accuracy even in the presence of noise.

[0089] Contrary to what is described above, the present invention can freely combine embodiments, modify any components in the embodiments, or omit any components in the embodiments within the scope of the invention. INDUSTRIAL APPLICABILITY

[0090] The noise elimination device according to the present invention is a device that is used in an environment where noise is generated besides a target sound, and can be applied to a recording device, a calling device, or a speech recognition device to detect only the target sound. LIST OF REFERENCE SYMBOLS 100 noise elimination device, 101 Observation Signal Acquisition Unit, 102 vector storage unit, 103 Target noise vector selection unit, 104 Interference noise vector selection unit, and 105 Signal processing unit.

Claims

[1] Noise elimination device, comprising: a target sound vector selection unit (103) for selecting, from control vectors acquired in advance and indicating directions of arrival of sounds with respect to a sensor array (200) comprising two or more acoustic sensors (200a, 200b, 200c), a target sound control vector indicating an arrival direction of a target sound; an interference noise vector selecting unit (104) for selecting, from the pre-acquired control vectors, an interference noise control vector indicating an arrival direction of interference noise other than the target noise; and a signal processing unit (105) for acquiring, based on two or more observation signals obtained from the sensor array, the target noise control vector selected by the target noise vector selection unit and the interference noise control vector selected by the interference noise vector selection unit, a signal obtained by eliminating the interference noise from the observation signals, wherein the signal processing unit acquires a signal obtained by eliminating the interference noise from the observation signals by time-frequency masking using a mask for blocking a time-frequency spectrum of the interference noise, and wherein in time-frequency masking a control vector is estimated for each time-frequency from the two or more observation signals, and a similarity between a control vector of the estimated observation signal and the target noise control vector and the interference noise control vector is calculated, and if the control vector, which has the maximum calculated similarity is the target noise control vector, a time-frequency spectrum of the observation signal is passed, and if the control vector having the maximum calculated similarity is not the target noise control vector, a time-frequency spectrum of the observation signal is blocked, where the mask is given by B(ω,τ)={1(‖atrg(ω)−a^(ω,τ)‖<‖adst(ω)−a^(ω,τ)‖)0(otherwise) where B(ω, τ) denotes the mask, ω represents a discrete frequency, τ represents a discrete frame number, a trg (ω) denotes the target noise control vector, â(ω, τ) denotes the estimated control vector of the observation signal, and a dst(ω) denotes the interference noise control vector. [2] The noise elimination device according to claim 1, wherein the signal processing unit acquires, by linear beamforming having a linear filter coefficient with the arrival direction of the target noise as a directivity forming direction and the arrival direction of the interference noise as a blind spot forming direction, a signal obtained by eliminating the interference noise from the observation signals. [3] A noise elimination device according to claim 1, further comprising a vector storage unit (102) for storing the control vectors acquired in advance and indicating the arrival directions of the noises. [4] The noise elimination device according to claim 1, wherein the control vectors acquired in advance and indicating the arrival directions of the noises are control vectors indicating arrival directions of noises from positions estimated to be occupied by users to the sensor array. [5] A noise elimination device according to claim 4, wherein the signal processing unit extracts or eliminates speech of users occupying the positions estimated to be occupied from the observation signals. [6] A noise elimination device according to claim 1, wherein the control vectors acquired in advance and indicating the arrival directions of the noises are control vectors indicating arrival directions of noises from a driver's seat (601) and a passenger's seat (602) in a vehicle to the sensor array. [7] A noise elimination device according to claim 6, wherein the signal processing unit extracts or eliminates speech of a user sitting in the driver's seat or the passenger's seat from the observation signals. [8] Noise elimination method comprising: Selecting, by a target sound vector selecting unit, from control vectors acquired in advance and indicating arrival directions of sounds with respect to a sensor array comprising two or more acoustic sensors, a target sound control vector indicating an arrival direction of target sound; selecting, by an interference noise vector selecting unit, from the pre-acquired control vectors, an interference noise control vector indicating an arrival direction of interference noise other than the target noise; and Acquiring, by a signal processing unit, based on two or more observation signals obtained from the sensor array, the selected target noise control vector, and the selected interference noise control vector, a signal obtained by eliminating the interference noise from the observation signals, wherein the signal processing unit acquires a signal obtained by eliminating the interference noise from the observation signals by time-frequency masking using a mask for blocking a time-frequency spectrum of the interference noise, and wherein in time-frequency masking a control vector is estimated for each time-frequency from the two or more observation signals, and a similarity between a control vector of the estimated observation signal and the target noise control vector and the interference noise control vector is calculated, and if the control vector, which has the maximum calculated similarity is the target noise control vector, a time-frequency spectrum of the observation signal is passed, and if the control vector having the maximum calculated similarity is not the target noise control vector, a time-frequency spectrum of the observation signal is blocked, where the mask is given by B(ω,τ)={1(‖atrg(ω)−a^(ω,τ)‖<‖adst(ω)−a^(ω,τ)‖)0(otherwise) where B(ω, τ) denotes the mask, ω represents a discrete frequency, τ represents a discrete frame number, a trg (ω) denotes the target noise control vector, â(ω, τ) denotes the estimated control vector of the observation signal, and a dst(ω) denotes the interference noise control vector.

Images