Noise processing circuit, signal processing apparatus, noise processing method, and recording medium

Abstract

A noise processing circuit according to one embodiment of the present invention is provided with: a detection unit that detects a sound source on the basis of a plurality of signals; a flag signal generation unit that generates a flag signal indicating a period in which the plurality of signals include the first signal component; a beamforming processing unit that generates a first signal by emphasizing the first signal component and generates a second signal by suppressing the first signal component on the basis of the plurality of signals and the flag signal; a noise model generation unit that generates a plurality of noise models corresponding to the spectrum of the first signal on the basis of the flag signal; a noise model selection unit that selects one of the first signal and the second signal on the basis of the flag signal, and selects one of the plurality of noise models by calculating the degree of similarity between the frequency spectrum of the selected signal and each of the plurality of noise models; and a spectrum subtraction unit that performs spectrum subtraction processing on the basis of the spectrum of the first signal and the selected noise model.

Description

Technical Field

[0001] The present invention relates to a noise processing circuit, a signal processing device, and a noise processing method for performing noise processing based on a plurality of signals obtained by a plurality of microphones, and a recording medium storing software capable of performing noise processing based on a plurality of signals obtained by a plurality of microphones. Background Art

[0002] Some signal processing devices perform processing based on a plurality of signals obtained by a plurality of microphones. In such a signal processing device, for example, noise processing is performed based on these signals, and predetermined processing is performed based on the signals after the noise processing. For example, in Patent Document 1, a technique for reducing the wind noise component of non-stationary noise without degrading the quality of an audio signal is disclosed. [Prior Art Documents] [Patent Documents]

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2014-126856 Summary of the Invention

[0004] In the noise processing of a plurality of signals obtained by a plurality of microphones as described above, it is desired to reduce non-stationary noise without degrading the quality of a desired signal component, and it is expected to effectively reduce non-stationary noise.

[0005] It is desired to provide a noise processing circuit, a signal processing device, a noise processing method, and a recording medium capable of effectively reducing non-stationary noise.

[0006] A noise processing circuit according to an embodiment of the present invention includes a detection unit, a flag signal generation unit, a beamforming processing unit, a noise model generation unit, a noise model selection unit, and a spectral subtraction unit. The detection unit detects a sound source based on a plurality of signals supplied from a plurality of microphones. The flag signal generation unit generates a flag signal based on the detection result of the detection unit, the flag signal indicating a period during which the plurality of signals include a first signal component. The beamforming processing unit generates a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component based on the plurality of signals and the flag signal, and generates a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component. The noise model generation unit generates a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component based on the flag signal. The noise model selection unit selects one of the first signal and the second signal based on the flag signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models. The spectral subtraction unit performs spectral subtraction processing based on the spectrum of the first signal and the noise model selected by the noise model selection unit.

[0007] A signal processing apparatus according to an embodiment of the present invention includes a noise processing circuit and a processing circuit. The noise processing circuit performs noise processing based on a plurality of signals supplied from a plurality of microphones. The processing circuit performs signal processing based on the processing result of the noise processing circuit. The noise processing circuit includes a detection unit, a flag signal generation unit, a beamforming processing unit, a noise model generation unit, a noise model selection unit, and a spectral subtraction unit. The detection unit detects a sound source based on the plurality of signals supplied from the plurality of microphones. The flag signal generation unit generates a flag signal based on the detection result of the detection unit, the flag signal indicating a period during which the plurality of signals include a first signal component. The beamforming processing unit generates a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component, based on the plurality of signals and the flag signal, and generates a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component. The noise model generation unit generates a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component, based on the flag signal. The noise model selection unit selects one of the first signal and the second signal based on the flag signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models. The spectral subtraction unit performs spectral subtraction processing based on the spectrum of the first signal and the noise model selected by the noise model selection unit.

[0008] A noise processing method according to an embodiment of the present invention includes: detecting a sound source based on a plurality of signals supplied from a plurality of microphones; generating a flag signal based on the detection result of the sound source, the flag signal indicating a period during which the plurality of signals include a first signal component; generating a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component, based on the plurality of signals and the flag signal; generating a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component, based on the plurality of signals and the flag signal; generating a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component, based on the flag signal; selecting one of the first signal and the second signal based on the flag signal, and selecting one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and performing spectral subtraction processing based on the spectrum of the first signal and the selected noise model among the plurality of noise models.

[0009] A recording medium according to an embodiment of the present invention records software that causes a processor to perform processing. The above processing includes: detecting a sound source based on a plurality of signals supplied from a plurality of microphones; generating a flag signal based on the detection result of the sound source, the flag signal indicating a period during which the plurality of signals include a first signal component; generating a first signal based on the plurality of signals and the flag signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component; generating a second signal based on the plurality of signals and the flag signal by suppressing the first signal component during a period when the plurality of signals include the first signal component; generating a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component based on the flag signal; selecting one of the first signal and the second signal based on the flag signal, and selecting one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and performing a spectral subtraction process based on the spectrum of the first signal and the selected noise model among the plurality of noise models.

[0010] According to a noise processing circuit, a signal processing device, a noise processing method, and a recording medium according to an embodiment of the present invention, non-stationary noise can be effectively reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a block diagram showing a structural example of a signal processing device according to an embodiment of the present invention. Figure 2 It shows Figure 1 An explanatory diagram showing an operation example of the noise processing circuit shown. Figure 3 It shows Figure 1 A waveform diagram showing an operation example of the sound source detection unit and the sound source selection unit shown. Figure 4 It shows Figure 1 A block diagram showing a structural example of the beamforming processing unit shown. Figure 5 It shows Figure 1 An explanatory diagram showing an operation example of the Fourier transform unit shown. Figure 6 It shows Figure 1 An explanatory diagram showing an operation example of the noise model generation unit shown. Figure 7 It shows Figure 1 An explanatory diagram showing an operation example of the noise model selection unit shown. Figure 8 It is a block diagram showing a structural example of a beamforming processing unit of a modified example. Figure 9 It is a block diagram showing a structural example of a beamforming processing unit of another modified example. Figure 10 It is a block diagram showing a structural example of a beamforming processing unit representing other modification examples. Figure 11 It is a block diagram showing a structural example of a beamforming processing unit representing other modification examples. Detailed implementation mode

[0012] Hereinafter, the implementation mode of the present invention will be described in detail with reference to the accompanying drawings.

[0013] <Implementation mode> [Structural example] Figure 1 It shows a structural example of a signal processing device 1 including a noise processing circuit according to an implementation mode of the present invention. In this example, the signal processing device 1 is configured to: perform noise processing on four signals supplied from four microphones, thereby generating a signal with reduced noise, and perform predetermined signal processing based on this signal. The signal processing device 1 includes microphones 91 to 94, AD (Analog to Digital) conversion circuits 11 to 14, a user interface 18, a noise processing circuit 20, and a processing circuit 19.

[0014] Each of the microphones 91 to 94 is configured to: convert sound waves into electrical signals. The microphones 91 to 94 are, for example, arranged separately from each other. Thus, each of the microphones 91 to 94 can detect sound waves from mutually different directions.

[0015] The AD conversion circuit 11 is configured to: perform AD conversion on the electrical signal supplied from the microphone 91, thereby generating a signal S11. The AD conversion circuit 11 performs AD conversion at a sampling frequency fs, sequentially generates data x1, and outputs these data x1 as the signal S11. Figure 1 The data x1(n) shown represents the nth data x1. The sampling frequency fs is, for example, 16 kHz.

[0016] Similarly, the AD conversion circuit 12 is configured to: perform AD conversion on the electrical signal supplied from the microphone 92, thereby generating a signal S12. The AD conversion circuit 12 performs AD conversion at a sampling frequency fs, sequentially generates data x2, and outputs these data x2 as the signal S12. Figure 1 The data x2(n) shown represents the nth data x2. The AD conversion circuit 13 is configured to: perform AD conversion on the electrical signal supplied from the microphone 93, thereby generating a signal S13. The AD conversion circuit 13 performs AD conversion at a sampling frequency fs, sequentially generates data x3, and outputs these data x3 as the signal S13. Figure 1The data x3(n) shown represents the n-th data x3. The AD conversion circuit 14 is configured to perform AD conversion based on the electrical signal supplied from the microphone 94, thereby generating the signal S14. The AD conversion circuit 14 performs AD conversion at the sampling frequency fs, sequentially generates the data x4, and outputs these data x4 as the signal S14. Figure 1 The data x4(n) shown represents the n-th data x4. The AD conversion circuits 11 to 14 perform AD conversion in synchronization with each other.

[0017] The user interface 18 is configured to present information to the user of the signal processing device 1 and accept user operations. The user interface 18 includes, for example, a display panel, an indicator, and operation buttons. By operating the user interface 18, the user can perform various settings of the signal processing device 1.

[0018] The noise processing circuit 20 is configured to perform noise processing based on the signals S11 to S14 supplied from the AD conversion circuits 11 to 14, thereby generating the noise-reduced signal S29. The noise processing circuit 20 performs noise processing using the spectral subtraction method. The noise processing circuit 20 is constituted by, for example, a processor, a memory, etc., and operates by executing software.

[0019] Figure 2 Schematically shows an operation example of the noise processing circuit 20. (A) represents the signal before the noise processing is performed, and (B) represents the signal after the noise processing is performed. The waveform W1 shown by the thin line represents the noise component, and the waveform W2 shown by the thick line represents the desired signal component such as sound. Before the noise processing is performed, as Figure 2 (A) shows, the noise component (waveform W1) is large, and the desired signal component (waveform W2) is buried in the noise component. In particular, in this example, the noise component includes not only stationary noise but also non-stationary noise. During Figure 2 this period, non-stationary noise is generated during the periods T1 and T2. After the noise processing is performed, as Figure 2 (B) shows, the noise component (waveform W1) is reduced, and the desired signal component (waveform W2) is substantially maintained. In this way, the noise processing circuit 20 substantially maintains the desired signal component and can reduce the noise component including stationary noise and non-stationary noise.

[0020] The noise processing circuit 20 ( Figure 1 ) includes a sound source detection unit 21, a sound source selection unit 22, a beamforming processing unit 30, Fourier transform units 24, 25, a noise model generation unit 26, a noise model selection unit 27, a spectrum subtraction unit 28, and an inverse Fourier transform unit 29.

[0021] The sound source detection unit 21 is configured to detect the type of the sound source based on the signals S11 to S14 supplied from the AD conversion circuits 11 to 14.

[0022] Figure 3 This shows an example of the operation of the sound source detection unit 21 and the sound source selection unit 22. The sound source detection unit 21 detects the signal components of which sound sources are included in the signals S11 to S14, and generates metadata indicating the types of sound sources. In this example, "V" represents the human voice, "M" represents music, and "C" represents the running sound of a vehicle. For example, the signals S11 to S14 include signal components of the running sound of a vehicle during, for example, times t10 to t12; include signal components of the human voice during times t11 to t14; and include signal components of music during times t13 to t15.

[0023] The sound source detection unit 21 performs sound source detection, for example, based on a predetermined number (for example, 512) of data x1 included in the signal S11, a predetermined number (for example, 512) of data x2 included in the signal S12, a predetermined number (for example, 512) of data x3 included in the signal S13, and a predetermined number (for example, 512) of data x4 included in the signal S14. In this sound source detection, the sound source detection unit 21 detects the sound source represented by the signal component based on the signal components of each sound source in the signals S11 to S14 whose S / N ratio is equal to or higher than a predetermined value. Specifically, in Figure 3 this example, during times t10 to t12, if the S / N ratio of the signal component of the running sound of the vehicle is equal to or higher than the predetermined value, then metadata indicating the running sound of the vehicle is generated; during times t11 to t14, if the S / N ratio of the signal component of the human voice is equal to or higher than the predetermined value, then metadata indicating the human voice is generated; during times t13 to t15, if the S / N ratio of the signal component of music is equal to or higher than the predetermined value, then metadata indicating music is generated. In addition, the sound source detection unit 21 provides this metadata to the user interface 18 and the sound source selection unit 22.

[0024] The user interface 18 ( Figure 1)Based on the metadata supplied from the sound source detection unit 21, information regarding the type of the sound source is presented to the user. For example, the user knows that the signal processing device 1 has detected human voices, music, the running sound of a vehicle, etc. Additionally, the user performs a selection operation to choose which sound source signal component among these sound sources causes the noise processing circuit 20 to operate. The noise processing circuit 20 performs processing according to the user's selection operation. For example, when the user performs a selection operation to choose a human voice, the noise processing circuit 20 processes the human voice as the target sound source. In this case, for example, music becomes a noise component. For example, when the user performs a selection operation to choose music, the noise processing circuit 20 processes the music as the target sound source. In this case, for example, music becomes a noise component. The user interface 18 accepts such a selection operation by the user. Additionally, the user interface 18 provides information regarding such a selection operation by the user to the sound source selection unit 22.

[0025] The sound source selection unit 22 is configured to generate a flag signal CTL based on the metadata supplied from the sound source detection unit 21 and the information regarding the user's selection operation supplied from the user interface 18. This flag signal CTL is a signal that becomes active during the period when the signal components of the sound source selected by the user are included in the signals S11 to S14 and becomes inactive during other periods.

[0026] In Figure 3 the example of, for example, when the user performs a selection operation to choose a human voice, the sound source selection unit 22 makes the flag signal CTL active (high level in this example) during the time periods t11 to t14 when the signal components of the human voice are included; during other periods, the sound source selection unit 22 makes the flag signal CTL inactive (low level in this example). The sound source selection unit 22 generates the flag signal CTL in this way and provides the generated flag signal CTL to the beamforming processing unit 30, the noise model generation unit 26, and the noise model selection unit 27.

[0027] The beamforming processing unit 30 is configured to generate a sound source signal S38 and a suspected noise signal S48 based on the signals S11 to S14 and the flag signal CTL. The sound source signal S38 is a signal in which the signal components of the sound source selected by the user are emphasized. The suspected noise signal S48 is a signal in which the signal components of the sound source selected by the user are suppressed and includes signal components corresponding to the noise components included in the signals S11 to S14.

[0028] Figure 4 Shows a structural example of the beamforming processing unit 30. The beamforming processing unit 30 includes: delay units 31 to 34, a delay control unit 35, an addition unit 36, a delay unit 37, subtraction units 41 to 43, adaptive filters 44 to 46, an adaptive filter control unit 47, an addition unit 48, and a subtraction unit 38.

[0029] The delay unit 31 is configured to generate a signal S31 by delaying only the signal S11 by a delay amount d1. Specifically, the delay unit 31 moves the phase of the signal S11 in units of the sampling period Ts (= 1 / fs) to delay only the signal S11 by the delay amount d1. Figure 4 The shown data x1(n - d1) is the data of the data x1(n) delayed only by the delay amount d1. The delay amount d1 of the delay unit 31 is set by the delay control unit 35.

[0030] Similarly, the delay unit 32 is configured to generate a signal S32 by delaying only the signal S12 by a delay amount d2. Specifically, the delay unit 32 moves the phase of the signal S12 in units of the sampling period Ts (= 1 / fs) to delay only the signal S12 by the delay amount d2. Figure 4 The shown data x2(n - d2) is the data of the data x2(n) delayed only by the delay amount d2. The delay amount d2 of the delay unit 32 is set by the delay control unit 35. The delay unit 33 is configured to generate a signal S33 by delaying only the signal S13 by a delay amount d3. Specifically, the delay unit 33 moves the phase of the signal S13 in units of the sampling period Ts (= 1 / fs) to delay only the signal S13 by the delay amount d3. Figure 4 The shown data x3(n - d3) is the data of the data x3(n) delayed only by the delay amount d3. The delay amount d3 of the delay unit 33 is set by the delay control unit 35. The delay unit 34 is configured to generate a signal S34 by delaying only the signal S14 by a delay amount d4. Specifically, the delay unit 34 moves the phase of the signal S14 in units of the sampling period Ts (= 1 / fs) to delay only the signal S14 by the delay amount d4. Figure 4 The shown data x4(n - d4) is the data of the data x4(n) delayed only by the delay amount d4. The delay amount d4 of the delay unit 34 is set by the delay control unit 35.

[0031] The delay control unit 35 is configured to generate the delay amounts d1 to d4 of the delay units 31 to 34 respectively according to the flag signal CTL and the signals S11 to S14 so that the phases of the signal components of the user-selected sound source included in the signals S11 to S14 are consistent. Specifically, the delay control unit 35 updates the delay amounts d1 to d4 during the period when the flag signal CTL is at a high level (activated) so that the phases of the signal components of the user-selected sound source are consistent. The delay units 31 to 34 delay the signals S11 to S14 respectively by using the updated delay amounts d1 to d4 to generate the signals S31 to S34 respectively. Thus, the phases of the signal components of the user-selected sound source of the signals S31 to S34 are controlled to be consistent with each other.

[0032] That is to say, microphones 91 to 94 respectively detect sound waves from different directions. Therefore, corresponding to the direction of the sound source seen from the signal processing device 1, the phases of the signal components of the sound source selected by the user included in signals S11 to S14 may deviate from each other. In addition, some of the microphones 91 to 94 can directly detect the sound waves from the sound source, and other microphones can detect the sound waves reflected from an object. Therefore, the phases of the signal components of the sound source selected by the user included in signals S11 to S14 may deviate from each other. In addition, due to the characteristic differences of the microphones 91 to 94 and the characteristic differences of the AD conversion circuits 11 to 14, the phases of the signal components of the sound source selected by the user included in signals S11 to S14 may also deviate from each other. The delay control unit 35 generates delay amounts d1 to d4 based on the flag signal CTL and signals S11 to S14 so that the phases of the signal components of the sound source (target sound source) selected by the user included in signals S11 to S14 are consistent. Thereby, the delay control unit 35 adjusts the difference in the direction of the sound source and also adjusts the characteristic differences of the microphones 91 to 94.

[0033] In addition, during the period when the flag signal CTL is at a low level (inactive), the delay control unit 35 does not update the delay amounts d1 to d4 and maintains the delay amounts d1 to d4. Thereby, the delay units 31 to 34 respectively delay signals S11 to S14 using the maintained delay amounts d1 to d4. That is to say, during the period when the flag signal CTL is at a low level (inactive), since signals S11 to S14 do not include the signal components of the sound source selected by the user, the delay control unit 35 does not update the delay amounts d1 to d4. Therefore, the delay units 31 to 34 respectively delay signals S11 to S14 using the maintained delay amounts d1 to d4.

[0034] The addition unit 36 is configured to generate a signal S36 by adding signals S31 to S34 respectively supplied from the delay units 31 to 34. Specifically, the addition unit 36 generates data d(n) by adding data x1(n - d1), data x2(n - d2), data x3(n - d3), and data x4(n - d4). As described above, the phases of the signal components of the sound source selected by the user of signals S31 to S34 are consistent with each other. The addition unit 36 generates a signal S36 in which the signal components of the sound source (target sound source) selected by the user are emphasized by adding these signals S31 to S34.

[0035] The delay unit 37 is configured to generate a signal S37 by delaying the signal S36 supplied from the addition unit 36 by only a delay amount ds. Specifically, the delay unit 37 delays the signal S36 by only a delay amount ds by shifting the phase of the signal S36 in units of the sampling period Ts(=1 / fs). Figure 4The data d(n - ds) shown is the data of data d(n) delayed only by the delay amount ds. The delay amount ds of the delay unit 37 is a set value set by a control unit (not shown).

[0036] The subtraction unit 41 is configured to generate a signal S41 by subtracting the signal S32 supplied from the delay unit 32 from the signal S31 supplied from the delay unit 31. Specifically, the subtraction unit 41 generates data y1(n) by subtracting the data x2(n - d2) from the data x1(n - d1). As described above, the phases of the signal components of the user-selected sound source of the signals S31 and S32 are consistent with each other. The subtraction unit 41 generates a signal S41 in which the signal component of the user-selected sound source is suppressed by subtracting the signal S32 from the signal S31.

[0037] Similarly, the subtraction unit 42 is configured to generate a signal S42 by subtracting the signal S33 supplied from the delay unit 33 from the signal S32 supplied from the delay unit 32. Specifically, the subtraction unit 42 generates data y2(n) by subtracting the data x3(n - d3) from the data x2(n - d2). As described above, the phases of the signal components of the user-selected sound source of the signals S32 and S33 are consistent with each other. The subtraction unit 42 generates a signal S42 in which the signal component of the user-selected sound source is suppressed by subtracting the signal S33 from the signal S32. The subtraction unit 43 is configured to generate a signal S43 by subtracting the signal S34 supplied from the delay unit 34 from the signal S33 supplied from the delay unit 33. Specifically, the subtraction unit 43 generates data y3(n) by subtracting the data x4(n - d4) from the data x3(n - d3). As described above, the phases of the signal components of the user-selected sound source of the signals S33 and S34 are consistent with each other. The subtraction unit 43 generates a signal S43 in which the signal component of the user-selected sound source is suppressed by subtracting the signal S34 from the signal S33.

[0038] The adaptive filter 44 is configured to generate a signal S44 by performing a filtering process on the signal S41 supplied from the subtraction unit 41. The adaptive filter 44 is a FIR (Finite Impulse Response) filter, and adjusts the amplitude and phase of the signal S41 by performing a convolution operation on the signal S41 using the filter coefficients supplied from the adaptive filter control unit 47.

[0039] Similarly, the adaptive filter 45 is configured to generate a signal S45 by filtering the signal S42 supplied from the subtraction unit 42. The adaptive filter 45 is a FIR filter, and adjusts the amplitude and phase of the signal S42 by performing a convolution operation on the signal S42 using the filter coefficients supplied from the adaptive filter control unit 47. The adaptive filter 46 is configured to generate a signal S46 by filtering the signal S43 supplied from the subtraction unit 43. The adaptive filter 46 is a FIR filter, and adjusts the amplitude and phase of the signal S43 by performing a convolution operation on the signal S43 using the filter coefficients supplied from the adaptive filter control unit 47.

[0040] The adaptive filter control unit 47 is configured to generate filter coefficients supplied to the adaptive filter 44, filter coefficients supplied to the adaptive filter 45, and filter coefficients supplied to the adaptive filter 46 based on the flag signal CTL and the sound source signal S38 supplied from the subtraction unit 38, so as to reduce the noise of the sound source signal S38. Specifically, during the period when the flag signal CTL is at a low level (inactive), the adaptive filter control unit 47 updates the filter coefficients supplied to the adaptive filters 44 to 46 respectively, so as to reduce the noise of the sound source signal S38. The adaptive filters 44 to 46 perform filtering processes using the updated filter coefficients respectively.

[0041] In addition, during the period when the flag signal CTL is at a high level (active), the adaptive filter control unit 47 does not update the filter coefficients and maintains the filter coefficients. Thus, the adaptive filters 44 to 46 perform filtering processes using the maintained filter coefficients respectively.

[0042] The addition unit 48 is configured to generate a pseudo-noise signal S48 by adding the signals S44 to S46 supplied from the adaptive filters 44 to 46. Specifically, the addition unit 48 generates data y(n) by adding the data supplied from the adaptive filters 44 to 46. The data y(n) represents the n-th data y.

[0043] The subtraction unit 38 is configured to generate a sound source signal S38 by subtracting the pseudo-noise signal S48 supplied from the addition unit 48 from the signal S37 supplied from the delay unit 37. Specifically, the subtraction unit 38 generates data e(n) by subtracting the data y(n) from the data d(n - ds). The data e(n) represents the n-th data e.

[0044] With this configuration, the adaptive filters 44 to 46, the adaptive filter control unit 47, the addition unit 48, and the subtraction unit 38 perform a negative feedback operation in such a way that the noise of the sound source signal S38 is reduced. In other words, the adaptive filters 44 to 46, the adaptive filter control unit 47, and the addition unit 48 perform a negative feedback operation in such a way that the pseudo-noise signal S48 is the same as the noise component included in the signal S37 supplied to the subtraction unit 38.

[0045] In this way, in the beamforming processing unit 30, the signals S31 to S34 in which the phases of the signal components of the sound source (target sound source) selected by the user are made consistent with each other are added by the addition unit 36, and the signal components of the user-selected sound source of the sound source signal S38 are emphasized. In addition, the pseudo-noise signal S48 is subtracted from the signal S37 by the subtraction unit 38, and the signal components of the user-selected sound source of the sound source signal S38 are emphasized. In this way, the beamforming processing unit 30 generates a sound source signal S38 in which the signal components of the user-selected sound source are emphasized.

[0046] Furthermore, the beamforming processing unit 30 excludes, for example, a signal that does not include the signal of the target sound source from the processing targets of the beamforming processing unit 30 when only a part of the signals S11 to S14 does not include the signal of the target sound source. For example, when the signal S14 does not include the signal of the target sound source, the addition unit 36 generates a signal S36 by adding the signals S31 to S33. In short, the addition unit 36 does not add the signal S34. In this way, the addition unit 36 adds the signals that include the signal of the target sound source among the signals S31 to S34. Similarly, the subtraction units 41 to 43 appropriately combine the signals that include the signal of the target sound source among the signals S31 to S34 and perform subtraction.

[0047] The Fourier transform unit 24 ( Figure 1 ) is configured to perform a discrete Fourier transform on the sound source signal S38 supplied from the beamforming processing unit 30, thereby calculating the amplitude spectrum data SD38 and the phase spectrum data SDP of the sound source signal S38.

[0048] Figure 5 FIG. shows an operation example of the Fourier transform unit 24. The Fourier transform unit 24, for example, performs a discrete Fourier transform on the basis of a predetermined number (for example, 512) of data e provided each time by the sound source signal S38, thereby generating the amplitude spectrum data SD38 and the phase spectrum data SDP of the sound source signal S38. Furthermore, in this Figure 5 the illustration of the phase spectrum data SDP is omitted. The horizontal axis of the amplitude spectrum data SD38 represents the frequency, and the vertical axis represents the magnitude of the amplitude. In this Figure 5During the processing period T, a predetermined number (e.g., 512) of data e are provided from the beamforming processing unit 30 to the Fourier transform unit 24. The Fourier transform unit 24 performs a discrete Fourier transform on the basis of the data e during each processing period T, thereby calculating the amplitude spectrum data SD38 and the phase spectrum data SDP of the sound source signal S38. In addition, the Fourier transform unit 24 provides the generated amplitude spectrum data SD38 to the noise model generation unit 26, the noise model selection unit 27, and the spectral subtraction unit 28, and provides the generated phase spectrum data SDP to the inverse Fourier transform unit 29.

[0049] The Fourier transform unit 25( Figure 1 ) is configured to perform a discrete Fourier transform on the basis of the pseudo-noise signal S48 supplied from the beamforming processing unit 30, thereby calculating the amplitude spectrum data SD48 of the pseudo-noise signal S48. Specifically, the Fourier transform unit 25 is the same as the Fourier transform unit 24. For example, each time a predetermined number (e.g., 512) of data y are provided by the pseudo-noise signal S48, a discrete Fourier transform is performed on the basis of the data, thereby generating the amplitude spectrum data SD48 of the pseudo-noise signal S48. In addition, the Fourier transform unit 25 provides the generated amplitude spectrum data SD48 to the noise model selection unit 27.

[0050] The noise model generation unit 26 is configured to generate a noise model NM on the basis of the flag signal CTL and the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24. The noise model NM represents the amplitude spectrum data of noise. The noise model generation unit 26 generates the noise model NM during a period in which the flag signal CTL is at a low level (inactive). That is, during a period in which the flag signal CTL is at a low level (inactive), since the sound source signal S38 does not include a signal component of a sound source (target sound source) selected by a user, the noise model generation unit 26 generates the noise model NM on the basis of the amplitude spectrum data SD38 of the sound source signal S38 during this period. In addition, the noise model generation unit 26 provides the generated noise model NM to the noise model selection unit 27.

[0051] The noise model selection unit 27 is configured to store the noise model NM supplied from the noise model generation unit 26, and select one of the stored multiple noise models NM according to the flag signal CTL, the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24, and the amplitude spectrum data SD48 of the pseudo-noise signal S48 supplied from the Fourier transform unit 25. Specifically, during the period when the flag signal CTL is at a low level (inactive), the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD38 of the sound source signal S38 and each of the multiple noise models NM, and selects one of the multiple noise models NM. That is, during the period when the flag signal CTL is at a low level (inactive), since the sound source signal S38 does not contain the signal component of the sound source (target sound source) selected by the user, the noise model selection unit 27 uses the amplitude spectrum data SD38 of this sound source signal S38 to select one of the multiple noise models NM during this period. The similarity can use, for example, cosine similarity. The noise model selection unit 27, for example, selects the noise model NM with the highest similarity among the multiple noise models NM. In addition, during the period when the flag signal CTL is at a high level (active), the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD48 of the pseudo-noise signal S48 and each of the multiple noise models NM, and selects one of the multiple noise models NM. That is, during the period when the flag signal CTL is at a high level (active), since the sound source signal S38 contains the signal component of the sound source (target sound source) selected by the user, the noise model selection unit 27 uses the amplitude spectrum data SD48 of the pseudo-noise signal S48 to select one of the multiple noise models NM during this period. The noise model selection unit 27, for example, selects the noise model NM with the highest similarity among the multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0052] The spectral subtraction unit 28 is configured to perform spectral subtraction processing of subtracting the amplitude spectrum data of the noise model NM supplied from the noise model selection unit 27 from the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24. In addition, the spectral subtraction unit 28 provides the amplitude spectrum data obtained through the spectral subtraction processing to the inverse Fourier transform unit 29.

[0053] The inverse Fourier transform unit 29 is configured to perform the inverse transform of the discrete Fourier transform according to the amplitude spectrum data supplied from the spectral subtraction unit 28 and the phase spectrum data SDP supplied from the Fourier transform unit 24, thereby generating the signal S29.

[0054] In this way, the noise processing circuit 20 generates the time-domain signal S29 with reduced noise. In addition, the noise processing circuit 20 provides this signal S29 to the subsequent processing circuit 19.

[0055] The processing circuit 19 is configured to perform a predetermined signal process based on the signal S29.

[0056] Here, the sound source detection unit 21 corresponds to a specific example of the "detection unit" of the present disclosure. The microphones 91 to 94 correspond to a specific example of the "plurality of microphones" of the present disclosure. The signals S11 to S14 correspond to a specific example of the "plurality of signals" of the present disclosure. The sound source selection unit 22 corresponds to a specific example of the "flag signal generation unit" of the present disclosure. The beamforming processing unit 30 corresponds to a specific example of the "beamforming processing unit" of the present disclosure. The noise model generation unit 26 corresponds to a specific example of the "noise model generation unit" of the present disclosure. The noise model selection unit 27 corresponds to a specific example of the "noise model selection unit" of the present disclosure. The spectral subtraction unit 28 corresponds to a specific example of the "spectral subtraction unit" of the present disclosure. The user interface 18 corresponds to a specific example of the "user interface" of the present disclosure. The processing circuit 19 corresponds to a specific example of the "processing circuit" of the present disclosure.

[0057] [Operation and Function] Next, the operation and function of the signal processing apparatus 1 of the present embodiment will be described.

[0058] [Overall Operation Outline] First, with reference to Figure 1 , the overall operation outline of the signal processing apparatus 1 will be described. Each of the microphones 91 to 94 converts a sound wave into an electric signal. The AD conversion circuits 11 to 14 perform AD conversion based on the electric signals supplied from the microphones 91 to 94, thereby generating the signals S11 to S14, respectively. The user interface 18 presents information to the user of the signal processing apparatus 1 and accepts user operations. The noise processing circuit 20 performs noise processing based on the signals S11 to S14, thereby generating the signal S29 with reduced noise.

[0059] The sound source detection unit 21 of the noise processing circuit 20 detects the type of the sound source based on the signals S11 to S14, and generates metadata indicating the type of the sound source. The sound source selection unit 22 generates a flag signal CTL based on the metadata supplied from the sound source detection unit 21 and the information about the user's selection operation supplied from the user interface 18. The flag signal CTL becomes active during the period including the signal component of the sound source selected by the user and becomes inactive during other periods. The beamforming processing unit 30 generates a sound source signal S38 and a suspected noise signal S48 based on the signals S11 to S14 and the flag signal CTL. The Fourier transform unit 24 performs a discrete Fourier transform on the sound source signal S38, thereby calculating the amplitude spectrum data SD38 and the phase spectrum data SDP of the sound source signal S38. The Fourier transform unit 25 performs a discrete Fourier transform on the suspected noise signal S48, thereby calculating the amplitude spectrum data SD48 of the suspected noise signal S48. The noise model generation unit 26 generates a noise model NM based on the flag signal CTL and the amplitude spectrum data SD38 of the sound source signal S38. The noise model selection unit 27 stores the noise model NM supplied from the noise model generation unit 26, and selects one of the multiple stored noise models NM based on the flag signal CTL, the amplitude spectrum data SD38 of the sound source signal S38, and the amplitude spectrum data SD48 of the suspected noise signal S48. The spectral subtraction unit 28 performs a spectral subtraction process of subtracting the amplitude spectrum data of the noise model NM supplied from the noise model selection unit 27 from the amplitude spectrum data SD38 of the sound source signal S38. The inverse Fourier transform unit 29 performs an inverse transform of the discrete Fourier transform based on the amplitude spectrum data supplied from the spectral subtraction unit 28 and the phase spectrum data SDP supplied from the Fourier transform unit 24, thereby generating a signal S29.

[0060] The processing circuit 19 performs a predetermined signal process based on the signal S29 generated by the noise processing circuit 20.

[0061] (Detailed operation) Next, the operations of the noise model generation unit 26 and the noise model selection unit 27 of the noise processing circuit 20 will be described in detail.

[0062] (Operation of the noise model generation unit 26) The noise model generation unit 26 generates a noise model NM based on the flag signal CTL and the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24.

[0063] Figure 6 An example of an operation of the noise model generation unit 26 is shown. In this Figure 6 example, the sound source signal S38 is depicted using an envelope.

[0064] In this example, at the moment t11, the sound source selection unit 22 changes the flag signal CTL from high level (activated) to low level (not activated). That is to say, during the period before the moment t11, the sound source signal S38 includes the signal component of the sound source selected by the user; during the period after the moment t11, it does not include the signal component of the sound source selected by the user.

[0065] During each processing period T, the Fourier transform unit 24 performs a discrete Fourier transform on the basis of a predetermined number (for example, 512) of data e included in the sound source signal S38, thereby calculating the amplitude spectrum data SD38 of the sound source signal S38.

[0066] During the period when the flag signal CTL is at low level (not activated), the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24.

[0067] Specifically, for example, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the period from t11 to t12. The noise model generation unit 26 can, for example, generate the noise model NM by multiplying the amplitude spectrum data SD38 by a coefficient. This coefficient is a so-called subtraction coefficient. Specifically, the noise model generation unit 26 can, for example, generate the noise model NM by multiplying the amplitude magnitude of each frequency in the amplitude spectrum data SD38 by, for example, "1.3". It is not limited to this. The noise model generation unit 26 can, for example, generate the noise model NM by multiplying the amplitude magnitude of each frequency in the amplitude spectrum data SD38 by, for example, "1.0", or can also generate the noise model NM by multiplying it by, for example, "0.7".

[0068] Next, in this example, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the period from t11 to t12 and the amplitude spectrum data SD38 obtained from the sound source signal S38 during the period from t12 to t13. The noise model generation unit 26 can, for example, generate the noise model NM by calculating the average value of these two amplitude spectrum data SD38. In addition, the noise model generation unit 26 can, for example, also generate the noise model NM by performing a weighted sum on the basis of these two amplitude spectrum data SD38 using a coefficient.

[0069] Next, in this example, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t11 to t12, the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t12 to t13, and the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t13 to t14. For example, the noise model generation unit 26 can generate the noise model NM by calculating the average value of these three amplitude spectrum data SD38. Additionally, for example, the noise model generation unit 26 can also generate the noise model NM by performing a weighted sum using coefficients based on these three amplitude spectrum data SD38.

[0070] In this example, during the time period from t14 to t15, the noise model generation unit 26 does not generate a noise model NM. That is, in this example, since the four amplitude spectrum data SD38 during the time period from t14 to t15 are each approximately the same as the amplitude spectrum data SD38 during the time period from t11 to t14, the noise model generation unit 26 does not generate a noise model NM. Furthermore, in this example, although a noise model NM is not generated during the time period from t14 to t15, it is not limited to this, and a noise model NM can also be generated.

[0071] Additionally, in this example, at time t15, the sound source signal S38 undergoes a significant change. Correspondingly, the amplitude spectrum data SD38 also undergoes a significant change.

[0072] The noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t15 to t16. That is, in this example, since the amplitude spectrum data SD38 during the time period from t15 to t16 has changed by more than a certain degree compared to, for example, the immediately preceding amplitude spectrum data SD38, the noise model generation unit 26 generates a noise model NM. For example, the noise model generation unit 26 can generate the noise model NM by multiplying the amplitude spectrum data SD38 by a coefficient. That is, in this example, since the amplitude spectrum data SD38 during the time period from t15 to t16 has a low correlation with the previous amplitude spectrum data SD38, instead of using the previous amplitude spectrum data SD38, the noise model NM is generated based on the amplitude spectrum data SD38 during the time period from t15 to t16.

[0073] Next, in this example, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t15 to t16 and the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t16 to t17.

[0074] Next, in this example, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t15 to t16, the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t16 to t17, and the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t17 to t18.

[0075] In this way, during the period when the flag signal CTL is at a low level (inactive), the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24.

[0076] Furthermore, in this example, although the noise model generation unit 26 generates a noise model NM immediately after changing the flag signal CTL from a high level (active) to a low level (inactive) and when the amplitude spectrum data SD38 changes significantly, it is not limited to this, and a noise model NM can also be generated in other cases. For example, the noise model generation unit 26 can also frequently generate a noise model NM when the flag signal CTL is at a low level (inactive).

[0077] In this way, the noise model generation unit 26 generates a noise model NM based on the amplitude spectrum data SD38 of the sound source signal S38 supplied from the Fourier transform unit 24. In addition, the noise model generation unit 26 sequentially provides the generated noise model NM to the noise model selection unit 27.

[0078] (Operation of the noise model selection unit 27) The noise model selection unit 27 stores the noise model NM supplied from the noise model generation unit 26, and selects one of the multiple stored noise models NM based on the flag signal CTL, the amplitude spectrum data SD38 of the sound source signal S38, and the amplitude spectrum data SD48 of the suspected noise signal S48.

[0079] Figure 7 Shows an example of an operation of the noise model selection unit 27. In Figure 7 The noise model list is generated by the noise model generation unit 26 and represents the multiple noise models NM stored in the noise model selection unit 27.

[0080] In this example, the sound source selection unit 22 changes the flag signal CTL from a low level (inactive) to a high level (active) at this time t23. That is, during the period before time t23, the sound source signal S38 does not contain the signal component of the user-selected sound source (target sound source); during the period after time t23, it contains the signal component of the user-selected sound source.

[0081] During each processing period T, the Fourier transform unit 24 performs a discrete Fourier transform on a predetermined number (e.g., 512) of data e included in the sound source signal S38, thereby calculating the amplitude spectrum data SD38 of the sound source signal S38. Similarly, during each processing period T, the Fourier transform unit 25 performs a discrete Fourier transform on a predetermined number (e.g., 512) of data e included in the suspected noise signal S48, thereby calculating the amplitude spectrum data SD48 of the suspected noise signal S48.

[0082] During the period when the flag signal CTL is at a low level (inactive), the noise model selection unit 27 selects one of the multiple noise models NM by calculating the similarity between the amplitude spectrum data SD38 of the sound source signal S38 and each of the stored multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0083] Specifically, for example, the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t21 to t22 and each of the multiple noise models NM. The noise model selection unit 27, for example, selects the noise model NM with the highest similarity among the multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0084] Next, the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD38 obtained from the sound source signal S38 during the time period from t22 to t23 and each of the multiple noise models NM. The noise model selection unit 27, for example, selects the noise model NM with the highest similarity among the multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0085] During the period when the flag signal CTL is at a high level (active), the noise model selection unit 27 selects one of the multiple noise models NM by calculating the similarity between the amplitude spectrum data SD48 of the suspected noise signal S48 and each of the multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0086] Specifically, for example, the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD48 obtained from the suspected noise signal S48 during the time period from t23 to t24 and each of the multiple noise models NM. The noise model selection unit 27, for example, selects the noise model NM with the highest similarity among the multiple noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0087] Next, the noise model selection unit 27 calculates the similarity between the amplitude spectrum data SD48 obtained from the pseudo-noise signal S48 during the time period from t24 to t25 and each of the plurality of noise models NM. The noise model selection unit 27 selects, for example, the noise model NM with the highest similarity among the plurality of noise models NM. In addition, the noise model selection unit 27 provides the selected noise model NM to the spectral subtraction unit 28.

[0088] The spectral subtraction unit 28 performs spectral subtraction processing of subtracting the amplitude spectrum data of the noise model NM supplied by the noise model selection unit 27 from the amplitude spectrum data SD38 of the sound source signal S38 supplied by the Fourier transform unit 24. In addition, the inverse Fourier transform unit 29 performs an inverse transform of the discrete Fourier transform based on the amplitude spectrum data supplied from the spectral subtraction unit 28 and the phase spectrum data SDP supplied from the Fourier transform unit 24, thereby generating the signal S29.

[0089] In this way, the noise processing circuit 20 generates the signal S29 in the time domain with reduced noise.

[0090] In this manner, in the noise processing circuit 20, a sound source detection unit 21, a sound source selection unit 22, a beamforming processing unit 30, a noise model generation unit 26, a noise model selection unit 27, and a spectral subtraction unit 28 are provided. The sound source detection unit 21 detects a sound source based on the four signals S11 to S14 supplied from the four microphones 91 to 94. The sound source selection unit 22 generates a flag signal CTL based on the detection result of the sound source detection unit 21, and the flag signal CTL indicates a period during which the four signals S11 to S14 include a first signal component (in this example, a signal component of the sound source selected by the user). The beamforming processing unit 30 generates a sound source signal S38 by emphasizing the first signal component during a period when the four signals S11 to S14 include the first signal component, and generates a pseudo-noise signal S48 by suppressing the first signal component during a period when the four signals S11 to S14 include the first signal component, based on the four signals S11 to S14 and the flag signal CTL. The noise model generation unit 26 generates a plurality of noise models NM corresponding to the spectrum of the sound source signal S38 during a period when the four signals S11 to S14 do not include the first signal component, based on the flag signal CTL. The noise model selection unit 27 selects one of the sound source signal S38 and the pseudo-noise signal S48 based on the flag signal CTL, and selects one of the plurality of noise models NM by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models NM. The spectral subtraction unit 28 performs spectral subtraction processing based on the spectrum of the sound source signal S38 and the noise model NM selected by the noise model selection unit 27. Thus, in the noise processing circuit 20, non-stationary noise can be effectively reduced.

[0091] That is to say, in general spectral subtraction, although stationary noise can be reduced, it is difficult to reduce non-stationary noise. Specifically, for example, when a noise processing circuit obtains a noise spectrum based on stationary noise and subtracts the noise spectrum from the spectrum of a sound source signal, the stationary noise included in the sound source signal can be reduced. However, when the sound source signal includes non-stationary noise, the non-stationary noise cannot be reduced, resulting in damage to the quality of the desired signal component. In addition, when the noise processing circuit obtains a noise spectrum based on non-stationary noise and subtracts the noise spectrum from the spectrum of the sound source signal, the non-stationary noise included in the sound source signal can be reduced. However, in this case, if the sound source signal does not include non-stationary noise, the quality of the desired signal component will be damaged.

[0092] On the other hand, in the noise processing circuit 20 of the present embodiment, a plurality of noise models NM are provided, and the similarity between the spectrum of one of the sound source signal S38 and the suspected noise signal S48 and each of the plurality of noise models NM is calculated. Thereby, one of the plurality of noise models NM is selected, and spectral subtraction processing is performed based on the spectrum of the sound source signal S38 and the noise model NM selected by the noise model selection unit 27. Therefore, for example, during a period when the sound source signal S38 does not include the signal component of the sound source selected by the user, a noise model NM similar to the spectrum of the sound source signal S38 is selected. In addition, for example, during a period when the sound source signal S38 includes the signal component of the sound source selected by the user, a noise model NM similar to the spectrum of the suspected noise signal S48 is selected. The suspected noise signal S48 is related to the noise signal included in the sound source signal S38. Therefore, regardless of whether the sound source signal S38 includes non-stationary noise, a noise model NM that can remove the noise included in the sound source signal S38 is selected from the plurality of noise models NM. Thereby, in the noise processing circuit 20, non-stationary noise can be reduced without damaging the quality of the desired signal component. As a result, non-stationary noise can be effectively reduced.

[0093] In addition, in the noise processing circuit 20, the sound source detection unit 21 generates a sequence of meta-information indicating the type of the sound source by detecting the sound source; the sound source selection unit 22 generates a flag signal CTL according to the sequence of meta-information. Thereby, since the noise processing circuit 20 can reduce the possibility of performing noise processing based on unintentional signal components, the accuracy of noise processing can be improved.

[0094] In addition, in the noise processing circuit 20, a sound source selection unit 22 generates a flag signal CTL based on the detection result of a sound source detection unit 21 and a sound source selection operation of a user accepted by a user interface 18. Thus, the user can select which sound source signal component to retain and reduce other signal components corresponding to the application program. Therefore, since the noise processing circuit 20 can perform noise processing based on the signal components expected by the user, the accuracy of noise processing can be improved.

[0095] In addition, in the noise processing circuit 20, during a period when the four signals S11 to S14 contain a first signal component (in this example, the signal component of the sound source selected by the user), a beamforming processing unit 30 adjusts a first processing setting of the beamforming processing unit 30 (in this example, the delay amounts d1 to d4 of the four delay units 31 to 34 and the filter coefficients of the three adaptive filters 44 to 46) according to the flag signal CTL to generate a sound source signal S38 in which the first signal component is emphasized; during a period when the four signals S11 to S14 do not contain the first signal component, the first processing setting is maintained, and the sound source signal S38 is generated using the first processing setting. Thus, for example, regardless of the direction of the sound source seen from the signal processing device 1 and the characteristic differences of the microphones 91 to 94, a sound source signal S38 in which the signal component of the sound source selected by the user is emphasized can be generated. Thus, in the noise processing circuit 20, the accuracy of noise processing can be improved.

[0096] In addition, in the noise processing circuit 20, during a period when the four signals S11 to S14 do not contain a first signal component (in this example, the signal component of the sound source selected by the user), a beamforming processing unit 30 adjusts a second processing setting of the beamforming processing unit 30 (in this example, the filter coefficients of the three adaptive filters 44 to 46) according to the flag signal CTL to suppress a second signal component (for example, a noise component) of the sound source signal S38 and generate a suspected noise signal S48 containing the second signal component; during a period when the four signals S11 to S14 contain the first signal component, the second processing setting is maintained, and the suspected noise signal S48 is generated using the second processing setting. Thus, for example, regardless of the direction of the sound source and the characteristic differences of the microphones 91 to 94, the signal component of the sound source selected by the user is suppressed, and a suspected noise signal S48 containing the noise component included in the signals S11 to S14 can be generated. Thus, in the noise processing circuit 20, the accuracy of noise processing can be improved.

[0097] In addition, in the noise processing circuit 20, when the spectrum of the sound source signal S38 changes by a certain degree or more, the noise model generation unit 26 generates a noise model NM corresponding to the spectrum after the change. Thus, in the noise processing circuit 20, even when new non-stationary noise occurs, the noise component of the non-stationary noise in the sound source signal S38 can be reduced, so that the non-stationary noise can be effectively reduced.

[0098] In addition, in the noise processing circuit 20, during the period when the four signals S11 to S14 do not include the first signal component (in this example, the signal component of the sound source selected by the user), the noise model selection unit 27 selects the sound source signal S38 from the sound source signal S38 and the suspected noise signal S48 according to the flag signal CTL, and selects one of the multiple noise models NM according to the sound source signal S38. Thus, in the noise processing circuit 20, the accuracy of noise processing can be improved. That is, during this period, since the sound source signal S38 does not include the signal component of the sound source selected by the user, the noise spectrum can be accurately obtained according to the sound source signal S38, and an appropriate one of the multiple noise models NM can be selected. As a result, in the noise processing circuit 20, the accuracy of noise processing can be improved.

[0099] In addition, in the noise processing circuit 20, when the four signals S11 to S14 include the first signal component (in this example, the signal component of the sound source selected by the user), the noise model selection unit 27 selects the suspected noise signal S48 from the sound source signal S38 and the suspected noise signal S48 according to the flag signal CTL, and selects one of the multiple noise models NM according to the suspected noise signal S48. Thus, in the noise processing circuit 20, the accuracy of noise processing can be improved. That is, during this period, since the sound source signal S38 includes the signal component of the sound source selected by the user, it is difficult to accurately obtain the noise spectrum according to the sound source signal S38. The signal component of the suspected noise signal S48 is related to the noise component included in the sound source signal S38. Therefore, by obtaining the noise spectrum according to the suspected noise signal S48, a more appropriate one of the multiple noise models NM can be selected. As a result, in the noise processing circuit 20, the accuracy of noise processing can be improved.

[0100] [Effect] As described above, in the present embodiment, a sound source detection unit, a sound source selection unit, a beamforming processing unit, a noise model generation unit, a noise model selection unit, and a spectral subtraction unit are provided. The sound source detection unit detects a sound source based on four signals supplied from four microphones. The sound source selection unit generates a flag signal based on the detection result of the sound source detection unit, and the flag signal indicates a period during which the four signals include a first signal component. The beamforming processing unit generates a sound source signal by emphasizing the first signal component during a period when the four signals include the first signal component based on the four signals and the flag signal, and generates a suspected noise signal by suppressing the first signal component during a period when the four signals include the first signal component. The noise model generation unit generates a plurality of noise models corresponding to the spectrum of the sound source signal during a period when the four signals do not include the first signal component based on the flag signal. The noise model selection unit selects one of the sound source signal and the suspected noise signal based on the flag signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models. The spectral subtraction unit performs spectral subtraction processing based on the spectrum of the sound source signal and the noise model selected by the noise model selection unit. Thus, non-stationary noise can be effectively reduced.

[0101] In the present embodiment, since the sound source detection unit generates a sequence of meta-information indicating the type of the sound source by detecting the sound source, and the sound source selection unit generates a flag signal based on the sequence of the meta-information, the accuracy of noise processing can be improved.

[0102] In the present embodiment, since the sound source selection unit generates a flag signal based on the detection result of the sound source detection unit and the sound source selection operation of the user accepted by the user interface, the accuracy of noise processing can be improved.

[0103] In the present embodiment, since the beamforming processing unit adjusts the first processing setting of the beamforming processing unit during a period when the four signals include the first signal component based on the flag signal to generate a sound source signal in which the first signal component is emphasized; and maintains the first processing setting during a period when the four signals do not include the first signal component and generates a sound source signal using the first processing setting, the accuracy of noise processing can be improved.

[0104] In the present embodiment, the beamforming processing unit adjusts the second processing setting of the beamforming processing unit during a period when the four signals do not include the first signal component based on the flag signal to suppress the second signal component of the sound source signal and generate a suspected noise signal including the second signal component; and maintains the second processing setting during a period when the four signals include the first signal component and generates a suspected noise signal using the second processing setting. Thus, the accuracy of noise processing can be improved.

[0105] In the present embodiment, since the noise model generation unit generates a noise model corresponding to the spectrum after the change when the spectrum of the sound source signal changes by a certain degree or more, non-stationary noise can be effectively reduced.

[0106] In the present embodiment, since the noise model selection unit selects the sound source signal from the sound source signal and the suspected noise signal during the period when the four signals do not include the first signal component according to the flag signal, and selects one of the multiple noise models according to the sound source signal, the accuracy of noise processing can be improved.

[0107] In the present embodiment, since the noise model selection unit selects the suspected noise signal from the sound source signal and the suspected noise signal when the four signals include the first signal component according to the flag signal, and selects one of the multiple noise models NM according to the suspected noise signal, the accuracy of noise processing can be improved.

[0108] [Modification Example 1] In the above embodiment, although the beamforming processing unit 30 has Figure 4 the structure shown, it is not limited thereto. Hereinafter, several examples will be given to illustrate this modification example.

[0109] Figure 8 FIG. shows a structural example of the beamforming processing unit 30A representing this modification example. The beamforming processing unit 30A includes delay units 31 to 34, a delay setting unit 35A, an addition unit 36, a delay unit 37, subtraction units 41 to 43, adaptive filters 44 to 46, an adaptive filter control unit 47, an addition unit 48, and a subtraction unit 38. That is, the beamforming processing unit 30A is obtained by replacing the delay control unit 35 in the beamforming processing unit 30 ( Figure 4 ) of the above embodiment with the delay setting unit 35A. The delay setting unit 35A is configured to set the delay amounts d1 to d4 of the delay units 31 to 34, respectively. In the delay setting unit 35A, the delay amounts d1 to d4 are preset according to the direction of the sound source seen from the signal processing device 1 and the characteristic differences of the microphones 91 to 94. The delay setting unit 35A stores the setting data of the delay amounts d1 to d4. The delay setting unit 35A may also store multiple such setting data and select one of these setting data according to, for example, user operation. The delay setting unit 35A sets the delay amounts d1 to d4 according to the setting data. In this case, the same effect as in the above embodiment can also be obtained.

[0110] Figure 9Shows a structural example of the beamforming processing unit 30B of this modified example. The beamforming processing unit 30B includes delay units 31 to 34, a delay control unit 35, a delay unit 37B, subtraction units 41 to 43, adaptive filters 44 to 46, an adaptive filter control unit 47, an addition unit 48, and a subtraction unit 38. That is, the beamforming processing unit 30B is obtained by omitting the addition unit 36 in the beamforming processing unit 30 ( Figure 4 ) of the above embodiment and replacing the delay unit 37 with the delay unit 37B. The delay unit 37B is configured to generate a signal S37 by delaying only the signal S11 by a delay amount ds. Furthermore, without being limited thereto, the delay unit 37B may also generate the signal S37 by delaying any one of the signals S12 to S14 by only the delay amount ds. The delay amount ds of the delay unit 37B is a predetermined value set by a control unit (not shown). In this case, since the noise component included in the signal S37 is reduced by subtracting the pseudo-noise signal S48 from the signal S37 by the subtraction unit 38, the beamforming processing unit 30B can generate a sound source signal S38 in which the signal component of the sound source selected by the user is emphasized. In this case, the same effect as in the above embodiment can also be obtained.

[0111] Figure 10 Shows a structural example of the beamforming processing unit 30C of this modified example. The beamforming processing unit 30C is composed of the beamforming processing unit 30A ( Figure 8 ) and the beamforming processing unit 30B ( Figure 9 ). The beamforming processing unit 30C includes delay units 31 to 34, a delay setting unit 35A, a delay unit 37B, subtraction units 41 to 43, adaptive filters 44 to 46, an adaptive filter control unit 47, an addition unit 48, and a subtraction unit 38. That is, the beamforming processing unit 30B is obtained by omitting the addition unit 36 in the beamforming processing unit 30 ( Figure 4 ) of the above embodiment and replacing the delay control unit 35 and the delay unit 37 with the delay setting unit 35A and the delay unit 37B, respectively. In this case, the same effect as in the above embodiment can also be obtained.

[0112] [Modified Example 2] In the above embodiment, although the beamforming processing unit 30 outputs the output signal of the subtraction unit 38 as the sound source signal S38, it is not limited thereto. As an alternative, it may also be like Figure 11Like the beamforming processing unit 30D shown, the signal S37 output from the delay unit 37 is output as the sound source signal. In this case as well, since the addition unit 36 adds the signals S31 to S34 in which the phases of the signal components of the sound source selected by the user are made to coincide with each other, the beamforming processing unit 30D can output, as the sound source signal, a signal in which the signal components of the sound source selected by the user are emphasized.

[0113] [Other modification examples] In addition, two or more of these modification examples can be combined.

[0114] As described above, although the present invention has been described by way of embodiments and modification examples, the present invention is not limited to these embodiments and the like, and various changes can be made.

[0115] For example, in the above-described embodiments and the like, although four microphones 91 to 94 are provided, the present invention is not limited thereto, and two or three microphones can be provided, or five or more microphones can be provided.

[0116] The effects described in this specification are merely examples, and the effects of the present disclosure are not limited to the effects described in this specification. Therefore, the present disclosure can also achieve other effects.

[0117] Furthermore, the present disclosure can have the following aspects. (1) A noise processing circuit, comprising: a detection unit that detects a sound source based on a plurality of signals supplied from a plurality of microphones; a flag signal generation unit that generates a flag signal based on the detection result of the detection unit, the flag signal indicating a period during which the plurality of signals include a first signal component; a beamforming processing unit that generates a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component, and generates a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component, based on the plurality of signals and the flag signal; a noise model generation unit that generates a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component, based on the flag signal; a noise model selection unit that selects one of the first signal and the second signal based on the flag signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and a spectrum subtraction unit that performs spectrum subtraction processing based on the spectrum of the first signal and the noise model selected by the noise model selection unit. (2) The noise processing circuit described in (1) above, wherein, The detection unit generates a sequence of meta-information representing the type of the sound source by detecting the sound source. The flag signal generation unit generates the flag signal based on the sequence of the meta-information. (3) The noise processing circuit described in (1) or (2) above, wherein, The flag signal generation unit generates the flag signal based on the detection result of the detection unit and the sound source selection operation of the user accepted by the user interface. (4) The noise processing circuit according to any one of (1) to (3) above, wherein, During the period when the plurality of signals include the first signal component, the beamforming processing unit adjusts a first processing setting of the beamforming processing unit according to the flag signal to generate the first signal in which the first signal component is emphasized; during the period when the plurality of signals do not include the first signal component, the first processing setting is maintained, and the first signal is generated using the first processing setting. (5) The noise processing circuit according to any one of (1) to (4) above, wherein, During the period when the plurality of signals do not include the first signal component, the beamforming processing unit adjusts a second processing setting of the beamforming processing unit according to the flag signal to suppress a second signal component of the first signal and generate the second signal including the second signal component; during the period when the plurality of signals include the first signal component, the second processing setting is maintained, and the second signal is generated using the second processing setting. (6) The noise processing circuit according to any one of (1) to (5) above, wherein, The noise model generation unit generates a first noise model based on the spectrum of the first signal in a first period. The plurality of noise models include the first noise model. (7) The noise processing circuit according to any one of (1) to (6) above, wherein, The noise model generation unit generates a second noise model based on the spectrum of the first signal in a first period and the spectrum of the first signal in a second period. The plurality of noise models include the second noise model. (8) The noise processing circuit according to any one of (1) to (7) above, wherein, When the spectrum of the first signal changes by a certain degree or more, the noise model generation unit generates a third noise model corresponding to the spectrum after the change. The plurality of noise models include the third noise model. (9) The noise processing circuit according to any one of (1) to (8) above, wherein, During the period when the plurality of signals do not include the first signal component, the noise model selection unit selects the first signal among the first signal and the second signal according to the flag signal, and selects one of the plurality of noise models according to the first signal. (10) The noise processing circuit according to any one of (1) to (9) above, wherein, During the period when the plurality of signals include the first signal component, the noise model selection unit selects the second signal among the first signal and the second signal according to the flag signal, and selects one of the plurality of noise models according to the second signal. (11) A signal processing device, comprising: A noise processing circuit that performs noise processing on a plurality of signals supplied from a plurality of microphones; and A processing circuit that performs signal processing according to the processing result of the noise processing circuit, The noise processing circuit has: A detection unit that detects a sound source according to a plurality of signals supplied from a plurality of microphones; A flag signal generation unit that generates a flag signal according to the detection result of the detection unit, and the flag signal indicates a period when the plurality of signals include a first signal component; A beamforming processing unit that, according to the plurality of signals and the flag signal, generates a first signal by emphasizing the first signal component during the period when the plurality of signals include the first signal component, and generates a second signal by suppressing the first signal component during the period when the plurality of signals include the first signal component; A noise model generation unit that, according to the flag signal, generates a plurality of noise models corresponding to the spectrum of the first signal during the period when the plurality of signals do not include the first signal component; A noise model selection unit that, according to the flag signal, selects one of the first signal and the second signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; A spectral subtraction unit performs spectral subtraction processing based on the spectrum of the first signal and the noise model selected by the noise model selection unit. (12) A noise processing method includes: detecting a sound source based on a plurality of signals supplied from a plurality of microphones; generating a flag signal based on the detection result of the sound source, the flag signal indicating a period during which the plurality of signals include a first signal component; generating a first signal based on the plurality of signals and the flag signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component; generating a second signal based on the plurality of signals and the flag signal by suppressing the first signal component during a period when the plurality of signals include the first signal component; generating a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component based on the flag signal; selecting one of the first signal and the second signal based on the flag signal, and selecting one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and performing spectral subtraction processing based on the spectrum of the first signal and the selected noise model among the plurality of noise models. (13) A recording medium records software that causes a processor to perform processing, the processing including: detecting a sound source based on a plurality of signals supplied from a plurality of microphones; generating a flag signal based on the detection result of the sound source, the flag signal indicating a period during which the plurality of signals include a first signal component; generating a first signal based on the plurality of signals and the flag signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component; generating a second signal based on the plurality of signals and the flag signal by suppressing the first signal component during a period when the plurality of signals include the first signal component; generating a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component based on the flag signal; selecting one of the first signal and the second signal based on the flag signal, and selecting one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and Perform spectral subtraction processing according to the spectrum of the first signal and the selected noise model among the multiple noise models.

Claims

1. A noise processing circuit, comprising: a detection unit that detects a sound source based on a plurality of signals supplied from a plurality of microphones; a flag signal generation unit that generates a flag signal based on a detection result of the detection unit, the flag signal indicating a period during which the plurality of signals include a first signal component; a beamforming processing unit that generates a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component, and generates a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component, based on the plurality of signals and the flag signal; a noise model generation unit that generates a plurality of noise models corresponding to a spectrum of the first signal during a period when the plurality of signals do not include the first signal component, based on the flag signal; a noise model selection unit that selects one of the first signal and the second signal based on the flag signal, and selects one of the plurality of noise models by calculating a similarity between a spectrum of the selected signal and each of the plurality of noise models; and a spectrum subtraction unit that performs a spectrum subtraction process based on a spectrum of the first signal and the noise model selected by the noise model selection unit.

2. The noise processing circuit according to claim 1, wherein the detection unit generates a sequence of meta-information indicating a type of the sound source by detecting the sound source, and the flag signal generation unit generates the flag signal based on the sequence of the meta-information.

3. The noise processing circuit according to claim 1, wherein the flag signal generation unit generates the flag signal based on a detection result of the detection unit and a sound source selection operation of a user accepted by a user interface.

4. The noise processing circuit according to claim 1, wherein the beamforming processing unit adjusts a first processing setting of the beamforming processing unit during a period when the plurality of signals include the first signal component, based on the flag signal, to generate the first signal in which the first signal component is emphasized; and during a period when the plurality of signals do not include the first signal component, maintains the first processing setting and generates the first signal using the first processing setting.

5. The noise processing circuit according to claim 1, wherein the beamforming processing unit adjusts a second processing setting of the beamforming processing unit during a period when the plurality of signals do not include the first signal component, based on the flag signal, to suppress a second signal component of the first signal and generate the second signal including the second signal component; and during a period when the plurality of signals include the first signal component, maintains the second processing setting and generates the second signal using the second processing setting.

6. The noise processing circuit according to claim 1, wherein the noise model generation unit generates a first noise model based on a spectrum of the first signal in a first period, and the plurality of noise models include the first noise model.

7. The noise processing circuit according to claim 1, wherein The noise model generation unit generates a second noise model based on the spectrum of the first signal in the first period and the spectrum of the first signal in the second period. The plurality of noise models includes the second noise model.

8. The noise processing circuit according to claim 1, wherein, when the spectrum of the first signal changes by a certain degree or more, the noise model generation unit generates a third noise model corresponding to the changed spectrum, and the plurality of noise models includes the third noise model.

9. The noise processing circuit according to claim 1, wherein, the noise model selection unit, according to the flag signal, during a period when the plurality of signals do not include the first signal component, selects the first signal from the first signal and the second signal, and selects one of the plurality of noise models according to the first signal.

10. The noise processing circuit according to claim 1, wherein, the noise model selection unit, according to the flag signal, during a period when the plurality of signals include the first signal component, selects the second signal from the first signal and the second signal, and selects one of the plurality of noise models according to the second signal.

11. A signal processing apparatus, comprising: a noise processing circuit that performs noise processing on a plurality of signals supplied from a plurality of microphones; and a processing circuit that performs signal processing according to a processing result of the noise processing circuit, wherein the noise processing circuit includes: a detection unit that detects a sound source according to a plurality of signals supplied from a plurality of microphones; a flag signal generation unit that generates a flag signal according to a detection result of the detection unit, the flag signal indicating a period when the plurality of signals include a first signal component; a beamforming processing unit that, according to the plurality of signals and the flag signal, generates a first signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component, and generates a second signal by suppressing the first signal component during a period when the plurality of signals include the first signal component; a noise model generation unit that, according to the flag signal, generates a plurality of noise models corresponding to the spectrum of the first signal during a period when the plurality of signals do not include the first signal component; a noise model selection unit that, according to the flag signal, selects one of the first signal and the second signal, and selects one of the plurality of noise models by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and a spectrum subtraction unit that performs spectrum subtraction processing according to the spectrum of the first signal and the noise model selected by the noise model selection unit.

12. A noise processing method, comprising: detecting a sound source according to a plurality of signals supplied from a plurality of microphones; generating a flag signal according to a detection result of the sound source, the flag signal indicating a period when the plurality of signals include a first signal component; generating a first signal according to the plurality of signals and the flag signal by emphasizing the first signal component during a period when the plurality of signals include the first signal component; A second signal is generated by suppressing the first signal component during a period in which the plurality of signals include the first signal component, based on the plurality of signals and the flag signal; A first signal with the first signal component emphasized and a second signal with the first signal component suppressed are generated. The second signal includes a second signal component corresponding to a noise component included in the plurality of signals; Based on the flag signal, a plurality of noise models corresponding to the spectrum of the first signal are generated during a period in which the plurality of signals do not include the first signal component; Based on the flag signal, one of the first signal and the second signal is selected, and one of the plurality of noise models is selected by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and Spectrum subtraction processing is performed based on the spectrum of the first signal and the selected noise model among the plurality of noise models.

13. A recording medium that records software for causing a processor to perform processing, The processing includes: Detecting a sound source based on a plurality of signals supplied from a plurality of microphones; Based on the detection result of the sound source, a flag signal is generated, the flag signal indicating a period in which the plurality of signals include the first signal component; Based on the plurality of signals and the flag signal, a first signal is generated by emphasizing the first signal component during a period in which the plurality of signals include the first signal component; Based on the plurality of signals and the flag signal, a second signal is generated by suppressing the first signal component during a period in which the plurality of signals include the first signal component; Based on the flag signal, a plurality of noise models corresponding to the spectrum of the first signal are generated during a period in which the plurality of signals do not include the first signal component; Based on the flag signal, one of the first signal and the second signal is selected, and one of the plurality of noise models is selected by calculating the similarity between the spectrum of the selected signal and each of the plurality of noise models; and Spectrum subtraction processing is performed based on the spectrum of the first signal and the selected noise model among the plurality of noise models.

Images