Signal processing device, signal processing method, and signal processing program

Abstract

This signal processing device uses a dereverberation learning dataset to perform training of a model (DNN) for estimating a switch such that a signal in which a reverberation component has been removed via switching WPE is optimized per an evaluation criterion indicated by a user. Thereafter, with respect to an observed signal, the signal processing device sets, for the switching WPE, a switch (optimized switch) estimated by the learned model. Further, the signal processing device performs WPE filter optimization with respect to the set switch. Thereafter, the signal processing device removes the reverberation component of the observed signal using switching WPE provided with the optimized switch and WPE filter.

Description

[Technical Field]

[0001] The present invention relates to a signal processing device, a signal processing method, and a signal processing program. [Background technology]

[0002] There is a technology to remove reverberation from voice or music signals recorded with a remote microphone. Reverberation is a signal component that arrives at a microphone later than the original signal due to reflections from the walls, floor, ceiling, etc.

[0003] Since signals containing reverberation can lead to performance degradation in speech recognition, noise reduction by signal processing, sound source separation, etc., performance degradation can be avoided by removing reverberation from the signal in advance. Note that the removal of reverberation components as described above can also be applied to, for example, improving the performance of hearing aids and automatic music notation creation.

[0004] WPE (Weighted Prediction Error) is a technique for removing reverberation components. WPE assumes an autoregressive model of reverberation and predicts current reverberation components from past observed signals. WPE then estimates an inverse filter that cancels out the predicted reverberation components, and performs dereverberation using this estimated inverse filter. WPE can remove reverberation, but when the number of sound sources is greater than or equal to the number of microphones, the MINT theorem shows that a causal inverse filter (an inverse filter that uses only past signals) cannot exist.

[0005] SwitchingWPE (see Non-Patent Document 1) is a technology that partially solves the above problem. SwitchingWPE is an improved version of WPE, and achieves dereverberation by switching between multiple WPE filters for each time-frequency bin of the signal. Here, the parameter used to select the time-frequency bin to which the WPE filter is applied is called Switch. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] Rintaro Ikeshita,et al., "Blind Signal Dereverberation Based on Mixture of Weighted Prediction Error Models", IEEE SIGNAL PROCESSING LETTERS, VOL. 28, 2021, 399. Summary of the Invention [Problem to be solved by the invention]

[0007] In SwitchingWPE, the switch is optimized using the weighted power minimization criterion (maximum likelihood criterion), so the optimized switch is not necessarily the optimal switch in terms of other evaluation criteria (e.g., speech recognition rate, signal distortion measure, etc.). For example, the switch optimized by SwitchingWPE is not necessarily the optimal switch for speech recognition. Therefore, the speech recognition rate for signals after dereverberation using SwitchingWPE may not be high.

[0008] Therefore, an object of the present invention is to solve the above-mentioned problems and to improve the performance of reverberation component removal in SwitchingWPE according to the purpose. [Means for solving the problem]

[0009] In order to solve the above-mentioned problems, the present invention provides a SwitchingWPE having a plurality of WPE filters that remove reverberation components of an observed signal and a Switch for switching between the plurality of WPE filters for each time frequency of the observed signal; a reception unit that receives an input of an evaluation criterion for the signal after reverberation components have been removed by the SwitchingWPE; a learning unit that uses a training dataset for removing the reverberation components of a signal and trains a model that outputs an estimation result for the Switch such that the signal after reverberation components have been removed by the SwitchingWPE is optimized using the evaluation criterion; a Switch setting unit that sets the Switch estimated by the trained model for the observed signal in the SwitchingWPE; and a filter setting unit that calculates an optimal WPE filter for the set Switch and sets it in the SwitchingWPE, wherein the SwitchingWPE removes the reverberation components of the input signal using the set Switch and the set WPE filter. [Effects of the Invention]

[0010] According to the present invention, it is possible to improve the performance of reverberation components in SwitchingWPE according to the purpose. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining an overview of SwitchingWPE. [Figure 2] FIG. 2 is a diagram illustrating an outline of the signal processing device. [Figure 3] FIG. 3 is a diagram illustrating an outline of a signal processing device when the evaluation standard is SDR. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of a signal processing device. [Figure 5] FIG. 5 is a diagram illustrating an example of a processing procedure of the signal processing device. [Figure 6] FIG. 6 is a diagram showing the evaluation results of the reverberation component removal performance of the signal processing device. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a computer that executes a signal processing program. DETAILED DESCRIPTION OF THE INVENTION

[0012] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a description will be given of an embodiment of the present invention with reference to the drawings, but the present invention is not limited to the embodiment.

[0013] [SwitchingWPE] First, an overview of SwitchingWPE, which is a basic technology used by the signal processing device of this embodiment, will be described using Fig. 1. SwitchingWPE clusters the time frequencies of an observed signal (e.g., a speech signal) and uses Switch to switch between multiple WPE filters for each time frequency, thereby achieving dereverberation of the observed signal. A WPE filter is calculated for each Switch. The Switch and WPE filters of SwitchingWPE are alternately optimized using a weighted power minimization criterion (maximum likelihood criterion).

[0014] [overview] Next, an overview of the signal processing device of this embodiment will be described with reference to Fig. 2. The signal processing device estimates an optimal Switch for dereverberation using SwitchingWPE from an observed signal using a model such as a DNN (Deep Neural Network). Then, the signal processing device dereverberates the observed signal using SwitchingWPE using the estimated Switch.

[0015] For example, a signal processing device uses a dereverberation training dataset to train a Switch estimation model that receives an input of an observed signal and outputs an optimal Switch estimation result for the observed signal. Note that this training dataset is a dataset that shows the input signal and a signal obtained by removing the reverberation component from the input signal (a dereverberation correct signal).

[0016] Here, before training the above-mentioned Switch estimation model, the signal processing device accepts input of evaluation criteria for signals after dereverberation by SwitchingWPE (e.g., SDR (Signal-to-Distortion Ratio), Scale-invariant SDR, intelligibility such as STOI (Short-Time Objective Intelligibility measure), Cepstral distance, WER (Word Error Rate) in ASR (Automatic Speech Recognition), etc.).

[0017] Then, the signal processing device uses the dereverberation training dataset to train a Switch estimation model that estimates a Switch such that the signal (dereverberated signal) after the reverberation components have been removed by SwitchingWPE is optimized using the above evaluation criteria.

[0018] For example, consider a case where the evaluation criterion for the dereverberated signal is SDR and the Switch estimation model is implemented by a DNN. In this case, the signal processing device optimizes the DNN to maximize the SDR between the dereverberated signal output by the SwitchingWPE for an input signal included in the training data set and the dereverberated target signal for that input signal included in the training data set (see Figure 3).

[0019] The signal processing device then inputs the observed signal into the optimized DNN and obtains an estimated Switch optimized for the observed signal.The signal processing device then uses the estimated Switch to dereverberate the observed signal using SwitchingWPE.In this way, the signal processing device can achieve improved performance in removing reverberation components in SwitchingWPE according to the purpose (evaluation criterion).

[0020] [Configuration example] Next, an example of the configuration of the signal processing device 10 will be described with reference to Fig. 4. The signal processing device 10 includes an input / output unit 11, a storage unit 12, and a control unit 13.

[0021] The input / output unit 11 is an interface that controls the input and output of various information. For example, the input / output unit 11 receives inputs of an observed signal to be dereverberated, evaluation criteria for the dereverberated signal, etc. Furthermore, for example, the input / output unit 11 outputs the dereverberated signal.

[0022] The storage unit 12 stores data to be referenced when the control unit 13 executes various processes. For example, the storage unit 12 stores evaluation criteria for dereverberated signals, a dereverberation learning data set, a switch estimation model (parameters of the switch estimation model) for estimating an optimal switch, and the like.

[0023] The above-described Switch estimation model is a model that receives an observed signal as input to the SwitchingWPE 131 and outputs an estimation result of an optimal Switch in the SwitchingWPE 131. This Switch estimation model is realized by, for example, a DNN. The Switch estimation model is learned by the learning unit 133.

[0024] The control unit 13 controls the entire signal processing device 10. The control unit 13 includes a SwitchingWPE 131, a receiving unit 132, a learning unit 133, a Switch setting unit 134, and a filter setting unit 135.

[0025] SwitchingWPE131 derives dereverberation from an observed signal using Switch and multiple WPE filters. Switch is a parameter for switching between multiple WPE filters for each time frequency of the observed signal. The WPE filters remove the reverberation components of the observed signal.

[0026] The receiving unit 132 receives an input of an evaluation criterion for the dereverberated signal. The evaluation criterion is, for example, an intelligibility such as SDR, Scale invariant SDR, or STOI, a Cepstrum distance, or a WER in ASR. The evaluation criterion for the dereverberated signal received by the receiving unit 132 may be any one of the above evaluation criteria or a combination of multiple evaluation criteria.

[0027] The learning unit 133 uses a dereverberation learning dataset, receives an observed signal as input, and learns a Switch estimation model that outputs a Switch estimation result such that the signal from which the reverberation components have been removed by the SwitchingWPE 131 is optimized using the evaluation criterion input by the receiving unit 132.

[0028] For example, when the evaluation criteria are SDR, Scale invariant SDR, STOI, or Cepstral distance, the learning unit 133 sets the dereverberated signal (correct signal) for the input signal of the training dataset as correct data. Then, using the correct data, the learning unit 133 learns a Switch estimation model that estimates the Switch of the SwitchingWPE 131 so that the result of evaluating the reverberant signal output by the SwitchingWPE 131 for the input signal of the training dataset using the evaluation criteria is optimized.

[0029] Note that when the evaluation criterion is ASR, the learning unit 133 uses the correct text (a sentence transcribed from the input speech) as correct data instead of the correct signal. In this case, the learning unit 133 inputs the dereverberated signal output by the SwitchingWPE 131 to the ASR, and learns the Switch estimation model so that the recognition result by the ASR matches the correct text as closely as possible (so that the ASR accuracy rate is improved). For example, the learning unit 133 learns the Switch estimation model so that the WER of the recognition result by the ASR is as small as possible.

[0030] The Switch setting unit 134 sets the Switch output by the learned Switch estimation model for the observed signal to SwitchingWPE 131. Furthermore, the filter setting unit 135 calculates an optimal WPE filter for the Switch set by the Switch setting unit 134, and sets the optimal WPE filter to SwitchingWPE 131. The calculation method of the optimal WPE filter here may be, for example, the same method as the calculation method of the WPE filter in conventional SwitchingWPE.

[0031] Thereafter, the SwitchingWPE 131 removes the reverberation components of the input observation signal using the Switch set by the Switch setting unit 134 and the WPE filter set by the filter setting unit 135 .

[0032] In this way, the signal processing device 10 can improve the performance of the SwitchingWPE 131 in removing reverberation components according to the purpose.

[0033] [Example of processing procedure] Next, an example of a processing procedure of the signal processing device 10 will be described with reference to Fig. 5. First, the receiving unit 132 of the signal processing device 10 receives an input of an evaluation criterion for a signal after reverberation components have been removed by the SwitchingWPE 131 (S1).

[0034] Next, the learning unit 133 uses a dereverberation learning dataset to learn a Switch estimation model that takes a signal as input and outputs a Switch estimation result that optimizes the signal from which the reverberation components have been removed by the SwitchingWPE 131 using the evaluation criterion accepted in S1 (S2: Model learning).

[0035] After S2, the signal processing device 10 accepts input of an observed signal (S3). Then, the Switch setting unit 134 sets the Switch estimated by the Switch estimation model learned in S2 for the observed signal input in S3 to the SwitchingWPE 131 (S4: Setting the Switch). Then, the filter setting unit 135 calculates an optimal WPE filter for the set Switch and sets it to the SwitchingWPE 131 (S5: Setting the WPE filter).

[0036] Thereafter, the SwitchingWPE 131 removes the reverberation component of the input observation signal using the Switch set by the Switch setting unit 134 and the WPE filter set by the filter setting unit 135 (S6).

[0037] In this way, the signal processing device 10 can improve the performance of the SwitchingWPE 131 in removing reverberation components according to the purpose.

[0038] [Evaluation results] Next, the evaluation results of the reverberation component removal performance of the signal processing device 10 will be described with reference to Fig. 6. Here, the signal processing device 10 was evaluated for its reverberation component removal performance on speech data containing reverberation components that was created by simulation.

[0039] The signal processing device 10 used a dereverberation training dataset to train a DNN that estimates the optimized Switch for a signal from which reverberation components have been removed by SwitchingWPE131, using SDR as an evaluation criterion. The audio data to be evaluated was audio data recorded with one microphone. The number of Switches used by SwitchingWPE131 was three. The comparison targets were the observed signal (unprocessed), WPE, and SwitchingWPE.

[0040] As shown in Figure 6, it was confirmed that the speech data dereverberated by the signal processing device 10 using SwitchingWPE, which uses the Switch estimated by the above-mentioned DNN, has a higher SDR than WPE and SwitchingWPE. It was also confirmed that the speech data dereverberated by the signal processing device 10 has a lower word recognition error rate than WPE and SwitchingWPE.

[0041] This confirms that the signal processing device 10 can achieve improved performance in removing reverberation components according to the purpose, compared to the SwitchingWPE131.

[0042] [System configuration, etc.] Furthermore, the components of each unit shown in the figure are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program executed by the CPU, or can be realized as hardware using wired logic.

[0043] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0044] [program] The signal processing device 10 can be implemented by installing a program (signal processing program) as package software or online software on a desired computer. For example, by executing the program on an information processing device, the information processing device can function as the signal processing device 10. The information processing device referred to here includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as terminals such as PDAs (Personal Digital Assistants).

[0045] 7 is a diagram showing an example of a computer that executes a signal processing program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0046] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0047] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes executed by the signal processing device 10 are implemented as program modules 1093 in which computer-executable codes are written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configuration of the signal processing device 10 are stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0048] Data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as needed and executes them.

[0049] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a LAN (Local Area Network) or WAN (Wide Area Network)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070. [Explanation of symbols]

[0050] 10. Signal Processing Device 11 Input / output section 12 Storage section 13 Control Unit 131 SwitchingWPE 132 Reception Department 133 Learning Department 134 Switch Settings 135 Filter setting section

Claims

1. a SwitchingWPE having a plurality of WPE filters that remove reverberation components of the observed signal and a Switch for switching the plurality of WPE filters for each time frequency of the observed signal; a receiving unit that receives an input of an evaluation criterion for the signal after the reverberation component has been removed by the SwitchingWPE; a learning unit that uses a dereverberation learning dataset to learn a model that estimates the Switch such that a signal from which reverberation components have been removed by the SwitchingWPE is optimized using the evaluation criterion; and A Switch setting unit that sets a Switch estimated by the model after learning to the SwitchingWPE for an observed signal; A filter setting unit calculates an optimal WPE filter for the set Switch and sets it to the SwitchingWPE, The SwitchingWPE is The reverberation component of the observed signal is removed using the set Switch and the set WPE filter. A signal processing device comprising:

2. The evaluation criteria are: Signal-to-distortion ratio, scale-invariant signal-to-distortion ratio, intelligibility, cepstral distance, and / or word error rate in automatic speech recognition.

2. The signal processing device according to claim 1.

3. the dereverberation training data set includes an input signal and a signal obtained by removing reverberation components from the input signal, which is correct data of the input signal; The learning unit The model is trained so that the evaluation result of the signal after the reverberation component removal, output by the SwitchingWPE, and the ground truth data is optimized for the input signal, based on the evaluation criteria.

2. The signal processing device according to claim 1.

4. The model is This is a model that estimates the switch using a DNN (Deep Neural Network).

2. The signal processing device according to claim 1.

5. A signal processing method performed by a signal processing device, comprising: receiving an input of an evaluation criterion for the signal after the reverberation components have been removed by a SwitchingWPE having a plurality of WPE filters that remove the reverberation components of the observed signal and a Switch for switching the plurality of WPE filters for each time frequency of the observed signal; a step of training a model that outputs Switch estimation results such that a signal from which reverberation components have been removed by the SwitchingWPE is optimized using the evaluation criterion, using a training dataset for dereverberation; A step of setting the Switch estimated by the model after learning to the SwitchingWPE for an observed signal; A process of calculating an optimal WPE filter for the set Switch and setting it as the SwitchingWPE; removing the reverberation component of the observed signal using the Switch and the SwitchingWPE to which the WPE filter is set; A signal processing method comprising:

6. receiving an input of an evaluation criterion for the signal after the reverberation components have been removed by a SwitchingWPE having a plurality of WPE filters that remove the reverberation components of the observed signal and a Switch for switching the plurality of WPE filters for each time frequency of the observed signal; a step of training a model that outputs Switch estimation results such that a signal from which reverberation components have been removed by the SwitchingWPE is optimized using the evaluation criterion, using a training dataset for dereverberation; A step of setting the Switch estimated by the model after learning to the SwitchingWPE for an observed signal; A process of calculating an optimal WPE filter for the set Switch and setting it as the SwitchingWPE; removing the reverberation component of the observed signal using the Switch and the SwitchingWPE to which the WPE filter is set; A signal processing program for executing the above on a computer.

Images