Multi-speaker speech separation method and system based on beam forming

Abstract

The invention discloses a multi-speaker speech separation method and system. The method comprises the following steps : acquiring mixed voice signals, obtaining multichannel multi-speaker mixed voicesignals, and scanning the multi-channel multi-speaker mixed voice signals to obtain MUSIC energy spectra; obtaining S peak values from the MUSIC energy spectra, wherein each peak value corresponds toa beam direction; respectively enhancing S beams to obtain mixed voice in S directions; performing short-time Fourier transform on the mixed voice corresponding to each direction to obtain short-timeFourier amplitude spectra of the S target speaker voice, and respectively inputting the short-time Fourier amplitude spectra into a depth neural network to estimate a phase sensing mask correspondingto each target speaker; and performing element-by-element multiplication between the phase sensing mask of each target speaker and amplitude spectra of the corresponding mixed speech to obtain the amplitude spectrum of the target speaker, and the time domain signal of the target speaker is recovered by inverse short-time Fourier transform using the phase spectrum of the corresponding mixed speech.

Description

technical field [0001] The invention belongs to the technical field of speech separation, and relates to beamforming and a deep neural network model, in particular to a method and system for speech separation based on beamforming. Background technique [0002] In a complex acoustic scene with interference such as noise or multiple speakers, picking out the voice of the target speaker has always been a difficult problem in the field of speech. This problem is called the "cocktail party problem". Normal people benefit from their own auditory attention mechanism, focusing on the target sound in the mixed sound, so that they can communicate in this complex environment. For machines, however, the "cocktail party problem" is a difficult task. Although the recognition rate of automatic speech recognition can be close to or even exceed that of ordinary people in clean speech, the recognition rate of automatic speech recognition will drop significantly in speech recognition with mul...

AI Technical Summary

Problems solved by technology

But in theory, the time complexity of matching calculation is factorial

[0007] However, there are two problems in the two ideas and the multi-channel separation methods based on them. One is that the more speakers there are, the more difficult the separation is, and the lower the intelligibility of the separated speech is; Artificially setting the number of speakers or the maximum number of speakers, even if there is an improved method later, it performs poorly in the separation of unknown speakers

Images