Multi-channel double-speaker separation method and system

Abstract

The invention relates to a multi-channel double-speaker separation method and system, and the method comprises the steps: carrying out the processing of a mixed voice audio, and obtaining the frequency spectrum of each frame of audio; according to each frame of audio and the sound source position estimation network, obtaining estimated frame-level Cartesian coordinates and corresponding weights; obtaining a first logarithmic energy spectrum and a first phase difference between sine and cosine channels according to the frequency spectrum of each frame of audio; according to the estimated frame-level Cartesian coordinates and the corresponding weights, obtaining Cartesian coordinate estimation of a target speaker in the mixed voice audio; obtaining a first angle feature according to the Cartesian coordinate of the target speaker; obtaining a target speaker and a first estimated speaker mask according to the first logarithmic energy spectrum, the first phase difference between sine and cosine channels, the first angle feature and a speaker mask estimation network; and obtaining separated voices of the at least two speakers based on the target speaker, the first estimated speaker mask and the mixed voice audio.

Description

technical field [0001] The embodiments of the present application relate to the field of speech separation, and in particular to a multi-channel dual-speaker separation method and system. Background technique [0002] The goal of speech separation is to separate different speakers from the mixed speech audio with reverberation and noise to obtain clean individual speaker speech. Speech separation, as the front end of speech recognition system, voice log and other technologies, is widely used in various environments such as teaching environment and conference environment. [0003] Deep clustering is a traditional method for speech separation. It obtains the separated speech of the target speaker by training the ideal binary masking of the target speaker on the mixed speech audio. During the training process, each time-frequency unit needs to be vectorized, and then the time-frequency units with similar distances are clustered together. However, for the influence of differe...

AI Technical Summary

Problems solved by technology

Studies have shown that in a conference environment, the ratio of speaker overlap is generally not higher than 20%, so the robust performance for speech separation with different low speaker overlap ratios still needs to be improved

On the other hand, for mixed speech audio with different overlapping ratios of speakers, the specific position of the target speaker in the speech cannot be known in advance, and the input for neural network training can only be the entire speech

In this case, if average pooling is used, interfering speaker speech and silent frames will seriously affect the estimation of target speaker position information, thereby degrading the performance of speech separation

Images