Joint model training method and system

Abstract

The embodiment of the invention provides a combined model training method. The method comprises the following steps: extracting the phase spectrum and the logarithm magnitude spectrum of a noisy voicetraining set in an implicit manner; by utilizing the magnitude spectrum fragments of the logarithm magnitude spectrum after expansion as the input features of a time frequency masking network, and byutilizing the noisy voice training set and a clear voice training set, determining a target masking label used for training the time frequency masking network, based on the input features and the target masking label, training the time frequency masking network, and estimating a soft threshold mask; and enhancing the phase spectrum of the noisy voice training set by utilizing the soft threshold mask, wherein the enhanced phase spectrum is adopted as the input features of a DOA (direction of arrival) estimation network, and training the DOA estimation network. The embodiment of the invention further provides a combined model training system. According to the embodiment of the invention, by setting the target masking label, the input features are extracted in an implicit manner, and the time frequency masking network and DOA estimation network combined training is more suitable for the DOA estimation task.

Description

technical field [0001] The invention relates to the field of sound source localization, in particular to a joint model training method and system. Background technique [0002] Sound source localization is the task of estimating the speaker DOA (Direction of arrival) from the received speech signal. DOA estimation is essential for various applications such as human-computer interaction and teleconferencing, and is also widely used in speech Enhanced beamforming. For example, sound source localization is added to the chat video. As the chat user's position changes, the voice received by the user at the other end can feel the change of the other party's position, improving the user experience. [0003] In order to determine the direction of arrival, the target speaker localization method based on keywords can be used: use the neural network to estimate the time-frequency mask separately, and then use the estimated mask to enhance the input features of the direction of arrival...

AI Technical Summary

Problems solved by technology

[0005] Keyword-based target speaker localization method: Because it trains the mask network separately, the obtained time-frequency mask and localization tasks are independent of each other, which cannot guarantee the best localization performance; and the input features it uses are pre-extracted The phase difference feature between the sine-cosine channels increases the amount of additional calculation

The joint training method of time-frequency mask and DOA estimation network based on acoustic vector sensor: it uses an acoustic vector sensor, which is more complex and costly than ordinary microphone arrays; the estimated time-frequency mask is in the complex domain Compared with the real number field, it is more complex and has a large amount of calculation; the input features used are the data ratio between channels of the sub-band, power spectrum, coherence vector, etc., and feature extraction needs to be performed explicitly in advance, which increases the amount of additional calculation.

Images