An end-to-end sound source localization method and system based on multi-task learning

Abstract

The invention discloses an end-to-end sound source positioning method and system based on multi-task learning. The method comprises the following steps: 1) for each sound source position to be scanned, calculating the delay of transmitting a sound signal from the sound source position to each microphone position; 2) performing corresponding delay compensation on the multi-channel frame-level timedomain signal acquired by each microphone when the microphone array scans each time according to the delay; 3) inputting each time domain signal after delay compensation into a corresponding CNN modelfor feature extraction, and inputting the feature into a deep neural network; 4) estimating a multi-channel sound source signal of each scanning position by the deep neural network according to the feature extracted by each CNN model; and 5) for each scanning position, calculating the cross correlation coefficient sum of the multi-channel sound source signals corresponding to the scanning position, and selecting the position with the maximum correlation coefficient sum as the sound source position. According to the invention, appropriate characteristics can be automatically extracted, a multi-task learning mechanism is introduced, and the positioning performance of the model is improved.

Description

technical field [0001] The invention belongs to the technical field of array signal processing, and relates to a microphone array and a sound source localization method, in particular to an end-to-end sound source localization method and system based on multi-task learning. Background technique [0002] With the development of artificial intelligence technology, machine hearing has attracted widespread attention, and many technologies and research fields related to machine hearing have emerged one after another. Sound source localization technology is a basic and important technology in the machine auditory system. Its essence is to imitate the function of human ears, collect sound signals through microphone arrays, and then judge the position of sound-emitting objects. Sound source localization technology can be independently applied in many fields, such as video conferencing, vehicle whistle recognition, etc., and can also provide basic location information for many techno...

AI Technical Summary

Problems solved by technology

This type of neural network method is very convenient when modeling. It simply relies on a large amount of data for iterative learning to learn a classifier with better performance, but it needs to design and screen an artificial feature with better performance in advance, and this type of method is very difficult. It is difficult to obtain a more general mapping relationship from features to sound source positions through learning, that is, classifiers. The actual positioning performance is relatively poor and the robustness is insufficient.

Images