Sound event detection method and system with class distribution and temporal context collaborative cues

Abstract

The application discloses a sound event detection method and system based on distribution and timing context cooperative prompting, converts an original audio signal into a signal frame sequence, extracts a mel filter bank feature through a pre-training model branch, and extracts a mel spectrum feature through a downstream model branch; a combination of a local timing prompt module and a global distribution prompt module is introduced into each layer of the pre-training model, the pre-training model outputs an audio sequence feature and the global distribution prompt module; the audio sequence and the output of the downstream model are fused in features, frame-level prediction probabilities of the downstream model are calculated, and an audio positioning task is realized; then, the frame-level prediction probabilities of the downstream model are used to generate sentence-level prediction probabilities; for the pre-training model, the global distribution prompt module is processed to obtain sentence-level prediction probabilities of the pre-training model; finally, the two sentence-level prediction probabilities are fused to obtain a classification result of a sound event. The application can significantly improve the audio classification and positioning performance of sound event detection.

Images