Automatic audio description method and system

Abstract

The invention provides an automatic audio description method and system. The method comprises the following steps: acquiring real-time audio data; inputting the real-time audio data into a pre-constructed encoder-decoder model to obtain an audio description result; the construction of the encoder-decoder model comprises the following steps: acquiring historical audio data and historical audio description; taking the historical audio data and the historical audio description as training samples, inputting the training samples into a precoder, determining parameters of the precoder, and taking the parameters of the precoder as initial parameters of a coder-decoder model for training; a bidirectional attention mechanism in an original Compact BERT model is replaced by a unidirectional attention mechanism, a cross attention layer is added behind a mask multi-head attention layer, and a linear layer and a Softmax layer are sequentially added behind a forward feedback layer, so that a decoder in the encoder-decoder model is obtained. According to the invention, the Compact BERT model is integrated into the decoder, so that the accuracy of audio description generation is improved.

Images