Chinese song emotion classification method based on multi-modal fusion

Abstract

The invention discloses a Chinese song emotion classification method based on multi-modal fusion. The Chinese song emotion classification method comprises the steps: firstly obtaining a spectrogram from an audio signal, extracting audio low-level features, and then carrying out the audio feature learning based on an LLD-CRNN model, thereby obtaining the audio features of a Chinese song; for lyricsand comment information, firstly constructing a music emotion dictionary, then constructing emotion vectors based on emotion intensity and part-of-speech on the basis of the dictionary, so that textfeatures of Chinese songs are obtained; and finally, performing multi-modal fusion by using a decision fusion method and a feature fusion method to obtain emotion categories of the Chinese songs. TheChinese song emotion classification method is based on an LLD-CRNN music emotion classification model, and the model uses a spectrogram and audio low-level features as an input sequence. The LLD is concentrated in a time domain or a frequency domain, and for the audio signal with associated change of time and frequency characteristics, the spectrogram is a two-dimensional representation of the audio signal in frequency, and loss of information amount is less, so that information complementation of the LLD and the spectrogram can be realized.

Description

technical field [0001] The invention relates to the fields of natural language processing technology, audio signal processing technology and deep learning, in particular to a multimodal fusion-based emotion classification method for Chinese songs. Background technique [0002] With the rapid development of computer network and multimedia technology, more and more multimedia data such as text, image, audio and video have emerged on the Internet. Music is an important part of multimedia data. Facing the explosive growth of the number of music works and the continuous increase of music types, the organization and retrieval of music works have attracted extensive attention from experts and scholars. Music is the carrier of emotion, emotion is the most important semantic information of music, and emotion words are the most commonly used words when retrieving and describing music. Therefore, music classification based on emotion can effectively improve the efficiency of music retr...

AI Technical Summary

Problems solved by technology

Most of the traditional music emotion classification methods focus on the analysis of lyrics or audio, but single-modal data can only obtain part of the characteristics of the object, and there is a certain degree of information loss when only using single-modal data for classification

Images