Music Tagging Transformer Hybrid CNN-Transformer Model
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional music classification techniques face limitations due to limited labelled data, time-consuming labelling processes requiring domain expertise, and failure to model music representation as a sequence, leading to inefficient music tagging performance, especially with large unlabelled datasets.
Innovation Solution
The Music Tagging Transformer (MTT) model combines a convolutional neural network (CNN) front end with a Transformer back end for semi-supervised learning, utilizing noisy student training and a cleaned artist-level dataset split to leverage both labelled and unlabelled data, and apply data augmentation, effectively capturing local audio characteristics and summarizing them temporally.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional CNN-based music tagging models are used, then the model structure is relatively simple, but the prediction accuracy is insufficient and long audio sequences cannot be handled effectively
Solution Approach 1:
The patent combines CNN front-end for local feature extraction with Transformer back-end for global sequence modeling, creating a hybrid architecture that leverages the strengths of both approaches to achieve high prediction accuracy while effectively handling long audio sequences
Solution Approach 2:
The model segments the audio processing into two distinct stages: local feature extraction by CNN on short audio chunks, and global temporal summarization by Transformer, allowing each component to specialize and achieve better overall performance
2Quantity of substance
If only labelled data is used for training, then the training process is straightforward, but the available training data is limited and labelling is time-consuming
Solution Approach 1:
The model performs self-training by generating pseudo-labels for unlabelled data using its own predictions, allowing it to leverage large amounts of unlabelled audio data without requiring time-consuming manual labelling, thus expanding the training dataset significantly
Solution Approach 2:
The model first trains on a small set of labelled data to establish initial capabilities, then uses this preliminary training to generate pseudo-labels for unlabelled data, which are subsequently used to further improve the model in an iterative self-training process
3Productivity
If short audio chunks are processed independently, then the processing is computationally efficient, but the sequential nature of music representation is lost
Solution Approach 1:
The Transformer model acts as an intermediary that takes the independently processed short audio chunks and reconstructs the sequential relationships through self-attention mechanisms, preserving temporal information while maintaining computational efficiency from the chunked processing approach
Data Source
AI summary
The present disclosure describes techniques for identifying music attributes. The described techniques comprises receiving audio data of a piece of music; determining at least one attribute of the piece of music based on the audio data of the piece of music using a model; the model comprising a convolutional neural network and a transformer; the model being pre-trained using training data, wherein the training data comprise labelled data associated with a first plurality of music samples and unlabelled data associated with a second plurality of music samples, the labelled data comprise audio data of the first plurality of music samples and label information indicative of attributes of the first plurality of music samples, and the unlabelled data comprise audio data of the second plurality of music samples.