Supervised Metric Learning Embeddings for Music Structure Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Music Structure Analysis (MSA) methods face challenges in devising suitable features that capture pitch, timbre, rhythm, and dynamics, and existing supervised learning approaches are limited in compatibility with MSA processing pipelines, specifically failing to address both segmentation and labeling tasks simultaneously.
Innovation Solution
A deep neural network is trained using supervised metric learning to produce embeddings that differentiate between music sections, with a batch sampling scheme ensuring meaningful label interpretation, allowing for improved feature extraction and integration with existing MSA algorithms to support both segmentation and labeling.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional features (MFCCs, PCPs) are used for Music Structure_analysis, then compatibility with existing MSA pipelines is maintained, but the ability to simultaneously optimize for both segmentation and labeling tasks is limited
Solution Approach 1:
The patent applies universality by designing a supervised metric learning framework that simultaneously performs both segmentation and labeling tasks through a unified loss function. The deep neural network learns embeddings that are optimized for multiple MSA objectives at once, making the system multi-functional rather than requiring separate specialized models for each task.
Solution Approach 2:
The patent segments the learning objective into distinct loss components (segmentation loss and labeling loss) that are combined in a unified framework. This allows the system to handle multiple tasks simultaneously while maintaining clear separation of objectives, resolving the contradiction between versatility and complexity.
2Measurement precision
If supervised learning methods are used to model music boundaries and choruses, then performance on specific tasks improves, but compatibility with existing MSA processing pipelines deteriorates
Solution Approach 1:
The patent introduces embeddings as an intermediary representation that bridges supervised learning and existing MSA pipelines. The deep neural network produces embedding vectors that capture musical structure information, which can then be fed into conventional MSA algorithms, thus maintaining pipeline compatibility while incorporating supervised learning benefits.
3Adaptability or versatility
If multiple features are combined for MSA, then comprehensive music structure analysis is achieved, but difficulty in selecting suitable features for different timescales increases
Solution Approach 1:
The patent changes the parameter representation from raw conventional features to learned embedding vectors. This transformation allows the system to capture multiple music aspects (pitch, timbre, rhythm, dynamics) in a unified representation space, eliminating the need for manual feature selection and timescale matching while maintaining comprehensive analysis capability.
Data Source
AI summary
Devices, systems, and methods related to implementing supervised metric learning during a training of a deep neural network model are disclosed herein. In examples, audio input may be received, where the audio input includes a plurality of song fragments from a plurality of songs. For each song fragment, an aligning function may be performed to center the song fragment based on determined beat information, thereby creating a plurality of aligned song fragments. For each song fragment of the plurality of song fragments, an embedding vector may be obtained from the deep neural network. Thus, a batch of aligned song fragments from the plurality of aligned song fragments may be selected, such that a training tuple may be selected. A loss metric may be generated based on the selected training tuple and one or more weights of the deep neural network model may be updated based on the loss metric.