Multilabel Classification via Non-Negative Matrix Factorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multilabel classification techniques face challenges with high training and prediction runtimes, especially when dealing with a large number of labels, and are computationally expensive, particularly when label vectors are sparse and belong to few classes.
Innovation Solution
The method involves non-negative matrix factorization to compute a basis matrix, generating a group testing matrix by sampling this basis matrix, and using Boolean OR operations to reduce label vectors, training a reduced number of binary classifiers, and predicting labels using these classifiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional multilabel classification techniques are used, then classification accuracy can be maintained, but training and prediction runtimes become excessively long and computational cost increases significantly when dealing with large numbers of labels
Solution Approach 1:
The patent segments the multilabel classification problem into multiple binary classification problems. Each label is classified independently using binary classifiers, transforming a complex multilabel problem into simpler binary problems that can be solved more efficiently. This segmentation reduces the computational complexity from O(d^2) to O(d) where d is the number of labels.
Solution Approach 2:
The patent extracts and removes redundant information from the label vectors through dimensionality reduction techniques. By identifying and eliminating correlated labels or redundant features, the system reduces the effective number of labels that need to be classified, thereby decreasing training and prediction runtimes while maintaining classification accuracy.
2Reliability
If traditional multilabel classification techniques are used, then comprehensive label coverage can be achieved, but computational expense increases significantly especially when label vectors are sparse
Solution Approach 1:
The patent applies local quality by treating different labels with different classification strategies based on their characteristics. Labels are grouped by similarity or correlation, and different classification approaches are applied to different groups. This allows the system to optimize computational resources by applying more expensive methods only where necessary while using simpler methods for other labels.
Solution Approach 2:
The patent changes parameters such as the number of classifiers, dimensionality reduction factors, and sparsity thresholds to optimize the balance between classification accuracy and computational cost. By dynamically adjusting these parameters based on the specific problem characteristics, the system achieves reliable classification while minimizing computational expense.
3Loss of time
If the number of binary classifiers is reduced for faster prediction, then prediction runtime decreases, but the ability to accurately classify all labels may be compromised
Solution Approach 1:
The patent performs preliminary actions during the training phase by pre-computing label correlations, feature importances, and optimal classifier subsets. This preliminary analysis allows the system to identify which labels can be accurately predicted with fewer classifiers and which require more comprehensive classification. The preliminary computation enables faster prediction runtime without sacrificing accuracy for labels that don't require full classification.
Data Source
AI summary
A computer-implemented method for automatic multilabel classification includes receiving a label matrix Y for multiple training instances. The label matrix Y includes multiple labels, each label representing a respective category. The method further includes computing an intermediate matrix YYT, where YT is a transpose of the label matrix Y. The method further includes computing a basis matrix H by a non-negative matrix factorization of the intermediate matrix YYT. The method further includes generating a group testing matrix A by sampling the basis matrix H. The method further includes generating, for each training instance from the training instances, a reduced label vector z by computing a product of the group testing matrix A and a label vector y for respective training instance from the label matrix Y. The method further includes predicting multiple labels associated with an input based on the reduced label vector z.