Supervised Nonnegative Matrix Factorization for Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing nonnegative matrix factorization (NMF) methods lack classification capability due to their unsupervised nature and inability to effectively incorporate relationships like class labels, leading to inefficiencies in applications such as face recognition and multimedia analysis.
Innovation Solution
The introduction of Supervised Nonnegative Matrix Factorization (SNMF) that combines the objectives of NMF and graph embedding using intrinsic and penalty graphs, allowing for the incorporation of similarity and dissimilarity relationships through iterative multiplicative updates, enabling effective classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If nonnegative matrix factorization (NMF) is used for data factorization, then ease of implementation and physical interpretation are improved, but classification capability deteriorates due to unsupervised factorization not utilizing class labels
Solution Approach 1:
The patent merges NMF with graph embedding by combining their objective functions. The NMF objective function decomposes the data matrix into basis and coefficient matrices, while the graph embedding objective function incorporates class label information through similarity and dissimilarity graphs. This combination allows the method to maintain the ease of implementation and physical interpretation of NMF while gaining the classification capability of supervised learning through graph-based relationships.
2Measurement precision
If Fisher NMF is used to incorporate between-class and within-class scatter, then classification features are improved, but convergence is not guaranteed due to non-convex objective function
Solution Approach 1:
The patent introduces graph embedding as an intermediary framework that mediates between the data matrix and class labels. Instead of directly incorporating between-class and within-class scatter into the NMF objective (which creates non-convexity), the method uses graph Laplacians derived from similarity and dissimilarity graphs as intermediate representations. These graph-based intermediaries encode class relationships in a way that maintains convexity and ensures convergence while still providing discriminative classification features.
3Loss of information
If graph regularized NMF (GNMF) is used to append favorable relationship terms, then feature relationships are improved, but unfavorable relationships are not considered
Solution Approach 1:
The patent applies the anti-weight principle by introducing both similarity graphs (favorable relationships) and dissimilarity graphs (unfavorable relationships) with opposite signs in the objective function. The similarity graph term encourages points within the same class to be close, while the dissimilarity graph term penalizes points from different classes from being close. This counterbalancing approach ensures that both favorable and unfavorable relationships are considered, enhancing the discriminative power of the factorization.
4Ease of manufacture
If non-negative graph embedding (NGE) is used with approximate formulation, then implementation is simplified, but classification effectiveness deteriorates particularly when intra-class variations are large
Solution Approach 1:
The patent changes the parameter formulation from the approximate NGE approach to the exact graph embedding formulation. Specifically, it uses the normalized Laplacian matrix formulation with proper scaling parameters that ensure numerical stability and convergence. This parameter change allows the method to handle large intra-class variations effectively by properly weighting the graph terms, while maintaining implementation feasibility through iterative multiplicative update rules that preserve non-negativity constraints.
Data Source
AI summary
Graph embedding is incorporated into nonnegative matrix factorization, NMF, while using the original formulation of graph embedding. Negative values are permitted in the definition of graph embedding without violating the nonnegative requirement of NMF. The factorized matrices of NMF are found by an iterative process.