Ratio-Sum LDA Dimension Reduction Without Covariance Inversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional Linear Discriminant Analysis (LDA) methods tend to select features with small variances and low discriminant ability, and require matrix inversion, leading to inefficiencies and low accuracy in image classification, particularly for hyperspectral images.
Innovation Solution
A data dimension reduction method based on maximizing the ratio sum for linear discriminant analysis, using an optimization problem and the alternating direction method of multipliers to obtain a projection matrix without matrix inversion, enhancing feature selection for classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional LDA method is used for dimension reduction, then the within-class similarity is maximized and between-class similarity is minimized, but the method tends to select features with small variances and low discriminant ability, and requires matrix inversion which fails when within-class covariance matrix is singular
Solution Approach 1:
The patent changes the optimization parameter from maximizing the ratio of between-class to within-class scatter (traditional LDA) to maximizing the sum of ratios for each dimension individually. This parameter transformation allows the method to avoid matrix inversion while maintaining or improving classification accuracy, especially in high-dimensional spaces where the within-class covariance matrix is often singular.
Solution Approach 2:
Instead of inverting the within-class covariance matrix as traditional LDA does, the patent inverts the approach by formulating the optimization problem to maximize the sum of ratios directly, eliminating the need for matrix inversion and thus avoiding the singularity problem entirely.
2Measurement precision
If traditional LDA method is used for dimension reduction, then the discriminant information is preserved, but the data processing flow becomes complex due to preprocessing requirements to ensure non-singularity of within-class covariance matrix
Solution Approach 1:
The patent extracts the essential discriminant information maximization goal from traditional LDA while removing the problematic matrix inversion step. By formulating the objective function to maximize the sum of ratios for each dimension, the method preserves discriminant information without requiring complex preprocessing to ensure matrix non-singularity.
3Loss of information
If original high-dimensional data is used directly for classification, then all information is retained, but the processing speed is low and classification accuracy is reduced due to redundant information
Solution Approach 1:
The patent applies partial action by selecting only the most discriminative dimensions for classification. By maximizing the sum of ratios for each dimension and selecting top k dimensions, the method retains the most important information while discarding redundant features, thus improving processing speed without significant loss of classification accuracy.
Data Source
AI summary
This invention relates to a data dimension reduction method based on maximizing a ratio sum for linear discriminant analysis, which belongs to the fields of image classification and pattern recognition. It includes constructing a data matrix, a label vector and a label matrix; calculating a within-class covariance matrix and a between-class covariance matrix; constructing the optimization problem based on maximizing the ratio sum for the linear discriminant analysis; using the alternating direction method of multipliers to obtain the projection matrix which can maximize an objective function. This invention establishes the objective function based on maximizing the ratio sum for the linear discriminant analysis to avoid the problem that the traditional linear discriminant analysis tends to select features with small variances and weak discriminating ability. It can select features which are more conducive to classification. Moreover, this method does not depend on the calculation of the inverse matrix of the within-class covariance matrix and does not require data preprocessing, which improves the adaptability of the data dimensionality reduction method to the original data feature.