Multi-Scale Extension for Dynamic Big Data Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for out-of-sample extension (OSE) of multidimensional data points in big data, such as diffusion maps and Nystrom methods, face high computational costs due to diagonalization of kernels, potential ill-conditioning, and sensitivity to parameter choices, making them inefficient for real-time classification and dimensionality reduction in dynamic datasets.
Innovation Solution
The proposed multi-scale extension (MSE) method uses a coarse-to-fine hierarchy of Gaussian kernel decomposition to determine a well-conditioned basis, enabling efficient single-scale and multi-scale OOSE, which reduces computational complexity and avoids numerical singularity, allowing for accurate mapping of newly arrived data points into a lower-dimensional space without re-learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If diffusion maps method is used for dimensionality reduction and out-of-sample extension, then data can be embedded into lower-dimensional space with preserved affinities, but computational cost becomes prohibitively high due to spectral decomposition of large MDP matrices
Solution Approach 1:
The patent segments the large MDP matrix into smaller blocks and applies blockwise spectral decomposition. Instead of decomposing the entire n×n matrix at once, the method divides it into manageable blocks that can be processed independently or in parallel, significantly reducing computational complexity while preserving the essential spectral information needed for accurate embedding.
Solution Approach 2:
The patent uses random projection matrices as temporary, disposable objects to approximate the spectral decomposition. These random matrices are generated once and used to project the high-dimensional data into a lower-dimensional space, providing a computationally inexpensive alternative to full spectral decomposition that still captures the essential data structure.
2Adaptability or versatility
If traditional OOSE methods are used to map newly arrived data points, then dimensionality reduction can be applied to training data, but the methods suffer from ill-conditioning and sensitivity to parameter choices
Solution Approach 1:
The patent performs preliminary spectral decomposition on the training data to obtain the eigenvectors and eigenvalues that form the basis of the embedded space. This preliminary action creates a stable reference frame that can be used for mapping new data points without requiring re-decomposition, ensuring numerical stability and reducing sensitivity to parameter choices when extending to out-of-sample data.
Solution Approach 2:
The patent introduces regularization parameters and adjusts the spectral cutoff criteria to improve the conditioning of the decomposition problem. By carefully selecting and tuning these parameters, the method achieves better numerical stability while maintaining the ability to extend to new data points effectively.
3Ease of operation
If full spectral decomposition is performed on combined training and test sets, then classification can be performed in lower-dimensional space, but the computational overhead increases significantly for dynamic datasets with continuously arriving data
Solution Approach 1:
The patent implements a dynamic dimensionality reduction approach where the spectral decomposition is performed only on the training set initially. As new data points arrive, they are projected into the pre-computed embedded space using the stored eigenvectors, allowing the system to adapt dynamically to new data without repeating the expensive decomposition process, thus reducing processing time for continuously arriving data.
Solution Approach 2:
The patent performs the computationally intensive spectral decomposition in advance on the training data before classification is needed. This preliminary action creates a ready-to-use embedded space that can quickly accommodate new data points through simple projection operations, significantly reducing the time required for real-time classification of dynamically arriving data.
Data Source
AI summary
A method for classification of a newly arrived multidimensional data point (MDP) in a dynamic data uses multi-scale extension (MSE). The multi-scale out-of-sample extension (OOSE) uses a coarse-to-fine hierarchy of the multi-scale decomposition of a Gaussian kernel that established the distances between MDPs in a training set to find the coordinates of newly arrived MDPs in an embedded space. A well-conditioned basis is first generated in a source matrix of MDPs. A single-scale out-of-sample extension (OOSE) is applied to the newly arrived MDP on the well-conditioned basis to provide coordinates of an approximate location of the newly arrived MDP in an embedded space. A multi-scale OOSE is then applied to the newly arrived MDP to provide improved coordinates of the newly arrived MDP location in the embedded space.