Dimensionality Reduction Model Augmented Space
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current dimensionality reduction methods, such as PCA and t-SNE, face limitations in preserving local structure and densities, and are not scalable for high-dimensional output spaces, making them inefficient for tasks requiring linear separability and out-of-sample generalization.
Innovation Solution
A dimensionality reduction model is trained using positive pairs of proximate vectors in a high-dimensional space, with a projector to an augmented space, optimizing similarity preservation and redundancy reduction losses to generate lower-dimensional representations that are invariant to distortions and maintain local neighborhood structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If PCA is used for dimensionality reduction, then linear projection is achieved, but local structure and densities are poorly preserved
Solution Approach 1:
The patent projects lower dimensional vectors to an augmented dimensional representation space (d' > d) before computing loss. This dimensional transformation enables the model to capture non-linear local structures while maintaining the benefits of linear projection in the original lower dimensional space, resolving the contradiction between linearity and local structure preservation.
Solution Approach 2:
The patent introduces an intermediate augmented dimensional space as a mediator between the lower dimensional output space and the loss computation. This intermediate space allows the model to preserve local structures through non-linear projections while the original lower dimensional space maintains linearity for efficient computation.
2Loss of information
If t-SNE is used for dimensionality reduction, then local structure is better encoded, but densities and distances are poorly preserved and scalability is limited
Solution Approach 1:
The patent uses an augmented dimensional representation space (d' > d) as an intermediate step for loss computation. This allows the model to achieve scalability to high-dimensional output spaces while still preserving local structures through the non-linear transformations in the augmented space.
Solution Approach 2:
The patent separates the dimensionality reduction function from the loss computation by introducing an intermediate projection step. The dimensionality reduction operates in the scalable lower dimensional space, while the local structure preservation is achieved through the separate projection to augmented space for loss calculation, enabling both scalability and local structure preservation.
3Productivity
If dimensionality reduction is applied, then computational cost and training time are reduced, but information loss occurs
Solution Approach 1:
The patent introduces an augmented dimensional representation space as an intermediary that temporarily expands the feature space to capture non-linear local structures. This intermediate expansion allows the model to preserve more information during the dimensionality reduction process, while the final output remains in the computationally efficient lower dimensional space.
Solution Approach 2:
The patent transforms the problem by moving to an augmented dimensional space (d' > d) for loss computation, which provides additional degrees of freedom to preserve information. This dimensional change enables the model to capture complex local structures without increasing the final output dimensionality, thus maintaining computational efficiency.
Data Source
AI summary
Methods and systems for training a dimensionality reduction model. Pairs of proximately located training vectors in a higher dimensional space are generated. Lower dimension vector pairs are generated by encoding first and second training vectors using the dimensionality reduction model, and augmented dimension vector pairs are generated by projecting to an augmented dimensional representation space having a greater number of dimensions. A similarity preservation loss and a redundancy reduction loss are computed and used to optimize parameters of the dimensionality reduction model.