Fractal Dimension Estimation for Robust Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models struggle to accurately estimate the latent feature dimensionality of high-dimensional datasets, leading to potential learning of spurious or non-causal relationships due to noise in the data.
Innovation Solution
The method involves monitoring N observed variables of a training dataset, translating them into equisized bin indexes to generate hypercells, and computing the fractal dimension based on data coverage to estimate the fundamental dimensionality of the dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If machine learning models use high-dimensional datasets with many observed variables, then the model can capture more potential relationships and improve prediction accuracy, but the model may learn spurious or non-causal relationships due to noise and over-specification
Solution Approach 1:
The patent extracts and removes noise and spurious relationships from high-dimensional datasets by computing the fundamental dimensionality of latent features. This extraction process separates meaningful signal from noise, allowing the model to focus on genuine causal relationships while discarding irrelevant dimensions that contribute to instability and spurious correlations.
Solution Approach 2:
The patent changes the parameter of dimensionality from the observed high-dimensional space to the fundamental latent dimensionality computed through fractal analysis. By transforming the dimensionality parameter from N observed variables to d fundamental latent features, the model achieves both high prediction accuracy and stability by operating in the true underlying dimension space rather than the noisy observed space.
2Measurement precision
If the model complexity is increased to capture nonlinear relationships, then the model can better fit the data, but the model becomes more sensitive to noise and less robust
Solution Approach 1:
The patent performs preliminary action by computing the fundamental dimensionality and determining the appropriate model complexity before training the machine learning model. This advance determination of the fundamental dimension d guides the selection of model architecture and complexity, ensuring that the model is neither under-specified nor over-specified, thereby achieving robust nonlinear fitting without excessive sensitivity to noise.
Solution Approach 2:
The patent changes the complexity parameter from being arbitrarily high to being precisely matched to the fundamental dimensionality d of the latent features. By adjusting the model complexity parameter to align with the computed fundamental dimension, the model achieves optimal balance between fitting accuracy and robustness, capturing nonlinear relationships without overfitting to noise.
3Reliability
If dimensionality reduction techniques are applied to reduce noise, then model stability improves, but the risk of losing important information increases
Solution Approach 1:
The patent performs preliminary action by computing the fundamental dimensionality d before applying dimensionality reduction. This advance knowledge of the true latent dimension guides the reduction process, ensuring that exactly d dimensions are retained while reducing from N observed dimensions. This preliminary determination prevents both under-reduction (retaining noise) and over-reduction (losing information), achieving stable models with complete information preservation.
Solution Approach 2:
The patent replaces conventional dimensionality reduction techniques (which are mechanical and arbitrary) with a physics-based fractal dimension computation that naturally identifies the fundamental dimension d. This substitution replaces arbitrary dimension selection with a scientifically grounded measurement, ensuring that dimensionality reduction preserves all meaningful information while eliminating only the noise dimensions.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach allows for the estimation of robust latent feature dimensionality, improving the stability and accuracy of machine learning models by avoiding over-specification and capturing the correct nonlinear relationships.
Implementation Method 1
computing a fractal dimension of the equisized hypercube phase space based on count of cells with data coverage of at least one data point
Data Source
Figure 1
Figure 2(a)~2(d)
Figure 3
AI summary
Computer-implemented methods and systems for quantifying appropriate machine learning model complexity corresponding to training dataset are provided. The method comprises monitoring, using one or more processors, N observed variables, vi through vN, of a training dataset for a machine learning model; translating the N observed variables into m equisized bin indexes which generate mN possible equisized hypercells to estimate a fundamental dimensionality for the dataset; generating one or more samples by assigning a record in the dataset with numbers j through k as set id; generating a merged sample Si, for one or more values of the set id i, where i goes from j to k; and computing a fractal dimension of the equisized hypercube phase space based on count of cells with data coverage of at least one data point.