Deep Random Forest Model for Small Data Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks require large amounts of data for training and have complex hyperparameter tuning, making them unsuitable for small-scale data sets and computationally intensive, while existing deep learning methods like gcForest face performance issues due to high computational demands and inefficient use of training data.
Innovation Solution
The development of a machine learning model using deep random forests with a reduced set of hyperparameters, where each layer of random forests is grown from a bootstrapped data set, with Out-of-bag predictions appended to create new feature information, allowing for automatic model expansion until accuracy ceases to improve significantly, thereby reducing computational resources and eliminating the need for cross-validation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are used for data analysis, then learning accuracy can be achieved, but the model complexity and hyperparameter tuning requirements increase significantly
Solution Approach 1:
The patent segments the deep learning model into multiple cascade layers of random forests, where each layer processes features independently and contributes to the final prediction. This segmentation reduces the complexity of tuning a single large neural network while maintaining deep learning capabilities through layered feature processing.
Solution Approach 2:
The patent changes the parameter structure by using random forests with fixed hyperparameters (number of trees, maximum depth) instead of neural networks with numerous tunable hyperparameters. This parameter standardization eliminates the need for extensive hyperparameter tuning while preserving model accuracy through the cascade architecture.
2Quantity of substance
If deep neural networks are trained on small data sets, then model development is possible, but the required data quantity becomes insufficient for effective training
Solution Approach 1:
The patent applies preliminary feature processing by growing random forest trees on bootstrap samples before final prediction. This preliminary action creates robust feature representations that work effectively with small data sets, eliminating the need for large quantities of training data required by traditional deep neural networks.
Solution Approach 2:
The patent uses bootstrap sampling to create multiple copies of the training data for growing individual trees in the random forest ensemble. This copying mechanism allows effective utilization of small data sets by generating diverse training samples through random sampling with replacement, maintaining model performance without requiring additional data.
3Measurement precision
If gcForest uses cross-validation for feature generation, then model accuracy can be improved, but computational resources and training time increase significantly
Solution Approach 1:
The patent extracts the cross-validation step from the training process by using out-of-bag (OOB) samples for validation. This extraction eliminates the need for separate validation sets and repeated cross-validation folds, significantly reducing training time while maintaining accuracy through OOB error estimation.
Solution Approach 2:
The patent implements self-service validation where each random forest tree validates itself on its out-of-bag samples during the training process. This self-validation mechanism provides accurate model assessment without requiring external validation data or additional computational passes, reducing training time while maintaining accuracy.
4Adaptability or versatility
If multiple random forest learners are used in each layer of gcForest, then model coverage increases, but the number of learners and computational cost increase with data set size
Solution Approach 1:
The patent makes each random forest learner universal by designing it to handle the entire data set through bootstrap sampling, rather than requiring multiple specialized learners for different data partitions. This multi-functional approach maintains model coverage while reducing the total number of learners needed, improving computational efficiency.
Data Source
AI summary
Automated development and training of deep forest models for analyzing data by growing a random forest of decision trees using data, determining Out-of-bag (OOB) predictions for the forest, appending the OOB predictions to the data set, and growing an additional forest using the data set including the appended OOB predictions, and combining the output of the additional forest, then utilizing the model to classify data outside the training data set.