Incremental Machine Learning with Varying Feature Spaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing incremental machine learning methods face challenges in maintaining robustness when dealing with continuously varying feature spaces, are limited to binary class predictions, and suffer from high computational complexity, especially when handling multi-class predictions and adapting to new features.
Innovation Solution
A device and method that generates probability tables and conditional probability tables for incremental machine learning, extracts relevance and redundancy between features using mutual information, and updates weights based on these correlations to create a prediction model capable of handling varying feature spaces and multi-class predictions with minimal hyperparameter adjustments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a linear model is used for incremental machine learning in varying feature spaces, then the model can adapt to new instances effectively, but the model becomes vulnerable to maintaining knowledge constructed in the past (loss of robustness)
Solution Approach 1:
The patent applies parameter changes by dynamically adjusting the weighting of different features based on their relevance to the target feature. The system calculates relevance scores for each feature and uses these scores to weight their contribution to predictions, allowing the model to adapt to new features while maintaining stability through controlled parameter adjustment rather than complete model retraining.
Solution Approach 2:
The patent implements feedback mechanisms by continuously evaluating the performance of the incremental learning model and using this feedback to adjust feature weights and selection. The system monitors prediction accuracy and uses this information to refine which features are included and how they are weighted, creating a closed-loop system that balances adaptability with robustness.
2Ease of operation
If a linear regression-based method is used, then the method is valid for binary class prediction, but it requires separate techniques for multi-class prediction (increased complexity)
Solution Approach 1:
The patent achieves universality by developing a feature selection and weighting framework that works uniformly across both binary and multi-class classification problems. The relevance-based feature weighting mechanism is class-type agnostic, allowing the same core algorithm to handle any number of classes without requiring fundamentally different approaches, thus eliminating the need for separate techniques.
3Measurement precision
If many hyperparameters are optimized for the linear model, then the model can achieve good performance, but it takes much computation time (increased computational complexity)
Solution Approach 1:
The patent extracts and focuses on the most critical parameters for model performance by identifying and weighting only the features with highest relevance to the target. This selective approach eliminates the need to optimize numerous hyperparameters for all features, concentrating computational effort on the subset of features that matter most, thereby reducing overall computation time while maintaining accuracy.
Solution Approach 2:
The patent reduces the number of parameters requiring optimization by dynamically determining feature weights based on relevance scores rather than treating all features equally. This parameter transformation converts a high-dimensional optimization problem into a lower-dimensional one, where only the weights of relevant features need adjustment, significantly reducing computational burden.
4Adaptability or versatility
If the feature space varies continuously, then the model can handle new features, but the model loses knowledge constructed by existing data (catastrophic forgetting)
Solution Approach 1:
The patent applies parameter changes by adjusting feature weights incrementally rather than retraining the entire model. When new features are introduced, the system calculates their relevance and assigns appropriate weights, while maintaining the weights of existing features. This gradual parameter adjustment allows the model to incorporate new information without discarding previously learned patterns, preventing catastrophic forgetting.
Solution Approach 2:
The patent performs preliminary evaluation of feature relevance before fully integrating new features into the model. By pre-assessing the importance of new features through relevance calculation, the system can smoothly integrate them with existing features, ensuring that new information is incorporated in a way that complements rather than overrides previously learned knowledge.
Data Source
AI summary
The present invention relates to a device and method for incremental machine learning in a varying feature space. A device for machine learning according to the present invention includes a probability table generator configured to generate a probability table for a target feature of a dataset and a conditional probability table for each input feature of the dataset, based on the received dataset, a correlation extractor configured to extract relevance between the target feature and each of the input features and redundancy between the input features, based on the dataset, and a feature weight extraction and model generator configured to extract weights for each of the input features based on the relevance and the redundancy, and generate a prediction model based on the probability table for the target feature, the conditional probability table, and the weight.