Rule Learning Method for Unbiased Machine Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In machine learning, particularly in boosting algorithms, the learning time becomes excessively long when the number of rule candidates and generations is high, leading to biased learning and a lack of reproducibility due to dependence on initial bucket distribution.
Innovation Solution
A method where features are distributed into buckets based on weights, with rules selected based on maximum gain, and confidence values calculated and updated, ensuring unbiased learning and reproducibility by redistributing features across buckets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of rule candidates and generations is increased to improve learning accuracy, then the classification accuracy is improved, but the learning time becomes excessively long
Solution Approach 1:
The patent divides the feature set into multiple buckets, where each bucket contains a subset of features. Rules are generated separately for each bucket rather than considering all features simultaneously. This segmentation reduces the computational complexity per rule generation step while maintaining overall learning accuracy through aggregated results from multiple buckets.
2Productivity
If features are distributed into fixed initial buckets to speed up rule generation, then the learning speed is improved, but learning bias occurs and reproducibility is lost due to dependence on initial distribution
Solution Approach 1:
The patent introduces dynamic redistribution of features across buckets based on confidence values. Instead of using a fixed initial bucket distribution, the system periodically redistributes features to different buckets according to their confidence values from previous rule generations. This dynamic adjustment eliminates dependence on initial distribution, prevents learning bias, and ensures reproducible results across different runs.
Solution Approach 2:
The system uses confidence values from previously generated rules as feedback to guide the redistribution of features in subsequent iterations. Features with higher confidence values are redistributed to different buckets, while low-confidence features are retained or adjusted. This feedback mechanism ensures that the bucket distribution adapts to actual learning performance rather than relying on arbitrary initial assignments.
3Reliability
If the number of buckets is increased to reduce learning bias, then learning fairness is improved, but the device complexity and computational overhead increase
Solution Approach 1:
The patent uses a moderate number of buckets (e.g., 10 buckets) rather than attempting to create a bucket for every possible feature combination or using a very large number of buckets. This partial action approach achieves sufficient learning fairness by ensuring diverse feature coverage while avoiding the excessive computational overhead and complexity that would result from using too many buckets. The confidence-based redistribution compensates for the limited number of buckets.
Data Source
AI summary
A rule learning method in machine learning includes distributing features to a given number of buckets based on a weight of the features which are correlated with a training example; specifying a feature with a maximum gain value as a rule based on a weight of the training example from each of the buckets; calculating a confidence value of the specified rule based on the weight of the training example; storing the specified rule and the confidence value in a rule data storage unit; updating the weights of the training examples based on the specified rule, the confidence value of the specified rule, data of the training example, and the weight of the training example; and repeating the distributing, the specifying, the calculating, the storing, and the updating, when the rule and the confidence value are to be further generated.