Bayesian Classifier Feature Partitioning for Accuracy and Speed
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning systems, such as Naive Bayes Classifiers and Bayesian networks, face challenges in handling feature dependencies, especially in large data sets, due to the need for extensive domain knowledge, computationally complex structure-learning procedures, and the risk of over-fitting with limited data.
Innovation Solution
The system partitions sets of features for a Bayesian classifier by identifying input subtuples with a minimum threshold support count, computing class entropy or predictive power scores, and constructing a partition list that accounts for feature dependencies, using a tuple instance count data structure to improve classification accuracy and efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If Bayesian networks are used to model feature dependencies, then classification accuracy is improved, but computational complexity and device complexity increase significantly
Solution Approach 1:
The patent segments the feature set into multiple independent partitions, where each partition is processed separately by a Naive Bayes classifier. This segmentation allows the system to capture feature dependencies within each partition while avoiding the computational complexity of modeling all feature interactions globally, thus resolving the contradiction between accuracy and complexity.
Solution Approach 2:
The patent introduces a new dimension of partitioning by transforming the single feature space into multiple partitioned feature spaces. By organizing features into discrete partitions and processing them separately, the system achieves dependency modeling without the computational burden of traditional Bayesian networks, effectively resolving the complexity-accuracy tradeoff.
2Measurement precision
If structure-learning algorithms are used to learn feature dependencies, then classification accuracy is improved, but the problem becomes computationally complex and requires extensive training data
Solution Approach 1:
The patent performs preliminary partitioning of features into independent sets before classification. This preliminary organization eliminates the need for complex structure-learning algorithms during training, as the partitioning structure is established in advance and allows straightforward Naive Bayes classification, thus improving training efficiency while maintaining accuracy.
3Measurement precision
If feature dependencies are modeled using undirected graphical models, then classification accuracy is improved, but the system requires extensive domain knowledge and laborious hand-coding
Solution Approach 1:
The patent enables the system to automatically partition features into dependency groups without requiring manual domain knowledge or hand-coding of graphical models. The automatic partitioning process allows the system to self-organize feature dependencies, eliminating the laborious hand-coding requirement while maintaining the accuracy benefits of dependency modeling.
4Productivity
If Naive Bayes Classifier is used to assume feature independence, then computational efficiency is improved, but classification accuracy deteriorates when feature dependencies exist
Solution Approach 1:
The patent segments features into multiple partitions and applies Naive Bayes classification to each partition separately. This segmentation preserves the computational efficiency of Naive Bayes while capturing feature dependencies within each partition, thus resolving the contradiction between speed and accuracy by maintaining independence assumptions locally while allowing dependencies globally.
Data Source
AI summary
The technology disclosed relates to methods for partitioning sets of features for a Bayesian classifier, finding a data partition that makes the classification process faster and more accurate, while discovering and taking into account feature dependence among sets of features in the data set. It relates to computing class entropy scores for a class label across all tuples that share the feature-subset and arranging the tuples in order of non-decreasing entropy scores for the class label, and constructing a data partition that offers the highest improvement in predictive accuracy for the data set. Also disclosed is a method for partitioning a complete set of records of features in a batch computation, computing increasing predictive power; and also relates to starting with singleton partitions, and using an iterative process to construct a data partition that offers the highest improvement in predictive accuracy for the data set.