Variable Subset Selection for Predictive Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning technologies require large amounts of data for training and prediction, leading to high storage and processing demands, and existing feature selection methods are suboptimal for selecting relevant variables for predicting specific events or characteristics of physical entities.
Innovation Solution
A data reduction device and method that selects a subset of variables based on a combination of mutual information and a penalty value, optimizing the selection to maintain predictive accuracy while reducing the number of variables, using a searching unit to find a local optimum of an optimization function that balances mutual information and the number of variables.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a huge amount of data is used for training and prediction, then predictive accuracy is improved, but storage requirements and processing power increase significantly
Solution Approach 1:
The patent extracts only the most relevant variables from the complete data set using an optimization function that combines mutual information with a penalty term. This selective extraction maintains predictive accuracy while dramatically reducing the volume of data that needs to be stored and processed, directly resolving the contradiction between predictive reliability and storage requirements
Solution Approach 2:
The patent transforms the data selection problem by changing the selection criterion from traditional variance-based methods to an optimization function that incorporates mutual information and a penalty parameter. This parameter change enables more effective variable selection that preserves predictive power while reducing data volume, addressing both sides of the contradiction
2Ease of operation
If traditional feature selection methods like PCA or LDA are used, then data processing is simplified, but predictive performance for specific events is suboptimal
Solution Approach 1:
The patent changes the fundamental parameter used for feature selection from variance (second-order statistic) to mutual information combined with a penalty term. This parameter change enables the method to capture non-linear relationships and specific event predictions more effectively while maintaining computational tractability through the optimization framework
Solution Approach 2:
The optimization function incorporates feedback through the penalty term that adjusts variable selection based on the relationship between reduced data and target events. This feedback mechanism allows the system to iteratively improve predictive performance for specific events while maintaining processing efficiency, resolving the contradiction between simplicity and performance
Data Source
AI summary
A data reduction device (150) for and a method of reducing a data set based on a subset of variables from a set of variables are provided. Instances of the plurality of variables comprise information to predict an instance of a further type of data. The device comprises a first data set unit (102), a second data set unit (104), a searching unit (110) and a data reduction unit (152). The first data set unit obtains a first set comprising tuples of instances of data. The second data set unit obtains a second set comprising instances of the further type of data. Each instance of the second set corresponds to one of the tuples of the first set. The searching unit obtains a reduced set of variables that represents an at least local optimum of an optimization function being a combination of a first mutual information value between the reduced first set and the second set and a penalty value being based on a number of variables in the reduced set of variables.