Mutual Information Feature Selection for Predictive Data Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning technologies face inefficiencies in data reduction for predictive tasks, as they often require large amounts of storage and processing power, and current feature selection methods like PCA and LDA are suboptimal in retaining predictive information.
Innovation Solution
A data reduction device that selects a subset of variables based on mutual information and a penalty value to minimize the number of variables while maintaining predictive capability, using clustering and iterative search methods like simulated annealing to optimize the selection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the whole data history is used for training and prediction, then the predictive accuracy is improved, but the storage requirement and processing power increase significantly
Solution Approach 1:
The patent extracts only the most relevant variables from the complete data history using mutual information analysis. Instead of storing and processing all historical data, the system identifies and retains only those variables that have the highest predictive value for the target event, thereby reducing storage requirements while maintaining prediction accuracy.
Solution Approach 2:
The patent transforms the approach from using all historical data to using a selected subset of variables by changing the parameter of data representation. Mutual information is used to quantify the relevance of each variable, and only variables exceeding a certain information threshold are retained, effectively changing the data parameter from comprehensive to optimized.
2Reliability
If the whole data history is processed to find similarities, then the prediction quality is improved, but the processing power and time consumption increase
Solution Approach 1:
The patent extracts only the essential variables that contain predictive information, eliminating redundant data before processing. This extraction based on mutual information reduces the computational burden of finding similarities in historical data while preserving the quality of predictions.
Solution Approach 2:
Instead of processing the complete data history, the patent applies partial action by processing only the subset of variables that are most relevant to the prediction task. This selective processing achieves sufficient prediction quality without the excessive computational cost of analyzing all available data.
3Device complexity
If traditional feature selection methods like PCA or LDA are used, then the data dimensionality is reduced, but the predictive information retention is suboptimal
Solution Approach 1:
The patent changes the parameter used for feature selection from variance-based metrics (PCA, LDA) to mutual information-based metrics. This parameter change allows the system to identify variables that are directly relevant to the target prediction rather than those that simply explain the most variance, thereby reducing dimensionality while better preserving predictive information.
Solution Approach 2:
The patent introduces mutual information as an intermediary measure to evaluate the relationship between variables and the target event. This intermediary metric serves as a bridge between the input variables and the prediction task, enabling more accurate feature selection compared to direct variance-based methods.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data reduction device (150) for and a method of reducing a data set based on a subset of variables from a set of variables are provided. Instances of the plurality of variables comprise information to predict an instance of a further type of data. The device comprises a first data set unit (102), a second data set unit (104), a searching unit (110) and a data reduction unit (152). The first data set unit obtains a first set comprising tuples of instances of data. The second data set unit obtains a second set comprising instances of the further type of data. Each instance of the second set corresponds to one of the tuples of the first set. The searching unit obtains a reduced set of variables that represents an at least local optimum of an optimization function being a combination of a first mutual information value between the reduced first set and the second set and a penalty value being based on a number of variables in the reduced set of variables.