Feature Importance Diagram for Data Analysis Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data mining techniques generate a large number of features through arithmetic operations, leading to long calculation times and the need for manual analysis to identify important features for accurate data analysis, which is time-consuming and inefficient.
Innovation Solution
A data analyzing device and method that calculates the importance of features using a prediction model, displays high-importance features, and generates diagrams to visualize relationships and contributions to prediction accuracy, facilitating the identification of key features for knowledge acquisition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If a preliminarily defined series of arithmetic operators are comprehensively applied to generate new features, then a large amount of new features are automatically generated, but the calculation time in the subsequent model learning step becomes excessively long
Solution Approach 1:
The patent extracts and removes ineffective features from the generated feature set. The post-processing step identifies and eliminates features that do not contribute to prediction accuracy, retaining only the effective subset of features for model learning. This extraction principle resolves the contradiction by maintaining feature quantity while eliminating time-wasting ineffective features.
Solution Approach 2:
The patent applies different processing treatments to different features based on their effectiveness. Effective features are retained and used for model learning, while ineffective features are removed. This local quality differentiation allows the system to handle features selectively, improving calculation efficiency without sacrificing the benefits of automated feature generation.
2Productivity
If a large number of features are generated through automated processes, then manual feature design time is reduced, but it becomes difficult to intuitively understand relationships between features and objective variables, requiring additional manual analysis work
Solution Approach 1:
The patent implements feedback mechanisms that provide information about feature effectiveness and relationships. The post-processing step analyzes the generated features and provides feedback on their utility, enabling analysts to understand which features are effective and why. This feedback loop maintains automated generation efficiency while improving interpretability.
Solution Approach 2:
The patent introduces an intermediary analysis step between feature generation and model learning. This intermediary post-processing step bridges the gap by analyzing and explaining feature relationships, making the connection between automated feature generation and final model outcomes more transparent and understandable for analysts.
3Measurement precision
If trial and error by experienced analysts is used to design optimal features, then analysis accuracy can be improved, but the data analysis time becomes excessively long
Solution Approach 1:
The patent performs preliminary automated feature generation and filtering before the model learning step. By pre-generating and pre-filtering features to identify effective ones, the system prepares optimized input data in advance, reducing the need for time-consuming trial and error during the actual analysis while maintaining high accuracy through systematic feature selection.
Solution Approach 2:
The patent enables the system to automatically perform feature generation, evaluation, and selection without requiring continuous manual intervention. The automated pipeline generates features, evaluates their effectiveness, and selects optimal features independently, reducing reliance on experienced analysts while maintaining analysis accuracy and significantly reducing time investment.
Data Source
AI summary
A data analyzing device generates a diagram that shows a relationship between a first feature and an objective variable. The first feature is selected in accordance with an input of a user from among features having higher degrees of importance. The data analyzing device divides analysis target data into a plurality of clusters on the basis of values of the first feature, calculates a representative value of the objective variable of each of the clusters, extracts a second feature having a representative value of the objective variable, which is determined as having a significant difference relative to the representative value of the objective variable of the first feature, from at least one of the clusters. The data analyzing device generates a diagram that shows a relationship between the second feature and the objective variable.