Machine Learning Data Relationship Mining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Analyzing large data sets is inefficient due to the generation of numerous rules, many of which are not useful, requiring expert knowledge and time to identify relevant ones, and machine learning approaches can lead to inaccurate results if the sample set is not diverse enough.
Innovation Solution
A method is provided to preprocess data by selecting relevant attributes, removing redundant relationships, and using a machine learning classifier trained on a diverse sample set to automatically label and classify data relationships, with user interfaces to assist in the process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all relationships are analyzed manually to ensure accuracy, then labeling precision is improved, but time consumption and resource usage increase significantly
Solution Approach 1:
The patent applies partial action by having users manually label only a diverse subset of relationships (e.g., 100-500 samples) rather than all relationships. This partial manual labeling provides sufficient training data for the machine learning classifier to achieve high accuracy while dramatically reducing time consumption compared to manual labeling of all relationships.
Solution Approach 2:
The patent replaces the mechanical system of manual labeling with a machine learning-based automated classification system. The classifier is trained on manually labeled diverse samples and then automatically classifies remaining relationships, substituting human effort with computational processing that is both faster and scalable.
2Productivity
If a small sample set is used for training, then training time is reduced, but classification accuracy deteriorates due to insufficient diversity
Solution Approach 1:
The patent changes the parameter of sample selection from random or uniform sampling to distance-based diverse sampling. By selecting samples that maximize diversity (e.g., using clustering or distance metrics in the feature space), the training set achieves better representativeness with fewer samples, improving classification accuracy without increasing training size.
Solution Approach 2:
The patent performs preliminary action by pre-selecting a diverse training subset before the actual training process. This pre-selection ensures that the training data is optimally representative of the full relationship space, allowing the model to achieve high accuracy with minimal training data and reduced training time.
3Reliability
If redundant relationships are included in analysis, then comprehensive coverage is maintained, but processing complexity and resource usage increase
Solution Approach 1:
The patent extracts and removes redundant relationships from the dataset before training and analysis. By identifying and eliminating duplicate or highly similar relationships (using similarity metrics and clustering), the system maintains coverage of all meaningful relationship types while reducing the total volume of data to be processed, thereby lowering computational complexity.
Solution Approach 2:
The patent discards redundant relationships that do not add new information, while preserving the essential diversity of relationship types. This selective discarding reduces processing load without losing important relationship patterns, as the diverse subset retains representativeness of the full relationship space.
Data Source
AI summary
Techniques and solutions are described for determining relationships in data with improved efficiency, including computing resource use. A plurality of attributes are selected for analysis. The attributes can be processed, such as to facilitate relationship determination. Relationships between attribute values are determined. Redundant relationships can be removed. Distances are determined between relationships and used to select a sample of relationships. The sample is labelled by a user and used to train a machine learning classifier. The machine learning classifier labels determined relationships.