AI Dataset Refinement via Perturbed Data Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies fail to accurately interpret machine learning model decisions due to the presence of perturbed data, leading to inaccurate analysis.
Innovation Solution
A computer-implemented method that generates a plurality of data points to interpret AI model decisions, identifies a subset that satisfies specific constraints, applies a linear model to these data points, and generates insights, while deleting data points that do not meet the constraints, thereby refining the dataset for accurate representation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If perturbed data is used to interpret machine learning model decisions, then data generation is simple, but analysis accuracy deteriorates
Solution Approach 1:
The patent extracts and removes perturbed data points from the dataset by comparing generated data against original training data characteristics. This separation isolates the harmful perturbed data while retaining useful data for accurate model interpretation.
Solution Approach 2:
The patent converts the harmful effect of perturbed data into a benefit by using the perturbation process itself to generate additional training data. The perturbed data points that satisfy certain conditions are retained and used to enhance model robustness, while only clearly harmful points are removed.
2Quantity of substance
If all generated data points are retained for analysis, then data quantity is sufficient, but data quality deteriorates due to inclusion of perturbed data
Solution Approach 1:
The patent segments the generated data points into distinct categories: original training data points, perturbed data points that satisfy conditions, and perturbed data points that do not satisfy conditions. This segmentation allows selective retention of high-quality data while maintaining sufficient data quantity through the linear model application.
Solution Approach 2:
The patent changes the state of data points by applying a linear model that transforms original data points into new data points with modified characteristics. This parameter change creates a refined dataset that maintains quantity while improving quality through systematic transformation.
3Productivity
If perturbed data is not removed, then data processing is fast, but interpretation accuracy deteriorates
Solution Approach 1:
The patent performs preliminary filtering of perturbed data points before the main analysis process. By identifying and removing harmful perturbed data in advance through comparison with original training data characteristics, the subsequent interpretation process works only with clean data, maintaining both speed and accuracy.
4Device complexity
If a linear model is applied to all data points, then model complexity is high, but computation efficiency deteriorates
Solution Approach 1:
The patent extracts only the necessary subset of data points that satisfy specific conditions before applying the linear model. This extraction reduces the input size for the linear model from all generated data points to only those that contribute meaningfully to accurate interpretation, significantly improving computation efficiency while maintaining model complexity at acceptable levels.
Data Source
AI summary
A computer-implemented method for refining dataset to accurately represent output of an artificial intelligence model includes generating a plurality of data points used to interpret a decision of an artificial intelligence model. A subset of data points from the generated plurality of data points satisfying one or more constraints is identified. A linear model is applied on the identified subset of data points satisfying the one or more constraints. One or more insights illustrating the decision of the artificial intelligence model is generated.