Machine Learning Apparatus Pseudo-Anomaly Data Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in automatically assigning ground truth labels when the number of data points belonging to a certain class is small, leading to incorrect clustering and label assignment.
Innovation Solution
A machine learning apparatus generates pseudo-anomaly data by combining normal data with anomaly data through a linear combination, allowing for correct clustering and label assignment even when the number of anomaly data points is small, by adjusting the distribution of anomaly data to match true anomaly data distributions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If clustering is performed based on actual data distribution, then clustering accuracy is improved, but the method fails when the number of data points belonging to a certain class is small
Solution Approach 1:
The patent generates pseudo-anomaly data by copying and transforming normal data through linear combination with anomaly data. This creates synthetic data samples that replicate the characteristics of the minority class, enabling accurate clustering even when actual anomaly data points are scarce. The pseudo-data serves as a copy that preserves the essential distribution patterns needed for effective clustering.
Solution Approach 2:
The patent transforms the parameter distribution of normal data by applying linear combination with anomaly data using specific weighting coefficients. This parameter transformation generates pseudo-anomaly data with adjusted statistical properties that match the target anomaly distribution, allowing the clustering algorithm to work effectively despite limited actual anomaly samples.
2Productivity
If ground truth labels are automatically assigned through clustering, then productivity is improved, but measurement precision deteriorates when data distribution is unbalanced
Solution Approach 1:
The patent performs preliminary action by generating pseudo-anomaly data before the clustering process. This synthetic data preparation ensures that the feature space is properly populated with representative samples from both classes, enabling the subsequent clustering algorithm to assign ground truth labels accurately without manual intervention, thus maintaining both efficiency and precision.
Solution Approach 2:
The pseudo-anomaly data acts as an intermediary between the imbalanced actual data and the clustering algorithm. It mediates the clustering process by providing the missing distributional information that would otherwise be unavailable due to the scarcity of actual anomaly data, enabling accurate label assignment while maintaining automated efficiency.
3Reliability
If the machine learning model is retrained with new training data, then detection performance is maintained, but loss of time occurs due to repeated training
Solution Approach 1:
The patent applies partial action by generating only the necessary pseudo-anomaly data required to maintain detection performance, rather than collecting and training on all possible data. This selective data generation approach maintains model reliability while minimizing the time investment required for retraining, as the synthetic data can be generated quickly compared to manual data collection and annotation.
Data Source
AI summary
A non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute a process including: inputting pieces of data to a machine learning model, and acquiring prediction results of the pieces of data; generating one or more pieces of data based on first data of which the prediction result indicates a first group among the pieces of data; executing clustering of the pieces of data and the one or more pieces of data based on features of the pieces of data and the one or more pieces of data, which are obtained based on a parameter of the machine learning model; and updating the parameter of the machine learning model based on training data including the pieces of data and the one or more pieces of data for which results of the clustering are used as ground truth labels.