Unbalanced Data Processing Device Using Reference Value Loss
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models face reliability issues due to unbalanced data distributions, where training data with uneven distributions lead to biased weight models, particularly in classification and regression tasks, resulting in poor prediction reliability.
Innovation Solution
A data processing device and method that calculates a reference value based on training and target data, calculates distances between these values and prediction data, and updates the weight model using a loss value to reflect the unbalanced distribution, thereby improving prediction reliability without distorting the training data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional machine learning is applied to unbalanced data, then training is simplified, but prediction reliability deteriorates due to biased weight models
Solution Approach 1:
The patent changes the loss calculation parameters by introducing a reference value (mode, median, or mean of training data) and calculating a first distance between target data and this reference value. This modifies the traditional loss function to account for data distribution characteristics, allowing the model to learn from unbalanced data without being biased toward majority classes while maintaining training simplicity.
2Reliability
If training data distribution is distorted to balance classes, then prediction reliability improves, but data integrity deteriorates
Solution Approach 1:
The patent introduces a reference value as an intermediary element that represents the central tendency of the training data distribution. Instead of directly modifying the training data distribution, the reference value mediates the loss calculation to account for unbalanced distribution, thereby improving prediction reliability while preserving the original data integrity and distribution characteristics.
3Productivity
If conventional loss calculation is used, then training efficiency is maintained, but model accuracy deteriorates on unbalanced data
Solution Approach 1:
The patent implements a feedback mechanism where the first distance (between target data and reference value) is incorporated into the loss calculation. This feedback loop allows the model to continuously adjust its predictions based on how far they deviate from the reference value, improving prediction accuracy on unbalanced data while maintaining training efficiency through a straightforward loss modification.
Data Source
AI summary
Disclosed is a data processing device that processes unbalanced data, which includes a preprocessor that calculates a reference value based on a plurality of training data and target data, and a learner that applies the plurality of training data to a first weight model to generate first prediction data, calculates a loss value based on a first distance between the target data and the reference value and a second distance between the target data and the first prediction data, and updates the first weight model based on the calculated loss value, and the plurality of training data and the target data have an unbalanced distribution.