Training Model Loss Normalization for Imbalanced Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models used for attribute prediction in accounting records face challenges in accurately identifying entities due to variations in accounting records and the presence of duplicate or missing datasets, leading to biased training and reduced effectiveness.
Innovation Solution
A method that determines a batch of training data by calculating similarity and difference measures for numerical representations of documents, normalizing these measures to mitigate the impact of duplicate datasets, and using a loss function to account for uneven data distribution, ensuring accurate entity identification without pre-filtering the training batch.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If training data includes duplicate or missing datasets, then the training process can proceed without pre-filtering, but the model accuracy and reliability are reduced due to biased training
Solution Approach 1:
The patent changes the parameter of loss function calculation by introducing a normalization factor (1/N_valid) that accounts for the number of valid datasets. This modifies how the loss is computed to penalize duplicate datasets appropriately, allowing training to proceed with imbalanced data while maintaining model accuracy.
Solution Approach 2:
The patent implements feedback through the loss function that automatically adjusts based on the distribution of datasets in the batch. By calculating similarity measures and normalizing by the number of valid datasets, the system provides feedback that guides the training process to handle duplicate and missing data without requiring manual pre-filtering.
2Reliability
If pre-filtering is applied to remove duplicate datasets, then model accuracy is improved, but the process complexity and time required increase
Solution Approach 1:
The patent performs preliminary calculation of similarity measures between datasets during the training process itself, rather than requiring separate pre-filtering. The similarity measures are computed and stored as part of the loss function calculation, eliminating the need for complex pre-processing steps to identify and remove duplicates.
Solution Approach 2:
The training system serves itself by automatically handling duplicate and missing datasets through the normalized loss function. The system self-adjusts to the presence of imbalanced data without requiring external intervention or complex pre-filtering mechanisms, simplifying the overall process while maintaining accuracy.
3Quantity of substance
If training data is imbalanced with duplicate datasets, then more comprehensive data coverage is achieved, but the loss function becomes skewed and training effectiveness is reduced
Solution Approach 1:
The patent changes the parameter of the loss function by introducing normalization that divides by the number of valid datasets (N_valid). This parameter change ensures that the loss function accounts for the imbalance in the data, preventing skewness and maintaining training effectiveness even when duplicate datasets are present.
Solution Approach 2:
The patent achieves equipotentiality by normalizing the loss function to account for the number of valid datasets in each batch. This ensures that all datasets, regardless of their frequency or duplication status, contribute equally to the training process, preventing any single dataset from having excessive influence on the model.
Data Source
AI summary
A method comprises determining a batch of training data for training a model comprising a plurality of datasets, each associated with a label and comprising at least one numerical representation of an example document; determining a number of classes of labels in the batch, each class associated with a unique attribute value; and determining a number of numerical representations associated with each class. The method comprises: determining a first similarity measure indicative of the similarity of the numerical representation to the other numerical representations associated with a same class; determining a second similarity measure for each of the other datasets associated with a different respective class, each second similarity measure indicative of the similarity of the numerical representation to each of the at least one numerical representations of the respective other datasets associated with respective different classes of the batch; determining a difference measure and a normalized difference measure.