Multi-Head Regression Metrics for Null-Label Loss Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for training multi-head machine learning models fail to accurately account for incompletely-labeled training data, leading to inaccurate loss and accuracy computations due to null ground truth labels.
Innovation Solution
Implement a masking technique in loss and accuracy functions to disregard loss and accuracy determinations for output variables with null ground truth labels, using masking values to replace null labels and predictions, and compute loss and accuracy based on available data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If existing loss functions are used to compute loss based on comparing output predictions to ground truth labels, then the loss computation is simple, but the loss values become inaccurate when ground truth labels include null values for some output variables
Solution Approach 1:
The loss function applies different treatment to different output variables based on whether their ground truth labels are present or null. For each output variable, the function checks if the ground truth label is not null before including it in the loss calculation, thereby applying local quality control to handle missing data appropriately while maintaining overall computation simplicity.
2Ease of manufacture
If existing accuracy determination techniques are used without accounting for null ground truth labels, then the accuracy computation is straightforward, but the accuracy values become incorrect when training data includes incompletely-labeled instances
Solution Approach 1:
The accuracy determination applies selective evaluation across different output variables. For each variable, it checks whether the ground truth label is available before evaluating prediction accuracy. This local quality approach ensures that only complete variable evaluations contribute to the overall accuracy metric, preventing null values from skewing the results while maintaining computational straightforwardness.
3Adaptability or versatility
If incompletely-labeled training data is used without masking techniques, then data utilization is maximized, but training accuracy and model performance deteriorate due to inaccurate loss values
Solution Approach 1:
The loss function changes its behavior based on the presence or absence of ground truth label parameters. When a ground truth label is null, the function changes its computation mode to exclude that variable from the loss calculation. This parameter-based adaptive approach allows the system to utilize incompletely-labeled data effectively while maintaining training reliability through appropriate loss computation adjustments.
Data Source
AI summary
Aspects of the present disclosure provide techniques for multi-head machine learning model training. Embodiments include receiving training data comprising training inputs associated with ground truth labels corresponding to a plurality of variables, wherein the ground truth labels include a null value for a given variable of the plurality of variables. Embodiments include providing the training inputs to a machine learning model that is configured to generate predictions corresponding to the plurality of variables. Embodiments include receiving the predictions from the machine learning model in response to the training inputs. Embodiments include evaluating a loss function that compares the ground truth labels to the predictions and uses a masking value to disregard loss that corresponds to the given variable. Embodiments include updating one or more parameters of the machine learning model based on the evaluating of the loss function.