Hierarchy-Preserving Loss for Multi-Label Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models for multi-label classification often fail to account for the hierarchical relationships between class labels, leading to inaccurate predictions and misclassifications, particularly when predicting parent classes over child classes.
Innovation Solution
The implementation of a hierarchical constraint loss function that ensures parent classes are predicted with higher probability than child classes, using constraints to limit the loss of child classes to a value less than or equal to that of parent classes, thereby guiding the training process to prioritize accurate identification of parent classes before child classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional multi-label classification methods are used without hierarchical constraints, then the model can predict all classes independently, but the predictions may violate hierarchical relationships (e.g., predicting child class without parent class)
Solution Approach 1:
The patent modifies the loss function parameters by introducing hierarchical constraints that adjust the loss calculation based on parent-child class relationships. The constrained loss function dynamically changes the optimization target to ensure parent class predictions are made before and with higher confidence than child class predictions, thereby maintaining hierarchical consistency without requiring architectural changes.
Solution Approach 2:
The patent introduces an intermediate loss function that acts as a mediator between the standard multi-label classification loss and the hierarchical consistency requirement. This intermediate loss function computes penalties for violations of hierarchical relationships and guides the model to learn proper hierarchical patterns during training, resolving the contradiction between independent prediction and hierarchical consistency.
2Reliability
If the model prioritizes predicting parent classes with higher probability, then hierarchical consistency is improved, but the model may struggle to capture fine-grained child class distinctions
Solution Approach 1:
The patent segments the training process into two distinct phases: first training on parent classes to establish hierarchical foundations, then training on child classes to capture fine-grained distinctions. This segmentation allows the model to learn coarse hierarchical patterns before refining detailed class discrimination, resolving the contradiction between hierarchical consistency and child class precision.
Solution Approach 2:
The patent applies preliminary action by pre-training the model on parent class predictions before proceeding to child class predictions. This preliminary training establishes the hierarchical framework and confidence thresholds needed for consistent parent class prediction, while subsequent fine-tuning on child classes maintains this hierarchy while improving granular precision.
3Measurement precision
If the loss function is modified to enforce hierarchical constraints, then prediction accuracy improves, but the training process becomes more complex and computationally intensive
Solution Approach 1:
The patent applies partial action by implementing hierarchical constraints only for parent-child class pairs where the hierarchy is explicitly defined, rather than enforcing constraints across all possible class combinations. This selective application maintains accuracy improvements while reducing the computational burden and training complexity associated with comprehensive hierarchical constraints.
Data Source
AI summary
A method implemented by one or more computing systems includes accessing a set of content objects, in which each content object of the set of content objects is pre-labeled with concepts of a plurality of concepts organized according to a hierarchical relationship. The method further includes training, by a machine-learning model, a classification model for classifying content objects within the set of content objects. Training the classification model includes determining, for each object, a plurality of classification values corresponding to the plurality of concepts, calculating a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object, utilizing a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values, and updating the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.