Progressive Multi-Level Training With Logit Masking for Label Consistency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical taxonomy classification frameworks face issues with label inconsistency and error propagation, particularly in retail scenarios, and are not effectively applicable to few-shot datasets.
Innovation Solution
A progressive multi-level training framework with a logit-masking strategy that trains models separately for each level, using a logit masking technique to restrict learning to relevant classes and incorporate global representation from previous levels, addressing label inconsistency and reducing error propagation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a parent node-based local classifier is trained to mitigate label inconsistency, then label consistency is improved, but computational cost increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-training a global classifier to establish a prior understanding of the hierarchical structure and label relationships. This pre-trained global model serves as a foundation that guides subsequent local classifier training, enabling them to learn more efficiently from fewer examples while maintaining label consistency without requiring excessive computational resources for full supervised training of each local classifier.
Solution Approach 2:
The patent changes parameters by dynamically adjusting the number of training samples per class based on the hierarchical level and parent node predictions. Instead of using a fixed number of samples for all local classifiers, the system adaptively selects sample sizes that optimize both computational efficiency and classification accuracy, reducing unnecessary computational overhead while maintaining reliability.
2Productivity
If existing hierarchical training frameworks are applied to few-shot datasets, then training efficiency is improved, but classification performance deteriorates due to insufficient data
Solution Approach 1:
The patent applies preliminary action by pre-training a global classifier on the full dataset before training local classifiers on few-shot data. This pre-training establishes a strong prior that compensates for the insufficient data available during few-shot training, enabling local classifiers to achieve better performance despite limited samples while maintaining training efficiency through the hierarchical structure.
Solution Approach 2:
The patent implements feedback by using predictions from the global classifier and parent node local classifiers to guide subsequent training stages. The system continuously refines local classifier training based on feedback from higher-level predictions and adjusts training samples dynamically, improving classification performance in few-shot scenarios while maintaining efficient training through iterative refinement rather than exhaustive training.
3Productivity
If level-based local classifiers are trained separately for each level, then training speed is improved, but label inconsistency problems occur
Solution Approach 1:
The patent merges the global classifier with local classifiers in a unified hierarchical framework where the global model provides overarching guidance while local models handle level-specific tasks. This combination allows fast parallel training of local classifiers while maintaining label consistency through the global model's hierarchical constraints and adaptive sample selection that ensures consistent labeling across all levels.
Solution Approach 2:
The patent uses feedback mechanisms where the global classifier's predictions and the hierarchical structure provide continuous guidance to local classifiers during training. This feedback ensures that even though local classifiers are trained separately for speed, they maintain label consistency by aligning with the global hierarchical framework through adaptive sample selection and loss function design that penalizes inconsistent predictions.
Data Source
AI summary
The embodiments of present disclosure address unresolved problems of label inconsistency, where outputs of different levels create impossible combinations, and error propagation from previous level outputs can significantly impact its performance. Embodiments provide a method and system for a Progressive Multi-level Training framework with a Logit-masking strategy (PMTL) for a retail taxonomy classification. PMTL enables neural network models to be trained separately for each level to reduce error propagation problems. To further enhance the model's performance at each level and get the label-wise constraint from the previous level, the global representation from model of previous level is augmented. Further, a logit masking strategy is used to restrict model(s) to learning only relevant classes through part of final classification layer, thereby addressing label inconsistency issue, and incorporating benefit of parent node-based local classifier. This framework is generalized irrespective of dataset size and is configured for attaching to any hierarchical classification network.