Consistent Self-Training for Content Categorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional content categorization systems face challenges in consistently categorizing products with variations, leading to inconsistent results that negatively impact user experiences in search engines and recommender systems.
Innovation Solution
The system generates augmented training data by combining supervised training data with weakly labeled data from unlabeled data clusters, using consistent self-training or consistent generative augmentation to enrich the training data and improve categorization models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional text categorization models are used to categorize products based on input text, then the categorization process is simple and fast, but the categorization consistency deteriorates when products have variations such as different titles or features
Solution Approach 1:
The system performs preliminary actions by generating augmented training data that includes multiple variations of product descriptions before the actual categorization task. This pre-processing step creates a more robust model that can handle variations in product titles and features, thereby improving categorization consistency without increasing the complexity of the categorization process itself.
Solution Approach 2:
The system creates copies of existing training data by generating augmented samples that mimic real-world product variations. These synthetic copies include modified product titles, different feature descriptions, and varied formulations while maintaining the same underlying category. Training on these copied variations enables the model to generalize better across different product representations.
2Reliability
If more training data with variations is used to improve categorization consistency, then the model robustness improves, but the data processing time and computational resources increase
Solution Approach 1:
The system applies parameter changes by systematically modifying specific attributes of training data such as product titles, feature descriptions, and text formulations while keeping the core category information intact. This approach generates diverse training samples efficiently without requiring extensive manual data collection or processing, thereby improving model robustness while controlling data processing time.
3Measurement precision
If product variations are handled by creating separate categories for each variation, then categorization accuracy for specific variations improves, but the category taxonomy complexity increases
Solution Approach 1:
The system segments the categorization task by separating the variation-handling functionality from the core category classification. Instead of creating separate categories for each product variation, the model learns to recognize and ignore variation-related features while focusing on the essential category-defining characteristics. This segmentation maintains a simple taxonomy structure while achieving high categorization accuracy across product variations.
Data Source
AI summary
The present teaching relates to content categorization. Supervised training data and unlabeled data clusters are used to generate augmented training data. Each unlabeled data cluster includes data samples with varying features. Weakly labeled training data is created based on supervised training data and the unlabeled data clusters with data samples therein with cluster labels via consistent self-training so that a labeled data sample in the supervised training data and a data sample in the weakly labeled training data with the same label have varying characteristics. Augmented training data is created from the supervised and the weakly labeled training data and is used to train a robust content categorization model via machine learning.