Soft Label Inheritance Using Capsule Ancestors for ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional soft label generation processes in machine learning are inexplicit, leading to abstraction of useful features in training data, and existing data set distillation and self-training methods face challenges with noise in pseudo-labels and high computational costs.
Innovation Solution
Implement label inheritance techniques using capsule neural networks and probabilistic bag of images (PBoI) representations to explicitly generate soft labels, leveraging capsules to derive meaningful patterns and identify ancestors for improved label generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional soft label generation processes are used, then labels can be generated with probability scores, but useful features associated with training data are abstracted and lost
Solution Approach 1:
The patent introduces capsules as intermediary representations between raw training data and soft labels. These capsules preserve useful features by maintaining structured representations of data instances while enabling probability score generation. The capsules act as mediators that prevent feature abstraction and loss occurring in conventional direct soft label generation processes.
Solution Approach 2:
The patent segments the label generation process into distinct components: (1) generating capsule representations from training data, (2) identifying ancestor capsules, and (3) computing soft labels based on ancestral relationships. This segmentation allows each component to preserve and propagate useful features systematically, preventing information loss that occurs in monolithic conventional approaches.
2Measurement precision
If data set distillation and self-training methods are used, then model training can be improved, but noise in pseudo-labels and high computational costs occur
Solution Approach 1:
The patent performs preliminary action by pre-computing capsule representations and establishing ancestral relationships before the actual distillation or self-training process. This preprocessing creates a structured framework of labeled capsules that can be efficiently reused, reducing computational requirements during subsequent training operations while maintaining high precision through explicit label inheritance.
Solution Approach 2:
The patent changes the parameter representation from conventional continuous probability distributions to discrete label inheritance relationships based on capsule ancestry. This parameter transformation reduces noise in pseudo-labels by grounding probabilities in explicit structural relationships, while the hierarchical capsule organization reduces computational complexity through efficient inheritance patterns.
Data Source
AI summary
Label inheritance techniques are disclosed for soft label generation in an information processing system that uses machine learning. For example, a method generates at least one label for a given data instance from a training data set useable to train a machine learning-based model. The at least one label is generated by assigning one or more labels associated with one or more ancestors of the data instance such that the data instance inherits the one or more labels associated with the one or more ancestors as the at least one label.