Pseudo-Label Reliability Assessment Through Model Agreement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for generating pseudo labels lack reliability, as evidenced by the limitations in Patent Literature 1, necessitating a more reliable method for assigning labels in supervised machine learning.
Innovation Solution
An information processing apparatus and system that calculates a reliability level for models based on the agreement degree between labels generated by multiple pseudo label generating means, using these reliability levels to determine a highly reliable pseudo label for target data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a pseudo label is generated using a single model, then the labeling process is simple and fast, but the reliability of the pseudo label is low
Solution Approach 1:
The patent combines multiple pseudo label generating means (different models or labeling methods) into a unified system. Each generating means produces labels independently, and their results are aggregated through a selection process that considers agreement degrees, thereby improving reliability while managing complexity through structured integration.
Solution Approach 2:
The system implements feedback by calculating agreement degrees between labels from different generating means and using this information to iteratively select and refine pseudo labels. The reliability levels are computed based on feedback from multiple sources, and this feedback loop continues until high-reliability labels are identified for training.
2Reliability
If multiple label generating means are used to improve reliability, then the reliability of the pseudo label increases, but the calculation complexity and processing time increase
Solution Approach 1:
The system does not require all possible label generating means to be executed fully. Instead, it performs partial actions by calculating agreement degrees and reliability levels only to the extent necessary to identify sufficiently reliable pseudo labels. Once a threshold is met, the process can stop, avoiding unnecessary computation time.
Solution Approach 2:
The system dynamically adjusts parameters such as the number of label generating means to invoke, the threshold for agreement degree, and the reliability level criteria based on the specific task requirements and available resources. This allows optimization of the balance between reliability and processing time for different scenarios.
3Measurement precision
If agreement degree calculation is performed between multiple labels, then the reliability assessment becomes more accurate, but the computational load increases
Solution Approach 1:
The agreement degree calculation is segmented into pairwise comparisons between labels from different generating means, rather than computing a complex multi-dimensional agreement metric. This segmentation reduces computational complexity while maintaining measurement precision by systematically evaluating each pair's consistency.
Solution Approach 2:
The patent introduces an intermediary reliability level calculation that mediates between the raw agreement degrees and the final pseudo label selection. This intermediary step aggregates agreement information in a computationally efficient manner, reducing the direct computational burden while preserving measurement accuracy.
Data Source
AI summary
An information processing apparatus includes: a target data acquiring section that acquires target data; a reliability level calculating section that, from at least one of a plurality of models and the target data, calculates a reliability level of the at least one of the plurality of models on the basis of an agreement degree between labels obtained by at least one of a plurality of label generating means; and a label determining section that refers to the reliability level to determine a label to be assigned to the target data.