Ground Truth Annotation via Graph Centrality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current supervised machine learning algorithms are data-hungry and require substantial annotated datasets, which is costly and inefficient, especially in industrial settings where large annotated datasets are not readily available.
Innovation Solution
A computer-implemented method using an auto-encoder with an embedding model and graph-based algorithms to identify central nodes in a compressed feature matrix, allowing for efficient annotation of only the most relevant data items, thereby reducing the need for extensive annotation and facilitating the generation of ground truth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If supervised machine learning algorithms are used to train models, then model accuracy can be improved, but the cost and time required for annotating large datasets increases substantially
Solution Approach 1:
The patent segments the dataset into different components: a small subset of central nodes that require manual annotation, and the remaining nodes that can be labeled through propagation. This segmentation allows the annotation process to focus only on the most informative samples, dramatically reducing the time and resources needed while maintaining model accuracy.
Solution Approach 2:
The patent performs preliminary actions by using graph-based algorithms (clustering and centrality analysis) to identify which nodes should be annotated before the actual annotation process. This preliminary identification of central nodes ensures that annotation efforts are concentrated on the most valuable samples, avoiding wasted time on less informative data points.
2Measurement precision
If supervised machine learning algorithms are used to train models, then model accuracy can be improved, but the cost of annotating large datasets increases substantially
Solution Approach 1:
The patent segments the annotation task into two parts: manual annotation of a small set of central nodes and automated propagation to the rest of the graph. This segmentation reduces the quantity of data requiring expensive manual annotation while preserving model accuracy through the propagation mechanism.
Solution Approach 2:
The patent uses label propagation to copy annotations from central nodes to their neighboring nodes in the graph. This copying mechanism allows the system to generate labels for the majority of the dataset without incurring the full cost of manual annotation, significantly reducing annotation expenses while maintaining data quality.
3Loss of information
If all data items in a training dataset are annotated, then complete ground truth is obtained, but the annotation burden and resource requirements become excessive
Solution Approach 1:
The patent segments the dataset into central nodes (annotated manually) and peripheral nodes (labeled through propagation). This segmentation maintains sufficient ground truth completeness for model training while dramatically improving annotation efficiency by avoiding the need to manually annotate every single data item.
Solution Approach 2:
The patent implements a self-service mechanism where the system automatically propagates labels from annotated central nodes to unannotated nodes using graph-based algorithms. This self-service approach allows the system to generate its own labels for the majority of the dataset, reducing the annotation burden on human annotators while maintaining ground truth completeness.
Data Source
AI summary
A computer-implemented method for determining a set of target items to be annotated for training a machine learning application. The method comprises providing a training data set with a set of data samples and an auto-encoder with a classifier. The auto-encoder comprises an embedding model that maps the set of data samples to a set of compressed feature vectors. The set of compressed feature vectors define a compressed feature matrix. Further provided are: a definition of a graph associated to the compressed feature matrix, applying a clustering-algorithm to identify node clusters of the graph and applying a centrality algorithm to identify central nodes of the node clusters, retrieving from an annotator node labels for the central nodes, propagating the annotated node labels to other nodes of the graph and performing a training of the embedding model and the classifier with the annotated and the propagated node labels.