Context-Based Labeling for Unobserved Entities in Sequential Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Obtaining high-quality, labeled sequential data for machine learning models in industrial applications like autonomous driving is expensive and challenging due to incomplete labeling, with entities often going unobserved and unlabeled.
Innovation Solution
A system and method that utilizes a knowledge graph representation of sequential data to infer and augment additional labels for unobserved entities by leveraging Piaget's theory of object continuity, applying a context-based method (CLUE) to enhance scene understanding and label augmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If traditional data augmentation methods are used assuming complete labeling, then data availability improves, but labeling accuracy deteriorates due to unobserved entities
Solution Approach 1:
The system performs preliminary analysis of future scenes to identify entities that will become observable, then uses this information to infer and label unobserved entities in current scenes. This proactive approach allows the system to anticipate and label entities before they appear in the field of view, resolving the contradiction between data availability and labeling accuracy.
Solution Approach 2:
The system introduces an intermediary inference mechanism that connects observed entities in future scenes to unobserved entities in current scenes. By using temporal relationships and object continuity principles as intermediaries, the system can transfer labeling information across time boundaries, improving both data availability and maintaining labeling accuracy.
2Measurement precision
If manual labeling of all entities is performed, then labeling accuracy improves, but processing time increases significantly
Solution Approach 1:
Instead of manually labeling all entities in all scenes, the system applies partial manual labeling to observed entities and uses automated inference for unobserved entities. This selective approach performs labeling action only where necessary (on observed entities) and lets the inference system handle the rest, significantly reducing processing time while maintaining accuracy.
Solution Approach 2:
The system uses feedback from future scene observations to improve current scene labeling. By continuously monitoring what entities appear in future frames and using this information to refine inferences about current unobserved entities, the system creates a feedback loop that improves accuracy without requiring exhaustive manual labeling of every scene.
3Quantity of substance
If data augmentation is applied without considering incomplete labeling, then dataset size increases, but data quality deteriorates due to missing entity labels
Solution Approach 1:
The system performs preliminary inference about unobserved entities using future scene information before data augmentation is applied. By pre-labeling unobserved entities based on temporal continuity and object persistence principles, the system ensures that augmented data maintains high quality and reliability, resolving the contradiction between dataset size and data quality.
Data Source
AI summary
A method includes receiving one or more datasets that includes one or more labels, identifying a first set of scenes associated with a first set of nodes and relations in a first sequence utilizing the datasets and labels to create a first window, identifying a second set of scenes associated with a second set of nodes and relations in a second sequence utilizing the e datasets and labels to create a second window, wherein the second window is in a future position compared to the first window, extracting a set of observed entities within the windows in response to inspecting the scenes across at the windows, determining the unobserved entities associated within a target scene utilizing at least the set of observed entities, wherein the target scene is sequentially between the first window and the second window, and augmenting the dataset.