Video Data Decimation Using Feature Vector Dissimilarity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training machine learning systems, such as autonomous vehicles, require large amounts of video data that are costly to store and annotate, with redundant data being captured, leading to inefficiencies in data collection and increased costs.
Innovation Solution
A computing system that decimates video data by selecting dissimilar frames based on feature vectors and uncertainty scores, using a combination of convolutional neural networks and long-short term memory networks to identify salient frames for storage, reducing the data volume while maintaining performance improvement potential.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If large amounts of video data are collected for training machine learning systems, then the training data volume increases, but storage costs and annotation expenses increase
Solution Approach 1:
The patent extracts only the most salient frames from the complete video data by using uncertainty scores generated by a neural network. The system identifies and extracts only those frames that have high uncertainty (indicating they provide the most value for training), thereby reducing the quantity of data that needs to be stored and annotated while maintaining training effectiveness.
Solution Approach 2:
The patent changes the parameter of data selection from including all frames to including only frames above a certain uncertainty threshold. By adjusting this threshold parameter, the system optimizes the balance between data quantity and cost, selecting only the most valuable frames for training while discarding redundant low-uncertainty frames.
2Reliability
If all video frames are captured and stored, then no data is lost, but redundant data increases storage requirements
Solution Approach 1:
The patent applies local quality by differentiating between individual frames based on their uncertainty scores. Instead of treating all frames equally, the system assigns different values to different frames, retaining only those with high uncertainty (high quality for training) while discarding low uncertainty frames (redundant data), thus reducing storage volume without compromising overall data reliability.
3Manufacturing precision
If manual annotation of all video data is performed, then training data quality improves, but annotation costs and time increase
Solution Approach 1:
The patent extracts only the most valuable frames for annotation by using uncertainty scores as a filter. The neural network identifies frames with high uncertainty that are most likely to improve training, and only these extracted frames are subjected to manual annotation. This dramatically reduces the total annotation time and cost while maintaining high training data quality.
Solution Approach 2:
The patent performs preliminary action by using the neural network to pre-score and rank all video frames according to their uncertainty before human annotation begins. This preliminary sorting ensures that annotators focus their time on the most valuable frames first, optimizing the annotation process efficiency and reducing overall annotation time.
Data Source
AI summary
A computing system for decimating video data includes: a processor; a persistent storage system coupled to the processor; and memory storing instructions that, when executed by the processor, cause the processor to decimate a batch of frames of video data by: receiving the batch of frames of video data; mapping, by a feature extractor, the frames of the batch to corresponding feature vectors in a feature space, each of the feature vectors having a lower dimension than a corresponding one of the frames of the batch; selecting a set of dissimilar frames from the plurality of frames of video data based on dissimilarities between corresponding ones of the feature vectors; and storing the selected set of dissimilar frames in the persistent storage system, the size of the selected set of dissimilar frames being smaller than the number of frames in the batch of frames of video data.