Visual Action Data Augmentation via Object Motion Inversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for augmenting video data for machine learning applications, such as image and video augmentation, are insufficient when new constraints need to be learned from limited original data, as techniques like rotating and scaling do not provide new insights, and reversing video frames may not work for all tasks.
Innovation Solution
A computer-implemented method and system that defines actions into sets of objects and motions, determines relationships between these, and transforms detected objects and motions to create augmented visual data by rotating objects 180 degrees about the y-axis, providing opposing actions like 'entering a door' and 'exiting a door', which are not achievable by simply reversing frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If basic image manipulation techniques (rotating, scaling) are used for data augmentation, then the training dataset size increases, but the quality and relevance of new data points deteriorates
Solution Approach 1:
The patent applies inversion by reversing video frames to generate opposing actions (e.g., entering vs. exiting a door). This transforms existing action data into new training samples with semantically opposite meanings, thereby increasing dataset quantity while maintaining high information quality through meaningful semantic transformation rather than superficial geometric changes.
2Adaptability or versatility
If frame reversal is used to generate opposing actions, then diverse action data can be created, but it does not work for all action recognition tasks
Solution Approach 1:
The patent employs dynamics by selectively applying frame reversal based on task requirements. The system dynamically determines whether to reverse frames depending on the specific action recognition task, making the augmentation approach adaptable to different scenarios while maintaining reliability by avoiding inappropriate applications of reversal.
3Reliability
If extensive augmentation is applied to limited original data, then overfitting is prevented, but the computational complexity increases
Solution Approach 1:
The patent applies partial action by selectively augmenting only certain portions of the dataset through frame reversal, rather than applying exhaustive augmentation to all data. This targeted approach prevents overfitting by creating diverse training samples where needed, while avoiding unnecessary computational overhead by not over-augmenting already sufficient data.
Data Source
AI summary
Generating visual data by defining a first action into a first set of objects and corresponding first set of motions, and defining a second action into a second set of objects and corresponding second set of motions. A relationship is then determined for the second action to the first action in terms of relationships between corresponding constituent objects and motions. Objects and motions are detected from visual data of first action. Visual data is composed for the second action from the data by transforming the constituent objects and motions detected in first action based on the corresponding determined relationships.