Action Recognition Training With Pseudo Action-Object Compositions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning (ML) action recognition computer models struggle to accurately recognize unseen combinations of actions and objects due to the high cost and limitations of generating comprehensive labeled training datasets, leading to inaccurate results when faced with novel compositions.
Innovation Solution
The Pseudo Composition Inference (PCI) framework generates pseudo compositions by combining detected actions and objects in new combinations not present in the original training data, expanding the diversity of the training dataset and improving the ML action recognition model's ability to generalize to unseen situations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive labeled training datasets are generated to improve recognition accuracy of unseen action-object combinations, then the model's generalization performance improves, but the cost and time required for data preparation increases significantly
Solution Approach 1:
The system performs preliminary action by generating pseudo-composition training data before actual model training. It creates synthetic action-object combinations by combining verb representations with object features from the feature bank, preparing diverse training samples in advance without requiring manual labeling of each combination.
Solution Approach 2:
The system creates copies of existing data patterns to generate new training samples. It copies verb-action representations and combines them with object features from the feature bank to create pseudo-composition data that mimics real action-object interactions without requiring actual video recordings of each combination.
2Manufacturing precision
If manual labeling of training data is performed to ensure data quality, then the training data accuracy improves, but the cost and complexity of the process increases
Solution Approach 1:
The system performs self-service by automatically generating pseudo-composition training data without human intervention. The object feature bank and verb representations automatically combine to create training samples, eliminating the need for manual labeling while maintaining data quality through structured feature composition.
Solution Approach 2:
Instead of manual creation, the system copies and recombines existing verified data elements (verb representations and object features) to generate new training samples. This copying approach preserves the quality of original labeled data while creating diverse combinations automatically.
3Adaptability or versatility
If large datasets are collected to cover all possible action-object combinations, then the model's ability to handle unseen compositions improves, but the data storage and processing requirements increase
Solution Approach 1:
The system segments the training data into separate components: verb representations and object features stored in the feature bank. This segmentation allows the model to learn general patterns from smaller datasets and reconstruct unseen combinations by combining these segmented elements, reducing the need for exhaustive data collection.
Solution Approach 2:
The system transitions from storing complete action-object video pairs to storing decomposed feature representations in a high-dimensional feature space. This dimensional transformation enables the model to generate unseen combinations through feature manipulation rather than requiring actual video data for every possible combination.
Data Source
AI summary
Mechanisms are provided for performing machine learning (ML) training of a ML action recognition computer model which involves processing an original input dataset to generate an object feature bank comprising object feature data structures for a plurality of different objects. For an input video, a verb data structure and an original object data structure are generated and a candidate object feature data structure is selected from the object feature bank for generation of pseudo composition (PC) training data. The PC training data is generated based on the selected candidate object feature data structure and comprises a combination of the verb data structure and the candidate object feature data structure. The PC training data represents a combination of an action and an object not represented in the original input dataset. ML training of the ML action recognition computer model is performed based on an unseen combination comprising the PC training data.