Self-Supervised Neural Network Training for Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for training neural networks for action recognition in time-sequences of data samples are hindered by the need for extensive and costly manual annotation of labeled data, and the scarcity of skeleton-based training data, which limits the efficiency and accuracy of action recognition systems.
Innovation Solution
A system and method that leverages self-supervised learning by pre-training neural networks using unlabeled action sequences through data augmentation, where two neural networks interact to generate augmented views, with one network predicting the representation of the other, and then fine-tuning with a small labeled dataset, utilizing conservative and aggressive augmentation techniques to improve feature representation and reduce data processing complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used to label training data, then action recognition accuracy can be improved, but time consumption and cost increase significantly
Solution Approach 1:
The system performs preliminary self-supervised pre-training on unlabeled data before final supervised training. This preliminary action of learning from unlabeled data through data augmentation and predictive tasks reduces the burden of manual annotation while preparing the network for accurate action recognition with limited labeled data.
2Reliability
If extensive labeled data is collected for training, then neural network performance improves, but data processing complexity and cost increase
Solution Approach 1:
The system enables the neural network to self-train using unlabeled data through self-supervised learning. The network performs predictive tasks on augmented versions of itself and updates its parameters autonomously, eliminating the need for extensive manual labeling while maintaining performance through self-service learning mechanisms.
Solution Approach 2:
Data augmentation serves as an intermediary mechanism that transforms limited labeled data into diverse training samples. By applying various augmentation operations to the available data, the system effectively increases the training dataset size without requiring proportional increases in manual annotation effort.
3Quantity of substance
If self-supervised pre-training is performed on unlabeled data, then the need for labeled data is reduced, but computational resources and training time increase
Solution Approach 1:
The training process is segmented into two distinct phases: self-supervised pre-training on unlabeled data followed by supervised fine-tuning on labeled data. This segmentation allows the system to efficiently use unlabeled data for initial learning and then quickly adapt to specific tasks with minimal labeled examples, optimizing computational resource utilization.
Data Source
AI summary
A system for training a neural network for action recognition based on unlabeled action sequences includes a first neural network (NN1) and a second neural network (NN2). A first updating module is arranged to update parameters of NN1 to minimize a difference between representation data generated by NN1 and representation data generated by NN2. A second updating module is arranged to update parameters of NN2 as a function of the parameters of NN1. An augmentation module includes first and second sub-modules and is configured to include augmented versions of incoming action sequences in first and second input data. The first and second sub-modules are configured to apply at least partly different augmentation to the incoming action sequences. After NN1 and NN2 have been operated on one or more instances of the first and second input data, NN1 comprises a parameter definition of a pre-trained neural network.