Handed Activity Recognition Using Mirror-Flipped Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning-based activity recognition systems face challenges in distinguishing between left- and right-handed activities, leading to increased complexity and cost, and are sensitive to imbalanced population distributions, especially when using public datasets.
Innovation Solution
An activity recognition system that leverages left-right mirror symmetry by flipping input horizontally to infer both left- and right-handed activities, using a training method that balances the training dataset by flipping all instances and labels, and combining outputs from flipped and non-flipped inputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the system distinguishes between left- and right-handed activities by doubling the number of classes, then the recognition accuracy for asymmetric activities is improved, but the system complexity and training cost increase
Solution Approach 1:
Instead of training separate models for left-handed and right-handed activities, the patent inverts the approach by training a single model to recognize only left-handed activities and then horizontally flipping the input images to infer right-handed activities. This inversion reduces the number of classes from 2N to N while maintaining recognition accuracy for both handednesses.
Solution Approach 2:
The patent creates a synthetic copy of the training data by horizontally flipping existing left-handed activity images and labels to generate right-handed training examples. This copying approach balances the training dataset without requiring separate data collection for each handedness, reducing overall system complexity.
2Quantity of substance
If the training dataset is sampled from public data, then the data availability is improved, but the class distribution becomes imbalanced favoring right-handed activities
Solution Approach 1:
The patent uses data copying through horizontal flipping to transform available public dataset examples into balanced training data. By flipping left-handed examples to appear as right-handed examples (and vice versa), the system creates a balanced 50-50 distribution without needing to collect separate datasets for each handedness, thus maintaining data availability while achieving classification fairness.
Solution Approach 2:
The patent changes the parameter of data representation by horizontally flipping images and corresponding labels. This parameter transformation converts the original imbalanced dataset into a balanced dataset, ensuring equal representation of left-handed and right-handed activities while using the same source data.
3Measurement precision
If separate training is performed for left- and right-handed activities, then the training accuracy for each class is improved, but the training time and computational cost increase
Solution Approach 1:
The patent merges the training process for left-handed and right-handed activities into a single training run. By horizontally flipping the training data to create balanced representations of both handednesses, the system trains one model simultaneously for both classes, reducing training time and computational cost while maintaining high accuracy for both left- and right-handed activity recognition.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This disclosure describes an activity recognition system for asymmetric (e.g. left- and right- handed) activities that leverages the symmetry intrinsic to most human and animal bodies. Specifically, described is 1) a human activity recognition system that only recognizes handed activities but is inferenced twice, once with input flipped, to identify both left- and right- handed activities and 2) a training method for learning-based implementations of the aforementioned system that flips all training instances (and associated labels) to appear left-handed and in doing so, balances the training dataset between left- and right- handed activities.