On-Device Data Augmentation Engine for Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
On-device machine learning faces challenges such as limited training data due to storage constraints, leading to overfitting and a lack of personalization, as traditional data augmentation methods have limited degrees of freedom and cannot effectively diversify the training process.
Innovation Solution
A mobile data augmentation engine that processes images using a photometric augmentation engine and feature augmentation engines within multiple layers of a convolutional neural network, allowing for random geometric augmentation and increased degrees of freedom, thereby diversifying the training process and reducing the risk of overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional data augmentation methods are used on personal electronic devices, then storage constraints are addressed, but the degrees of freedom for diversifying training remain limited
Solution Approach 1:
The patent segments data augmentation into two distinct components: photometric augmentation (color, brightness, contrast adjustments) and geometric augmentation (rotation, flipping, cropping). This segmentation allows each type of transformation to be independently applied and controlled, significantly increasing the degrees of freedom for training diversification while maintaining manageable system complexity through modular design.
Solution Approach 2:
The patent extends data augmentation from traditional single-dimension transformations to multi-dimensional transformations by applying both photometric changes (affecting color space dimensions) and geometric transformations (affecting spatial dimensions). This dimensional expansion creates exponentially more varied training samples from limited input data, directly addressing the need for greater adaptability in on-device machine learning.
2Reliability
If more training data is collected to reduce overfitting, then model generalization improves, but storage requirements and processing time increase
Solution Approach 1:
The patent applies data augmentation transformations preliminarily to the training data before feeding it into the machine learning model. By pre-processing images with various photometric and geometric transformations, the system creates diverse training samples in advance, enabling the model to learn from varied data without requiring additional storage space or longer processing times during actual training execution.
Solution Approach 2:
The patent creates multiple transformed copies of each original training image through photometric and geometric augmentations. Instead of collecting additional unique images that would require more storage, the system generates synthetic variations by copying and transforming existing images, thereby improving model generalization while maintaining constant data storage requirements.
3Adaptability or versatility
If data augmentation is applied to increase training diversity, then overfitting is reduced, but processing power requirements increase
Solution Approach 1:
The patent implements dynamic data augmentation where photometric and geometric transformations are applied randomly and variably during each training epoch. This dynamic approach maximizes training diversity by constantly varying the transformations applied to training images, while the transformations themselves remain computationally efficient operations that do not require significant additional processing power compared to traditional static augmentation methods.
Data Source
AI summary
A method includes processing, using at least one processor of an electronic device, each of multiple images using a photometric augmentation engine, where the photometric augmentation engine performs one or more photometric augmentation operations. The method also includes applying, using the at least one processor, multiple layers of a convolutional neural network to each of the images, where each layer generates a corresponding feature map. The method further includes processing, using the at least one processor, at least one of the feature maps using at least one feature augmentation engine between consecutive layers of the multiple layers, where the at least one feature augmentation engine performs one or more feature augmentation operations.