Automatic Data Augmentation with Storage-Side Preprocessing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic data augmentation methods in machine learning, particularly for deep learning, face inefficiencies in preprocessing operations and model training, necessitating improved techniques to enhance their efficiency.
Innovation Solution
A method involving a central processing unit (CPU) determining hyper-parameters for sub-models, with storage devices and graphics processing units (GPUs) training and preprocessing datasets, followed by updating sub-models and hyper-parameters to optimize the training process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If CPU performs preprocessing operations for data augmentation, then model training can be conducted, but preprocessing efficiency is low and CPU power consumption is high
Solution Approach 1:
The patent extracts the preprocessing function from the CPU and assigns it to the storage device. The storage device controller includes a preprocessing unit that performs data augmentation operations directly on the storage medium, separating this computationally intensive task from the CPU's responsibilities. This extraction resolves the contradiction by improving preprocessing efficiency while reducing CPU power consumption.
Solution Approach 2:
The patent introduces a storage device controller as an intermediary component between the CPU and storage device. This controller includes both a preprocessing unit and a training unit, acting as a mediator that handles preprocessing operations and coordinates model training. This intermediary structure enables efficient preprocessing without burdening the CPU, thereby resolving the efficiency and power consumption contradiction.
2Reliability
If batch size is increased for model training, then training effectiveness improves, but CPU utilization increases and processing capacity is exceeded
Solution Approach 1:
The patent extracts the model training function from the CPU and implements it within the storage device controller. The training unit in the controller handles model training operations, allowing increased batch sizes without overloading the CPU. This extraction enables improved training effectiveness while maintaining CPU processing capacity within safe limits.
Solution Approach 2:
The patent moves the training operation from the CPU dimension to the storage device controller dimension. By implementing the training unit within the storage device controller, the system utilizes previously underutilized resources in the storage subsystem, effectively adding a new computational dimension. This allows larger batch sizes and improved training effectiveness without increasing CPU processing capacity requirements.
3Productivity
If preprocessing and training are performed using traditional CPU-GPU architecture, then model training can be conducted, but overall system efficiency is limited
Solution Approach 1:
The patent merges the preprocessing function and training function into a single integrated unit within the storage device controller. The controller contains both a preprocessing unit for data augmentation and a training unit for model training, combining these previously separate functions. This merging improves overall system efficiency by eliminating data transfer overhead and enabling coordinated operation, while avoiding the need for additional separate components that would increase system complexity.
Solution Approach 2:
The storage device controller is designed with multi-functionality, serving both as a storage management unit and as a preprocessing and training unit. The controller's preprocessing unit and training unit enable the storage device to perform multiple functions beyond traditional data storage. This universality improves training speed by utilizing the storage device's computational capabilities, while avoiding the need for separate dedicated preprocessing and training hardware that would increase system architecture complexity.
Data Source
AI summary
A method for automatic data augmentation, including: determining a plurality of hyper-parameters corresponding to a plurality of sub-models using a central processing unit (CPU); training the plurality of sub-models based on the plurality of hyper-parameters using a plurality of storage devices and a plurality of graphics processing units (GPUs); and updating at least one sub-model from among the trained plurality of sub-models and at least one hyper-parameter corresponding to the updated at least one sub-model using the CPU, wherein the plurality of sub-models are trained for a first predetermined number of epochs, and wherein, during each epoch, the training includes: preprocessing an original dataset based on the plurality of hyper-parameters to generate a plurality of preprocessed datasets corresponding to the plurality of hyper-parameters using the plurality of storage devices; and training the plurality of sub-models based on the plurality of preprocessed datasets using the plurality of GPUs.