Master Data Transforms for Faster Neural Network Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks requires significant data preparation and transformation, which is memory and computationally expensive, and existing methods face limitations in formatting and efficiency.
Innovation Solution
Utilizing parallel processing units (PPUs) like GPUs to accelerate data preparation and transformation, combining transforms into master transforms to optimize memory usage and compute time, and employing a training framework to prepare and transform data for neural networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data transforms are performed using traditional CPU-based methods, then memory requirements are manageable, but processing time becomes excessively long
Solution Approach 1:
The patent segments the data transform pipeline into distinct stages (e.g., decode, denoise, scale, normalize) and assigns different stages to different processing units based on their capabilities. CPU handles preprocessing stages while GPU handles computationally intensive stages, allowing each to operate in its optimal performance zone without requiring the entire dataset to reside in one device's memory at once.
Solution Approach 2:
The patent introduces an intermediary memory system (such as system memory or storage) that acts as a buffer between CPU and GPU operations. Data can be streamed through this intermediary memory, allowing transforms to proceed in a pipeline fashion where the CPU reads from storage while the GPU processes previous batches, eliminating the need for all data to simultaneously occupy high-capacity memory.
2Productivity
If all data transforms are performed on PPUs to reduce processing time, then speed improves, but device complexity and memory management become more difficult
Solution Approach 1:
The patent creates a universal transform execution framework that can automatically route different transform operations to the most appropriate processing unit. The system evaluates each transform's characteristics (computational intensity, memory requirements, data format) and dynamically assigns it to CPU or GPU, making the overall system adaptable to different workloads without requiring manual configuration or complex memory management for each specific case.
3Adaptability or versatility
If multiple separate transforms are applied sequentially to prepare data, then formatting flexibility is maintained, but processing time increases significantly
Solution Approach 1:
The patent merges multiple sequential transform operations into a single integrated transform pipeline that can execute atomically on the GPU. Instead of applying decode, denoise, scale, and normalize as separate CPU-based operations that each require data loading and processing, the system combines these into a unified GPU kernel that processes the data in one continuous operation, maintaining all the individual transform functionalities while eliminating the time losses from sequential execution and repeated data transfers.
Data Source
AI summary
Apparatuses, systems, and techniques to transform input data for training neural networks. In at least one embodiment, one or more data transforms are identified in a sequence of data transforms and combined into one or more master data transforms to be performed by one or more parallel processing units in order to prepare data for training an untrained neural network.