Tensor Streaming Processor Data Transformation for ResNet-50 Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CPU and GPU architectures face scalability, performance, and usability challenges in handling computationally intensive deep learning workloads, particularly for models like ResNet-50, due to hardware complexity and inadequate multi-core architectures.
Innovation Solution
A tensor streaming processor (TSP) with a functional slice architecture, organized into memory and arithmetic logic slices, processes data and instructions efficiently by streaming operands across dimensions and synchronizing timing using a compiler-aware approach, implementing data transformation algorithms for ResNet-50 models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional CPU and GPU architectures are used for deep learning workloads, then general-purpose computing capability is maintained, but performance and scalability are insufficient for computationally intensive models like ResNet-50
Solution Approach 1:
The processor is divided into multiple functional slices, each dedicated to specific operations in the deep learning pipeline. These slices include data transformation units, convolution units, activation units, and pooling units, allowing parallel processing of different stages of neural network computations simultaneously
Solution Approach 2:
The patent introduces a novel data transformation domain (SLT - Split-Less Transform) that changes the dimensional organization of data from conventional spatial arrangements to a transformed domain optimized for convolution operations. This dimensional change enables more efficient data access patterns and reduces computational complexity for specific deep learning operations
2Productivity
If microarchitectural enhancements such as caches, branch predictors, and prefetchers are added, then performance is improved, but worst-case performance is not limited and hardware complexity increases
Solution Approach 1:
The patent extracts and removes complex microarchitectural components such as caches, branch predictors, and prefetchers from the processor design. Instead, it uses a simplified architecture with deterministic data flow and explicit data movement control, eliminating the need for these complex enhancements while maintaining high performance for deep learning workloads
Solution Approach 2:
The patent changes the fundamental parameters of the processor architecture by adopting a fixed-function slice structure with deterministic execution semantics. This parameter change from general-purpose dynamic execution to specialized deterministic execution eliminates the need for complex microarchitectural enhancements
3Productivity
If data is packed into high-dimensional vectors for efficient memory utilization, then memory bandwidth is optimized, but data transformation to suitable domains for convolution operations becomes complex
Solution Approach 1:
The patent performs preliminary data transformation by converting input data from conventional spatial format into the SLT (Split-Less Transform) domain before convolution operations. This preliminary action in the data transformation unit simplifies subsequent convolution computations by pre-organizing data in a domain optimized for these operations, reducing the complexity of real-time transformations during inference
Data Source
AI summary
Embodiments are directed to data transformation algorithms performed at a deterministic streaming processor. Blocks of input data is streamed from a memory of the processor via a superlane of a processor to a first functional slice of the processor. The first functional slice permutes each block of input data, and each permuted block is streamed back to the memory. Permuted blocks of input data are then streamed from the memory via the superlane to a second functional slice of the processor. The second functional slice aligns portions of each permuted block to lanes within the superlane. Aligned portions of permuted blocks are then streamed to a third functional slice of the processor. The third functional slice merges the aligned portions of the permuted blocks to generate result data in a transformation domain suitable for at least one convolutional layer of the ResNet-50 model.