Neural Network Processing Unit with Block-Dependency Scheduling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing neural networks using pipeline processing is challenging due to the dependency of one operation's output being the input for a subsequent operation, leading to potential data hazards and increased hardware requirements.
Innovation Solution
A processing unit that receives block dependency data to divide layers into input blocks, generating output blocks, and performs subsequent operations only after a set number of output blocks are generated, defined by the block dependency data, thereby avoiding data hazards and reducing hardware needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pipeline processing is used to implement neural network operations, then processing efficiency is improved, but data hazards occur due to dependency between consecutive operations
Solution Approach 1:
The patent applies preliminary action by introducing a buffer to store output blocks before the second operation reads them. The buffer is prepared in advance and filled with output blocks from the first operation, ensuring that when the second operation needs data, it is already available. This eliminates data hazards while maintaining pipeline processing efficiency.
Solution Approach 2:
The buffer acts as an intermediary between the first and second operations. It receives output blocks from the first operation and provides them to the second operation, decoupling the two operations from direct data dependency. This intermediary mechanism allows both operations to proceed in parallel without data hazards.
2Productivity
If pipeline processing is implemented with proper data dependency management, then processing efficiency is improved, but hardware requirements increase
Solution Approach 1:
The patent segments the neural network operations into discrete blocks that can be processed independently. Each operation works on specific input blocks and produces output blocks that can be stored and consumed separately. This segmentation allows for simpler hardware implementation compared to managing complex data dependencies between entire operations.
Solution Approach 2:
The buffer creates a copy of the output blocks from the first operation, allowing the second operation to read from this copy rather than requiring direct access to the first operation's output. This copying mechanism simplifies hardware requirements by providing a straightforward data transfer path without complex synchronization logic.
Data Source
AI summary
A processing unit is described that receives an instruction to perform a first operation on a first layer of a neural network, block dependency data, and an instruction to perform a second operation on a second layer of the neural network. The processing unit performs the first operation, which includes dividing the first layer into a plurality of input blocks, and operating on the input blocks to generate a plurality of output blocks. The processing unit then performs the second operation after the first operation has generated a set number of output blocks defined by the block dependency data.