Neural Network PE Architecture With Depth-Wise Parallel Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural-network-processing architectures face inefficiencies in resource utilization due to limitations in depth-sequential processing, leading to underutilization of computing resources and reduced performance, especially in edge devices with power and packaging constraints.
Innovation Solution
Implementing depth-wise parallel processing techniques that allow for simultaneous processing of input portions at multiple depths, enhancing resource utilization and scalability by supporting both input and depth parallelism in neural-network architectures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If depth-sequential processing is used in neural-network-processing architectures, then device complexity is reduced and ease of manufacture is improved, but resource utilization deteriorates and productivity decreases
Solution Approach 1:
The processing architecture is segmented into multiple processing element groups, where each group can independently process different depth portions of input data. This segmentation enables parallel processing of multiple depths simultaneously, improving resource utilization without requiring a complete redesign of the entire processing system.
Solution Approach 2:
The patent introduces depth parallelism as an additional dimension of processing beyond traditional input parallelism. By organizing processing elements to handle different depth portions concurrently, the architecture transforms the processing paradigm from sequential depth handling to parallel depth handling, thereby improving productivity while maintaining manageable complexity through structured organization.
2Productivity
If dedicated hardware accelerators are used for machine learning processing, then processing capacity and productivity are improved, but use of energy and device packaging requirements worsen
Solution Approach 1:
The processing element groups are designed to be universal and reconfigurable, capable of handling different neural network operations and data formats. This multi-functionality allows the same hardware to process various machine learning tasks efficiently, improving processing capacity while reducing the need for specialized dedicated hardware that would consume more power and occupy more packaging space.
Solution Approach 2:
The architecture employs dynamic resource allocation where processing element groups can be activated or deactivated based on the specific processing requirements. This dynamic behavior allows the system to scale its processing capacity according to actual workload, avoiding the continuous power consumption of fully dedicated hardware accelerators while maintaining high productivity when needed.
3Device complexity
If data is moved across common data busses in accelerators, then device complexity is reduced, but use of energy increases and loss of time occurs due to latency
Solution Approach 1:
The patent implements local data storage and processing within each processing element group, reducing the need for frequent data transfers across common data busses. By keeping data locally where it is needed and enabling parallel processing, the architecture minimizes energy-consuming data movements while maintaining simple bus structures, thereby reducing power usage without significantly increasing device complexity.
Data Source
Figure 1A~1C
Figure 1D
Figure 2
AI summary
Methods and apparatus for performing machine learning tasks, and in particular, to a neural-network-processing architecture and circuits for improved performance through depth parallelism. One example neural-network-processing circuit generally includes a plurality of groups of processing element (PE) circuits, wherein each group of PE circuits comprises a plurality of PE circuits configured to process in parallel an input at a plurality of depths.