Neural Network Accelerator Command Processor Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current neural network execution technologies face inefficiencies due to the limitations of general-purpose processors in handling large neural network operations, particularly in matrix operations and convolutions, which require frequent external memory access and result in low processing efficiency.
Innovation Solution
A hardware accelerator is designed with a direct memory access module, on-chip memory, and a command processor that decomposes neural network operations into sub-operations, converts them into instruction sequences executable on processing clusters, and manages data distribution to improve parallel execution and reduce memory access latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If general-purpose processors are used to execute neural network operations, then versatility is maintained, but processing speed and execution efficiency deteriorate
Solution Approach 1:
The processor is segmented into a command processor and multiple processing clusters, where the command processor decomposes operations and distributes them to specialized processing clusters. This segmentation allows different parts of the system to handle different tasks optimally, maintaining versatility while improving speed.
Solution Approach 2:
The patent transitions from a single-dimensional general-purpose processing approach to a multi-dimensional architecture with command processing, data distribution, and parallel execution dimensions. This dimensional expansion enables both versatility and high-speed specialized processing.
2Device complexity
If external memory access is used for neural network operations, then device complexity is reduced, but memory access frequency increases and processing efficiency deteriorates
Solution Approach 1:
The patent implements a nested memory hierarchy with on-chip memory units embedded within processing clusters, which are nested within the broader memory system. This nested structure reduces the need for external memory access by providing fast local storage, thereby improving processing efficiency while managing complexity through hierarchical organization.
Solution Approach 2:
The on-chip memory units act as intermediaries between the external memory and the processing units. They buffer and manage data locally, reducing the frequency of external memory accesses and improving processing efficiency without significantly increasing overall device complexity.
3Productivity
If operations are decomposed into sub-operations and executed in parallel, then execution efficiency improves, but device complexity increases
Solution Approach 1:
The processor architecture is segmented into a command processor that handles decomposition and multiple processing clusters that execute sub-operations in parallel. This segmentation enables efficient parallel execution while managing complexity by distributing functions across specialized units.
Solution Approach 2:
The processing clusters are designed with multi-functionality to handle various types of neural network operations. This universality allows the same hardware structure to execute different sub-operations, reducing the need for highly specialized components and thereby controlling device complexity while maintaining high execution efficiency.
Data Source
AI summary
Disclosed are an acceleration unit for executing a neural network model and a server. The acceleration unit includes: a plurality of cluster groups, where each of the cluster groups includes a plurality of processing clusters; an on-chip memory, including a plurality of storage units, where each storage unit corresponds to each of the cluster groups, and is configured to store an instruction sequence and operation data of the corresponding cluster group; a command processor, configured to decompose an operation associated with a specified neural network model into a plurality of sub-operations, convert the plurality of sub-operations into a plurality of instruction sequences, specify operation data of each of the instruction sequences; and a plurality of distribution units, where each distribution unit reads the instruction sequence and operation data of the instruction sequence from the corresponding storage unit into the corresponding cluster group.