Neural Network Data Processing Circuit for CPU-Free Layer Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network processors experience significant efficiency losses due to the delay caused by frequent data exchange between the neural network processor and the CPU, particularly when handling operations beyond convolution and fully connected layers.
Innovation Solution
A data processing circuit with an input control circuit that can perform a combination of logarithmic, exponential, addition, comparison, and multiplication operations, allowing for more complex computations within the circuit, thereby reducing the need for CPU processing and minimizing data exchange.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the neural network processor is designed to handle only convolution and fully connected layers, then the computing engine can be optimized for high-density multiplication and accumulation operations, but the processor cannot handle other neural network layer operations independently
Solution Approach 1:
The computing engine is designed with multi-functionality to handle not only convolution and fully connected layers but also other neural network layer operations such as activation functions, normalization, and pooling operations. This universal design allows a single hardware architecture to perform diverse computational tasks without requiring separate dedicated circuits for each operation type.
Solution Approach 2:
The computing engine incorporates dynamic configuration capabilities that allow it to adapt its operational mode based on the specific neural network layer being processed. Through programmable control, the engine can dynamically switch between different computation modes (matrix multiplication, element-wise operations, etc.) to optimize performance for varying workloads while maintaining a unified hardware structure.
2Ease of operation
If data needs to be transmitted back and forth between the neural network processor and the CPU for each computation at other layers, then the CPU can provide flexible control, but a transmission delay arises that greatly decreases computing efficiency
Solution Approach 1:
The computing engine is designed with self-service capabilities to autonomously execute computations for various neural network layers without requiring constant CPU intervention. The processor can independently handle data processing for activation functions, normalization, and other operations, eliminating the need for frequent data back-and-forth transmissions to the CPU and significantly reducing transmission delays.
Solution Approach 2:
The computing engine incorporates preliminary computation capabilities that allow it to prepare and process data locally before final output is needed. By performing preliminary operations such as activation functions and normalization internally, the system reduces the amount of data that needs to be transmitted to the CPU, thereby minimizing transmission time and improving overall computing efficiency.
3Productivity
If the computing engine is designed for high-density multiplication and accumulation operations, then it can efficiently handle convolution and fully connected layers, but it cannot accommodate more complex operations requiring logarithmic, exponential, or other non-standard functions
Solution Approach 1:
The computing engine integrates universal computational primitives that can perform not only multiplication and accumulation operations but also complex mathematical functions such as logarithmic, exponential, and trigonometric operations. This multi-functional design enables the same hardware to handle diverse computational requirements across different neural network layers, from standard dense operations to more complex mathematical transformations.
Solution Approach 2:
The computing engine utilizes parameter changes and configurable operation modes to adapt its behavior for different types of computations. By dynamically adjusting operational parameters and selecting from multiple computation algorithms, the engine can efficiently perform both high-density multiplication operations and complex mathematical functions, optimizing performance across various workloads without requiring separate dedicated hardware for each operation type.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
A data processing circuit is disclosed. The data processing circuit relates to the field of digital circuits, and includes a first computing circuit and an input control circuit. The first computing circuit includes one or more computing sub-circuits. Each computing sub-circuit includes a first addition operation circuit, a multiplication operation circuit, a first comparison operation circuit, and a first nonlinear operation circuit. The first nonlinear operation circuit includes at least one of an exponential operation circuit and a logarithmic operation circuit. The input control circuit is configured to: control the first computing circuit to read input data and an input parameter, and control, according to a received first instruction, the operation circuit in the computing sub-circuit included in the first computing circuit, to perform an operation on the input data and the input parameter. The data processing circuit can not only implement a basic multiplication and accumulation operation, but also implement a more complex operation such as a logarithmic operation, an exponential operation, or an exponentiation operation by using the first nonlinear operation circuit, thereby adapting to computing of more neural network layers.