Embedded CPU Chip Operator Units Neural Network Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing chip-based data processing methods for neural networks face inefficiencies due to high overhead in data exchange between the server CPU and chip, particularly when handling multiple operator units and varying data lengths, leading to significant processing delays.
Innovation Solution
The introduction of an embedded CPU within the chip, which configures and invokes multiple operator units to perform computations based on to-be-processed data, allowing for efficient computation at each layer of the neural network using C language programs and asynchronous operations, and includes a DMA module for result transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If hardware acceleration method is used to shorten processing delay, then processing speed is improved, but device complexity increases due to need for flexible hardware design
Solution Approach 1:
The chip is segmented into multiple operator units, each dedicated to specific computation types (convolution, fully connected, activation functions). This segmentation allows parallel processing of different neural network operations simultaneously, improving processing speed while keeping each operator unit relatively simple in design.
Solution Approach 2:
The embedded CPU serves multiple functions: it configures operator units, manages data flow, handles memory operations, and coordinates the entire chip. This multi-functionality reduces the need for separate control hardware for each operator unit, simplifying overall device complexity while maintaining high processing speed.
2Adaptability or versatility
If multiple operator units are used to handle varying data lengths, then adaptability is improved, but data exchange overhead increases between server CPU and chip
Solution Approach 1:
Multiple operator units are merged into a single chip with shared memory and control resources. This consolidation allows the chip to process varying data lengths internally without requiring repeated data exchange with the server CPU, significantly reducing data exchange overhead time while maintaining adaptability to different input sizes.
Solution Approach 2:
The embedded CPU acts as an intermediary between the server CPU and operator units. It receives data from the server, configures appropriate operator units based on data characteristics, and manages data flow within the chip. This intermediary role eliminates the need for direct server-chip data exchange for each operation, reducing overhead time while maintaining adaptability.
3Manufacturing precision
If formatted binary instructions are used for each operator unit, then control precision is improved, but program instruction space increases
Solution Approach 1:
Instead of using separate formatted binary instructions for each operator unit, the system changes parameters dynamically through the embedded CPU. The CPU configures operator units by modifying their operational parameters (input/output addresses, computation type, data length) based on the specific task requirements. This parameter-based control achieves the same precision as formatted instructions while using significantly less program space.
Data Source
AI summary
Embodiments of the present specification provide chips and chip-based data processing methods. In an embodiment, a method comprises: obtaining data associated with one or more neural networks transmitted from a server; for each layer of a neural network of the one or more neural networks, configuring, based on the data, a plurality of operator units based on a type of computation each operator unit performs; and invoking the plurality of operator units to perform computations, based on neurons of a layer of the neural network immediately above, of the data for each neuron to produce a value of the neuron.