Neural Processor Circuit for Convolution Work Unit Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems that perform convolution neural network operations rely heavily on central processing units (CPUs), leading to significant bandwidth consumption and increased power usage.
Innovation Solution
A neural processor circuit is designed, comprising multiple neural engine circuits, a data buffer, and a kernel fetcher circuit. This configuration allows for efficient convolution operations by processing work units of input data and kernel data, with the data buffer managing data transfer between the neural engine circuits and system memory, and the kernel fetcher circuit handling kernel data retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU and main memory are used to execute machine learning systems, then ease of operation is improved, but power consumption and bandwidth consumption increase
Solution Approach 1:
The patent introduces a neural processor circuit as an intermediary component between the CPU and the machine learning operations. This dedicated neural processor handles the computationally intensive convolution operations, thereby reducing the workload on the CPU and lowering overall power consumption while maintaining ease of operation through centralized processing.
Solution Approach 2:
The patent segments the machine learning processing workload into separate functional components: a neural processor circuit for convolution operations, a data buffer for data management, and a kernel fetcher circuit for parameter retrieval. This segmentation allows each component to be optimized independently, reducing overall system power consumption while maintaining operational simplicity.
2Ease of operation
If CPU and main memory are used to execute machine learning systems, then ease of operation is improved, but bandwidth consumption increases
Solution Approach 1:
The data buffer acts as an intermediary between the neural processor circuit and system memory, caching frequently accessed input data and kernel parameters. This reduces the bandwidth required between these components while maintaining ease of operation through automated data management.
Solution Approach 2:
The kernel fetcher circuit performs preliminary actions by pre-fetching and storing kernel parameters in the data buffer before they are needed for convolution operations. This reduces real-time bandwidth consumption while maintaining operational simplicity through automated data preparation.
3Use of energy by moving object
If neural processor circuit with data buffer and kernel fetcher is used, then power consumption is reduced, but device complexity increases
Solution Approach 1:
The data buffer serves multiple functions: it stores input data for neural engine circuits, caches kernel parameters, and manages data transfer between memory and processing units. This multi-functionality reduces the need for separate dedicated components, thereby reducing overall device complexity while maintaining low power consumption.
Solution Approach 2:
The patent merges the data storage and data management functions into a single data buffer component, and combines the kernel parameter retrieval and distribution functions into a dedicated kernel fetcher circuit. This merging reduces the number of separate components and simplifies the overall device architecture while maintaining efficient low-power operation.
4Productivity
If neural processor circuit with parallel processing is used, then processing efficiency is improved, but device complexity increases
Solution Approach 1:
The neural processor circuit is segmented into multiple independent neural engine circuits that can process different portions of the input data simultaneously. Each neural engine circuit handles a specific work unit, enabling parallel processing and improving overall processing efficiency while maintaining manageable complexity through modular architecture.
Data Source
AI summary
Embodiments relate to a neural processor circuit that includes multiple neural engine circuits, a data buffer, and a kernel fetcher circuit. At least one of the neural engine circuits receives multiple sub-channels of a portion of input data from the data buffer. Neural engine circuit further receives a kernel of the one or more kernels from the kernel fetcher circuit, wherein the kernel was decomposed into a corresponding sub-kernel for each sub-channel of the portion of the input data. Neural engine circuit performs a convolution operation on each sub-channel of the portion of the input data and the corresponding sub-kernel. Neural engine circuit accumulates corresponding outputs of each sub-channel portion of the convolution operation to generate a single channel of the output data.