Multi-Dimensional Tensor Mapping Circuit for CPU Load Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems rely heavily on central processing units (CPUs), leading to significant bandwidth consumption and increased power usage due to the extensive computing operations required for machine learning tasks.
Innovation Solution
A tensor access operation circuit is introduced in a neural processor circuit, which indirectly accesses regions of a source tensor in system memory and maps components into an input tensor, allowing for efficient convolution operations through a data processor circuit and neural engine circuit, reducing the load on CPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but use of energy and bandwidth consumption increase
Solution Approach 1:
A dedicated neural processing unit (NPU) is introduced as an intermediary between the CPU and system memory. The NPU includes a tensor access operation circuit that indirectly accesses source tensor data from system memory and performs neural network operations locally, reducing the need for CPU intervention and minimizing bandwidth consumption while maintaining ease of operation through hardware acceleration
2Ease of operation
If CPU is used to perform neural network operations, then ease of operation is improved, but bandwidth consumption increases
Solution Approach 1:
The tensor access operation circuit acts as an intermediary that directly accesses system memory for source tensor data and performs operations locally within the NPU. This eliminates the need for continuous CPU-mediated data transfers, significantly reducing bandwidth consumption while maintaining operational simplicity
Solution Approach 2:
The processing system is segmented into distinct functional units: the CPU for control and coordination, the NPU for specialized neural network operations, and system memory for data storage. This segmentation allows the NPU to independently access memory and perform operations without burdening the CPU, reducing overall bandwidth consumption
3Adaptability or versatility
If tensor access operation circuit indirectly accesses source tensor, then flexibility in texture transforms is improved, but device complexity increases
Solution Approach 1:
The tensor access operation circuit serves as an intermediary between system memory and the neural processing units. It indirectly accesses source tensor data and performs rank mapping transformations, providing flexible texture transforms for different neural network configurations while encapsulating the complexity within the dedicated circuit rather than requiring complex CPU software implementations
Data Source
AI summary
Embodiments of the present disclosure relate to a tensor access operation circuit in a neural processor circuit. The neural processor circuit further includes a data processor circuit and at least one neural engine circuit. The tensor access operation circuit indirectly accesses at least a region of a source tensor in a system memory having a rank, and maps one or more source components of the source tensor into an input tensor having another rank. The data processor circuit stores an output version of the input tensor obtained from the tensor access operation circuit and sends the output version of the input tensor as multiple of units of input data to the at least one neural engine circuit. The at least one neural engine circuit performs at least convolution operations on the units of input data and at least one kernel to generate output data.