Texture Unit Circuit Indirection for Neural Network Processor Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processors rely solely on central processing units (CPUs) for operations, leading to significant bandwidth consumption and increased power usage.
Innovation Solution
A texture unit circuit within a neural processor circuit is introduced, which applies a level of indirection to input surfaces from system memory, enabling flexible higher-level texture transforms and indirectly accessing source tensors using index tensors, thereby offloading operations from the CPU.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a CPU is used to execute machine learning operations, then flexibility in configuring different models is achieved, but CPU bandwidth consumption increases significantly
Solution Approach 1:
The neural network processor is divided into multiple independent neural engine circuits, each capable of executing specific convolution operations. This segmentation allows parallel processing of different computational tasks, reducing the bandwidth requirement on any single CPU while maintaining overall system flexibility.
Solution Approach 2:
A texture unit circuit is introduced as an intermediary between the CPU and the neural engine circuits. This texture unit handles data loading and texture mapping operations, offloading bandwidth-intensive tasks from the CPU and enabling flexible model configuration without proportionally increasing CPU bandwidth consumption.
2Adaptability or versatility
If a CPU executes machine learning operations, then model configuration flexibility is maintained, but power consumption increases
Solution Approach 1:
The processing workload is segmented and distributed across multiple neural engine circuits that can operate independently. This allows the system to execute model operations with lower power consumption by utilizing specialized hardware paths rather than relying solely on the CPU, while maintaining flexibility through configurable neural engine architecture.
Solution Approach 2:
The texture unit circuit serves as an intermediary that manages data access and texture operations with optimized power consumption. By handling these tasks in dedicated hardware rather than through CPU instructions, the system reduces overall power consumption while preserving model configuration flexibility.
3Adaptability or versatility
If a texture unit circuit with indirection is introduced, then flexible texture transforms are enabled, but device complexity increases
Solution Approach 1:
The texture unit circuit is designed with multi-functionality, handling various texture transform operations through a unified architecture. By implementing universal functionality for different transform types rather than separate dedicated circuits for each operation, the system achieves flexible texture transforms while minimizing the increase in overall device complexity.
Data Source
AI summary
Embodiments of the present disclosure relate to a texture unit circuit in a neural processor circuit. The neural processor circuit includes a tensor access operation circuit with the texture unit circuit, a data processor circuit, and at least one neural engine circuit. The texture unit circuit fetches a source tensor from a system memory by referencing an index tensor in the system memory representing indexing information into the source tensor. The data processor circuit stores an output version of the source tensor obtained from the tensor access operation circuit and sends the output version of the source tensor as multiple of units of input data to the at least one neural engine circuit. The at least one neural engine circuit performs at least convolution operations on the units of input data and at least one kernel to generate output data.