Neural SoC Bus Bandwidth Control to Prevent Data Starvation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems experience data starvation periods and inefficient bus bandwidth utilization due to unequal memory access and computation times across multiple neural processing units, leading to increased processing time and power consumption.
Innovation Solution
A controller dynamically prioritizes memory access operations based on the length of memory and computation cycles, adjusting bus bandwidth to prevent data starvation by reallocating resources to processing cores in need, using quality of service parameters and counters to manage bus usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If bus bandwidth is allocated equally to all processing cores, then device complexity is reduced, but data starvation periods occur and productivity decreases
Solution Approach 1:
The patent implements dynamic bus bandwidth allocation where the controller adjusts bandwidth distribution in real-time based on the computational status of processing cores. When a core is compute-bound, it receives more bandwidth; when memory-bound, less bandwidth is allocated. This dynamic adjustment prevents data starvation while optimizing overall system productivity without requiring complex static allocation mechanisms.
Solution Approach 2:
The system changes the bandwidth allocation parameter dynamically based on operational conditions. The controller monitors computation progress and memory access patterns, then adjusts the bus bandwidth parameter accordingly. This allows the system to adapt to varying computational demands of different neural network layers and operations, resolving the contradiction between simple allocation and high productivity.
2Reliability
If bus bandwidth is increased for memory access, then data starvation is prevented, but power consumption increases
Solution Approach 1:
The patent employs dynamic bandwidth allocation that adjusts bus resource allocation based on real-time computational needs. When processing cores are compute-bound and do not require additional data, bandwidth is reduced to save power. When cores are memory-bound and would experience data starvation, bandwidth is increased to ensure data availability. This dynamic approach resolves the contradiction between reliability and power consumption.
Solution Approach 2:
The system uses feedback from processing cores about their computational status to automatically adjust bandwidth allocation. The controller receives status information and autonomously adjusts bus resources without external intervention. This self-regulating mechanism ensures data availability only when needed, minimizing unnecessary power consumption while maintaining reliability.
3Productivity
If bus bandwidth is dynamically adjusted based on computational status, then productivity is improved, but device complexity increases
Solution Approach 1:
The patent changes the bandwidth allocation parameter dynamically based on monitored computational status. The controller adjusts bandwidth according to whether cores are compute-bound or memory-bound, optimizing throughput. The complexity is managed by implementing straightforward status monitoring and conditional bandwidth adjustment logic rather than complex allocation algorithms.
Solution Approach 2:
The system implements a feedback mechanism where the controller monitors computational status from processing cores and adjusts bandwidth accordingly. This feedback loop enables productivity improvement through adaptive allocation while keeping the control mechanism relatively simple by using direct status-to-bandwidth mapping rather than complex optimization algorithms.
4Device complexity
If equal memory access time is provided to all tensors, then device complexity is minimized, but loss of time occurs due to data starvation
Solution Approach 1:
The patent implements dynamic memory access time allocation where the controller adjusts access timing based on computational status. When a processing core is compute-bound and ready for more data, memory access time is reduced to prevent data starvation. When cores are memory-bound, access time is extended naturally. This dynamic approach eliminates data starvation delays without requiring complex scheduling mechanisms.
Solution Approach 2:
The system performs preliminary assessment of computational status before allocating memory access resources. The controller determines whether cores are compute-bound or memory-bound in advance, then pre-adjusts bandwidth allocation accordingly. This prevents data starvation by ensuring data is available when needed, reducing lost time without complex real-time intervention.
Data Source
AI summary
According to one example of the present disclosure, a system may be provided. The system may comprise at least one processing core configured to process computations of the at least one neural network model comprising at least one tensor, at least one memory circuit configured to store the at least one tensor, a bus circuit, electrically coupled to the at least one processing core and the at least one memory circuit, configured to transmit the at least one tensor based on a memory access operation instruction, and a controller configured to control a priority of a memory access operation for each tensor of the at least one processing core.