Neural Processor Task Skew Management via Buffer Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing systems face inefficiencies due to the difference in processing times between convolution and non-convolution operations, leading to task skew between neural engine tasks and planar engine tasks, which can result in resource imbalances and increased power consumption.
Innovation Solution
A neural processor circuit with separate neural engine and planar engine circuits that perform operations in parallel, managed by a data processor circuit with buffer queues to prevent task skew through threshold-based task addition, ensuring balanced processing loads between the two types of tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If neural engine circuits and planar engine circuits process tasks in parallel without task skew management, then processing throughput is improved, but resource imbalance and power consumption increase
Solution Approach 1:
The data processor circuit continuously monitors the depth of task queues in both neural engine and planar engine buffers, using this feedback information to dynamically control task allocation. When one engine type falls behind, the system adjusts incoming task distribution to balance the workload, preventing resource idle time and optimizing power consumption.
Solution Approach 2:
The system dynamically adjusts task queue management based on real-time processing status. The data processor circuit modifies task allocation policies on-the-fly, changing which engine receives incoming tasks based on current queue depths and processing speeds, thereby maintaining balanced utilization and efficient power usage.
2Productivity
If task queues are allowed to grow without limits, then more tasks can be processed, but buffer overflow and resource imbalance occur
Solution Approach 1:
The system pre-establishes threshold values for task queue depths in both neural engine and planar engine buffers. Before buffer overflow can occur, the data processor circuit detects when queues approach these predetermined thresholds and proactively adjusts task allocation to prevent overflow, ensuring reliable operation.
Solution Approach 2:
The system takes preliminary counter-actions by stopping task addition to overloaded queues before problems occur. When a buffer reaches its threshold, the data processor circuit immediately prevents further tasks from being added to that queue, counteracting the tendency toward overflow and maintaining system stability.
3Device complexity
If all incoming tasks are directed to one engine type, then task allocation is simplified, but processing efficiency decreases due to task skew
Solution Approach 1:
The system segments the incoming task stream and directs different portions to neural engine or planar engine based on current processing status. Instead of a single allocation rule, tasks are divided and distributed to different engine types according to real-time queue depth comparisons, balancing simplicity with efficiency.
Solution Approach 2:
The data processor circuit acts as an intermediary between incoming tasks and the two engine types. It mediates task allocation by evaluating queue depths and determining optimal destination, simplifying the overall system while maintaining high processing efficiency through intelligent routing.
Data Source
AI summary
Embodiments relate to a neural processor circuit including one or more planar engine circuits that perform non-convolution operations in parallel with convolution operations performed by one or more neural engine circuits. The neural engine circuits perform the convolution operations on neural input data corresponding to one or more neural engine tasks to generate neural output data. The planar engine circuits perform non-convolution operations on planar input data corresponding to one or more planar engine tasks to generate planar output data. A data processor circuit that includes multiple buffer circuits performs task skew management between the one or more neural engine tasks and the one or more planar engine tasks. The data processor circuit stops addition of an incoming task to queues in response to one or more of the queues stored in the buffer circuits reaching a threshold.