On-Chip Data Buffering for Stalled Network Returns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In spatial architectures, data requested by processing elements can be returned out of order, causing data to stall and potentially leading to a network stall, which affects the entire system's performance.
Innovation Solution
Implement auxiliary memory control circuitry to monitor the network for stalled data and transfer it to an auxiliary storage buffer dynamically selected from local storage sectors, reducing the likelihood of network stalls by managing data flow efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is copied to a different queue after stalling detection, then data processing continuity is improved, but queue management complexity increases
Solution Approach 1:
The patent segments the data flow into different queues based on stall detection. When a stall is detected in the first queue, data is copied to a second queue, effectively dividing the processing path into separate segments. This allows continuous processing while managing complexity through structured queue organization.
Solution Approach 2:
The patent introduces a copy operation as an intermediary mechanism between the first and second queues. This intermediary copy process enables seamless data transfer between queues, maintaining processing continuity while providing a controlled method for queue management.
2Reliability
If data is held in a queue until processing completes, then data integrity is improved, but memory resource consumption increases
Solution Approach 1:
The patent implements a mechanism where data is copied to a second queue and then discarded from the first queue's buffer. This allows the system to maintain data integrity through proper queue handling while reducing memory consumption by freeing up buffers in the first queue once data is transferred.
Solution Approach 2:
The patent performs preliminary copying of data to the second queue before the first queue's processing completes. This preliminary action ensures data integrity is maintained while allowing the first queue to release its buffer resources, thereby reducing overall memory consumption.
3Loss of time
If stall detection threshold is set low, then response time to stalls is improved, but false positive rate increases
Solution Approach 1:
The patent uses feedback mechanisms to monitor queue depth and determine when to trigger data copying. The system continuously monitors the first queue's buffer depth and uses this feedback to intelligently decide when a stall has occurred, reducing false positives while maintaining fast response time.
Solution Approach 2:
The patent implements dynamic threshold adjustment based on queue depth monitoring. Rather than using a fixed low threshold, the system dynamically determines when to trigger copying based on real-time queue conditions, allowing fast response to actual stalls while avoiding false positives through adaptive decision-making.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
There is provided a data processing apparatus and method. The data processing apparatus comprises a plurality of processing elements connected via a network arranged on a single chip to form a spatial architecture. Each processing element comprising processing circuitry to perform processing operations and memory control circuitry to perform data transfer operations and to issue data transfer requests for requested data to the network. The memory control circuitry is configured to monitor the network to retrieve the requested data from the network. Each processing element is further provided with local storage circuitry comprising a plurality of local storage sectors to store data associated with the processing operations, and auxiliary memory control circuitry to monitor the network to detect stalled data (S60). The auxiliary memory control circuitry is configured to transfer the stalled data from the network to an auxiliary storage buffer (S66) dynamically selected from amongst the plurality of local storage sectors (S64).