Dynamic Thread Allocation for Low-Latency Compute-Near-Memory Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer architectures face challenges in efficiently processing large tasks in parallel due to significant time and energy consumption in data movement between processors and memory, constraining performance and capacity.
Innovation Solution
A compute-near-memory system employs a divide and conquer strategy with a hybrid threading processor that allows threads to participate in scheduling, using a logarithmic thread creation tree and 'busy fail' and 'no return' thread variants to optimize thread management, enabling efficient parallel processing of large tasks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a shared memory architecture with bus access is used, then data storage and access are simplified, but data movement between processors and memory consumes significant time and energy
Solution Approach 1:
The patent segments the monolithic processor architecture into multiple compute elements distributed across the memory array. Each compute element is a self-contained unit that can independently access and process data from the memory in which it is embedded, eliminating the need for centralized bus access and reducing energy consumption for data movement.
Solution Approach 2:
The patent transitions from a traditional von Neumann architecture with separate processor and memory to a three-dimensional stacked architecture where compute elements are embedded within or adjacent to memory layers. This vertical integration creates short-distance data paths that dramatically reduce the energy and time required for data movement compared to horizontal bus access.
2Ease of operation
If traditional thread scheduling is used, then thread management is straightforward, but scheduling overhead and latency increase for large numbers of threads
Solution Approach 1:
The patent implements self-service thread management where compute elements autonomously select and execute tasks from shared task queues without requiring centralized scheduler intervention. Each compute element monitors task availability and independently decides when to start, pause, or switch tasks, eliminating scheduling overhead and latency associated with traditional centralized thread management.
Solution Approach 2:
The patent employs dynamic task allocation where task queues and their locations in memory are not fixed but adaptively reconfigured based on current workload characteristics and compute element availability. This dynamic approach allows the system to optimize task distribution in real-time, reducing latency and improving throughput for varying workload patterns.
Data Source
AI summary
Devices and techniques for thread scheduling control and memory splitting in a processor are described herein. An apparatus includes a hardware interface configured to receive a first request to execute a first thread, the first request including an indication of a workload; and processing circuitry configured to: determine the workload to produce a metric based at least in part on the indication; compare the metric with a threshold to determine that the metric is beyond the threshold; divide, based at least in part on the comparison, the workload into a set of sub-workloads consisting of predefined number of equal parts from the workload; create a second request to execute a second thread, the second request including a first member of the set of sub-workloads; and process a second member of the set of sub-workloads in the first thread.