Multicore Neural Thread Scheduling Without a Centralized Scheduler
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network processing in multicore processors is inefficient due to hardware limitations, such as pin-limited memory access and computational complexity, leading to high energy consumption and inefficiencies in handling large and complex neural networks, especially in embedded device implementations.
Innovation Solution
The proposed solution involves a thread-based scheduling method within multicore neural network processors, utilizing a scoreboard to manage thread dependencies, a ready queue to identify unblocked threads, and asynchronous communication between cores to decouple processing and reduce data path dependencies, enabling efficient parallelism and low-power operation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If general-purpose programming environments are used to emulate neural networks, then implementation ease is improved, but processing efficiency deteriorates due to hardware limitations and compiler design
Solution Approach 1:
The system segments neural network processing into multiple independent threads that can be executed in parallel across multiple cores. Each thread represents a discrete computational unit that can be scheduled independently, allowing the system to maintain implementation simplicity while achieving high processing efficiency through parallel execution.
Solution Approach 2:
The patent introduces a scoreboard as an intermediary data structure that manages thread execution state and dependencies. This scoreboard acts as a mediator between the general-purpose programming environment and the underlying hardware, translating high-level neural network operations into efficient parallel execution while maintaining ease of implementation.
2Device complexity
If pin-limited memory access is used in multicore processors, then device complexity is reduced, but energy consumption increases due to memory access overhead
Solution Approach 1:
The system performs preliminary actions by pre-fetching and caching data into local memory buffers before it is needed for computation. This allows threads to access data from fast local memory rather than repeatedly accessing external memory through pin-limited interfaces, reducing energy consumption while maintaining simple device architecture.
Solution Approach 2:
The patent introduces a new dimension of memory hierarchy by implementing local memory buffers at each core. This creates a multi-level memory architecture where data can be accessed locally or externally, reducing the frequency of pin-limited memory accesses and lowering energy consumption without increasing overall device complexity.
3Productivity
If thread-based scheduling with scoreboard is implemented, then processing efficiency is improved through parallelism, but device complexity increases
Solution Approach 1:
The scoreboard is designed as a universal data structure that serves multiple functions: tracking thread execution state, managing data dependencies, coordinating inter-core communication, and optimizing memory access. This multi-functionality allows the system to achieve high processing efficiency through parallelism while minimizing the additional device complexity required.
4Productivity
If asynchronous communication between cores is used, then productivity is improved by decoupling processing, but loss of information increases due to synchronization challenges
Solution Approach 1:
The scoreboard implements a feedback mechanism where each core continuously reports its execution state and data dependency status to the scheduling system. This feedback allows the system to maintain accurate synchronization information despite asynchronous communication between cores, preventing information loss while enabling high productivity through decoupled processing.
Data Source
AI summary
Systems, apparatus, and methods for thread-based scheduling within a multicore processor. Neural networking uses a network of connected nodes (aka neurons) to loosely model the neuro-biological functionality found in the human brain. Various embodiments of the present disclosure use thread dependency graphs analysis to decouple scheduling across many distributed cores. Rather than using thread dependency graphs to generate a sequential ordering for a centralized scheduler, the individual thread dependencies define a count value for each thread at compile-time. Threads and their thread dependency count are distributed to each core at run-time. Thereafter, each core can dynamically determine which threads to execute based on fulfilled thread dependencies without requiring a centralized scheduler.