Vector Processing Scheduler Circuit for Operator Allocation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Vector processing in existing systems is inefficient due to sequential execution of operators, which loses parallelism and requires impractical intermediate storage, making it difficult to manage large vectors effectively.
Innovation Solution
A scheduler circuit and operator allocation circuit that parse a directed acyclic graph, track operator and hardware engine readiness, and allocate operators to available hardware resources based on resource type, status signals, and priority, enabling efficient allocation and processing without preemption or with preemption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If operators are executed in strict sequential fashion, then execution simplicity is maintained, but parallelism is lost and processing efficiency deteriorates
Solution Approach 1:
The patent segments the operator execution into independent units that can be processed in parallel. Each operator is treated as a discrete task that can be allocated to different hardware engines simultaneously, enabling parallel processing while maintaining manageable complexity through structured allocation protocols
Solution Approach 2:
The scheduler performs preliminary actions by pre-calculating operator dependencies and allocating operators to hardware engines before execution begins. This preliminary scheduling enables efficient parallel execution by establishing the execution plan in advance, resolving the complexity of managing parallel operations during runtime
2Quantity of substance
If full vectors are stored as intermediate results, then data availability for downstream operators is ensured, but intermediate storage requirements become impractical
Solution Approach 1:
The patent applies partial action by processing and storing only the necessary portions of vectors as intermediate results rather than full vectors. This selective storage approach reduces intermediate storage requirements while maintaining sufficient data availability for downstream operators, resolving the contradiction between storage capacity and processing efficiency
Solution Approach 2:
The system extracts only the essential data needed for downstream processing from the full vector results. By taking out only the necessary portions as intermediates rather than storing complete vectors, the system reduces storage burden while maintaining processing efficiency
3Productivity
If operators are allocated without considering readiness status, then allocation speed is increased, but allocation accuracy and resource utilization deteriorate
Solution Approach 1:
The scheduler continuously monitors readiness status of both operators and hardware engines, using this feedback to make informed allocation decisions. This feedback mechanism ensures accurate and efficient allocation by matching ready operators with ready hardware engines, resolving the contradiction between allocation speed and accuracy
Solution Approach 2:
The allocation system dynamically adjusts based on real-time readiness status changes. Operators and hardware engines transition between ready and not-ready states, and the scheduler dynamically responds to these changes, ensuring optimal allocation decisions that balance speed and accuracy
Data Source
AI summary
An apparatus includes a scheduler circuit and an operator allocation circuit. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more operators, (ii) track a plurality of first status signals indicating a readiness state of a plurality of unscheduled operators that are to be allocated to a plurality of hardware engines, (iii) track a plurality of second status signals indicating a readiness state of the hardware engines, and (iv) for each operator, track a resource type parameter. The operator allocation circuit may be configured to (a) select a resource type from a list of resource types in use, (b) determine available hardware engines corresponding to the selected resource type based on the second status signals, (c) generate scores for the unscheduled operators based on (i) the selected resource type, (ii) the first status signals, and (iii) the resource type parameters, and (d) allocate at least one of the unscheduled operators to at least one of the available hardware engines based on the scores.