Neural Task Manager Circuit Prioritizes ML Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently managing neural network tasks on conventional central processing units (CPUs), leading to high bandwidth consumption and increased power usage.
Innovation Solution
A neural processor circuit is introduced, comprising one or more neural engine circuits and a neural task manager circuit. The neural task manager includes multiple task queue circuits and a task arbiter circuit, which prioritize tasks, retrieve configuration data from external memory, and program the neural engines to execute tasks efficiently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a CPU is used to execute machine learning operations, then ease of operation is improved, but power consumption increases
Solution Approach 1:
The system segments machine learning operations into distinct tasks organized in task queues with different priorities. The task manager divides the execution workflow into manageable units that can be scheduled and executed efficiently, separating the management function from the execution function to optimize resource usage and reduce power consumption.
Solution Approach 2:
The system changes the operational parameters by introducing a task-based execution model with priority levels and configuration data retrieval mechanisms. This transforms the CPU's operation mode from continuous execution to scheduled task execution, improving ease of operation while reducing unnecessary power consumption through selective task processing.
2Ease of operation
If a CPU is used to execute machine learning operations, then ease of operation is improved, but bandwidth consumption increases
Solution Approach 1:
The system performs preliminary actions by pre-organizing machine learning operations into task queues with configuration data stored in memory. Task descriptors are prepared in advance with all necessary parameters, allowing the CPU to execute tasks without repeated data retrieval, thereby reducing bandwidth consumption while maintaining ease of operation.
Solution Approach 2:
The task manager acts as an intermediary between the CPU and machine learning operations. It manages task queues, retrieves configuration data, and coordinates execution, thereby reducing direct bandwidth requirements between the CPU and memory by batching and optimizing data access patterns.
3Adaptability or versatility
If multiple machine learning operations are managed without prioritization, then adaptability is improved, but productivity decreases
Solution Approach 1:
The system introduces dynamic task prioritization where different task queues can be assigned different priority levels based on their importance and urgency. This dynamic scheduling mechanism allows the system to adapt to varying operational requirements while maintaining high productivity by ensuring critical tasks are executed first.
Solution Approach 2:
The task manager implements periodic action by systematically retrieving and executing tasks from prioritized queues in a structured manner. This periodic task processing ensures that multiple machine learning operations are managed adaptively while maintaining consistent productivity through organized, sequential execution.
Data Source
AI summary
Embodiments relate to managing tasks that when executed by a neural processor circuit instantiates a neural network. The neural processor circuit includes neural engine circuits and a neural task manager circuit. The neural task manager circuit includes multiple task queues and a task arbiter circuit. Each task queue stores a reference to a task list of tasks for a machine learning operation. Each task queue may be associated with a priority parameter. Based on the priority of the task queues, the task arbiter circuit retrieves configuration data for a task from a memory external to the neural processor circuit, and provides the configuration data to components of the neural processor circuit including the neural engine circuits. The configuration data programs the neural processor circuit to execute the task. For example, the configuration data may include input data and kernel data processed by the neural engine circuits to execute the task.