Neural Processor Task Switching with Priority Queues for Lower CPU Load
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems relying solely on central processing units (CPUs) for neural network operations consume significant bandwidth and increase power consumption due to resource-intensive operations.
Innovation Solution
A neural processor circuit with a neural task manager that switches between task queues based on priority parameters, utilizing neural engines to perform neural operations efficiently, thereby reducing CPU load and power consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a central processing unit (CPU) is used to execute machine learning operations, then the system can handle various configurations of neural networks, but the CPU bandwidth consumption increases significantly and power consumption increases
Solution Approach 1:
The system segments machine learning operations into distinct task queues (first task queue, second task queue) that can be independently managed and executed by the neural processor circuit. This segmentation allows the CPU to offload specific neural network operations while maintaining overall system flexibility, thereby reducing CPU bandwidth consumption and power consumption without sacrificing configuration adaptability.
2Adaptability or versatility
If a central processing unit (CPU) is used to execute machine learning operations, then the system can handle various configurations of neural networks, but the CPU bandwidth consumption increases significantly
Solution Approach 1:
The neural processor circuit acts as an intermediary between the CPU and the execution of neural network operations. The task manager circuit receives and manages task queues, allowing the CPU to maintain flexibility in configuring neural networks while the neural processor handles the actual execution. This intermediary structure frees up CPU bandwidth while preserving configuration adaptability.
3Use of energy by moving object
If task switching is implemented in the neural processor circuit, then resource utilization is optimized and power consumption is reduced, but the device complexity increases
Solution Approach 1:
The neural processor circuit is designed with universal components that can handle multiple functions: the task manager circuit manages multiple task queues, the neural engine circuit executes various neural network operations, and the same hardware infrastructure supports different configurations of neural networks. This multi-functionality reduces the need for separate dedicated circuits for each function, thereby limiting the increase in device complexity while achieving power consumption reduction through efficient task switching.
Data Source
AI summary
Embodiments relate to managing tasks that when executed by a neural processor circuit instantiates a neural network. A neural task manager circuit within the neural processor circuit can switch between tasks in different task queues. Each task queue is configured to store a reference to a task list of tasks for instantiating a neural network. Each task queue can also be assigned a priority parameter. While the neural processor circuit is executing tasks of a first task list and prior to completion of each task, the neural task manager circuit can switch between task queues according to the priority parameters for execution of tasks of a second task list by the neural processor circuit. The neural processor circuit includes one or more neural engine circuits that are configured to perform neural operations by executing the tasks assigned by the task manager.