Deep Learning Node Scheduling for Hardware Utilization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning technologies face inefficiencies due to the low efficiency and high flexibility of conventional instruction set architectures, which fail to meet the increasing demand for computing power, leading to suboptimal performance and resource underutilization.
Innovation Solution
A scheduling method and apparatus based on deep learning node computation that asynchronizes graph reasoning processes, allowing for the efficient utilization of both software and hardware resources by distinguishing between hardware and software computation nodes and scheduling them accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a data flow architecture is used to improve computing efficiency, then computing efficiency is improved, but hardware utilization is reduced due to software nodes causing hardware to idle
Solution Approach 1:
The system dynamically adjusts the execution mode of computation nodes based on real-time hardware availability and node characteristics. Software nodes are executed on hardware when available to maximize utilization, while maintaining the flexibility to switch between software and hardware execution modes as conditions change, thus resolving the contradiction between efficiency and utilization.
Solution Approach 2:
An asynchronous graph computation mechanism is introduced as an intermediary layer between software nodes and hardware resources. This mechanism queues computation nodes and manages their execution timing, allowing hardware to remain busy by processing nodes asynchronously rather than idle等待 software node completion, thereby improving both efficiency and hardware utilization.
2Adaptability or versatility
If software nodes are used for auxiliary processing, then flexibility is improved, but computing speed deteriorates due to lack of hardware acceleration
Solution Approach 1:
The system makes software nodes multi-functional by enabling them to execute on both software environments and hardware accelerators. Computation nodes can dynamically switch between software-only execution (maintaining flexibility) and hardware-accelerated execution (improving speed), thus resolving the contradiction between adaptability and computing speed.
3Device complexity
If graph computation is run in a single thread, then simplicity is maintained, but resource utilization deteriorates due to hardware idling
Solution Approach 1:
The graph computation process is segmented into multiple independent computation nodes that can be executed in parallel. Instead of a single sequential thread, the system divides the computation graph into discrete nodes that can be queued and executed asynchronously on multiple hardware resources, improving resource utilization while maintaining manageable system complexity through modular design.
Solution Approach 2:
The asynchronous execution mechanism ensures continuous utilization of hardware resources by maintaining a queue of computation nodes ready for execution. Hardware accelerators remain continuously busy processing nodes from the queue rather than idling between tasks, achieving continuous useful action while keeping the overall system architecture relatively simple.
Data Source
AI summary
Provided are a scheduling method and apparatus based on a deep learning node computation, and a storage medium. The scheduling method includes: a to-be-computed node of a preset neural network computation graph is acquired; a node type of the to-be-computed node is determined, where the node type includes a hardware computation node and a software computation node; in a case where the node type is the hardware computation node, the hardware computation node is scheduled to a first queue, and whether a hardware computing power module corresponding to the hardware computation node is occupied or not is determined; and in a case where the hardware computing power module is not occupied, the hardware computation node is input into the hardware computing power module for computing.