Deep Learning Operator Grouping on FPGA for Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The inefficiency in processing deep learning network models on Field Programmable Gate Array (FPGA) hardware due to system interruptions, register configuration, and context switching when operators are dispatched and operated one by one, leading to low operation efficiency.
Innovation Solution
Grouping consecutive target operators in a deep learning network model based on their operation sequences to enable simultaneous operation on the FPGA, reducing the number of dispatching times and optimizing the operation chart to enhance computation efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If operators are dispatched and operated one by one on FPGA and CPU, then each operator can be processed individually, but system interruption, register configuration and context switching occur frequently, consuming extra time and reducing operation efficiency
Solution Approach 1:
The patent merges multiple consecutive operators into a single operator group that can be executed together on the FPGA. Instead of dispatching each operator individually between CPU and FPGA, the system combines multiple operators into one unified operation unit, eliminating repeated context switching and register configuration overhead.
Solution Approach 2:
The system performs preliminary analysis to identify and group consecutive operators that can be executed together before actual execution. By pre-processing the operator sequence and forming operator groups in advance, the system avoids the need for repeated dispatching and configuration during runtime, thus reducing time consumption.
2Productivity
If multiple operators are grouped and operated within the same target operation period on the same processor, then operation efficiency is improved, but the complexity of operator grouping and scheduling increases
Solution Approach 1:
The patent segments the sequence of operators into distinct operator groups based on their consecutiveness and compatibility for parallel execution. Each operator group is treated as an independent unit that can be dispatched together, simplifying the scheduling process while maintaining high computation efficiency.
Solution Approach 2:
The system changes the granularity parameter of operator execution from individual operators to operator groups. By adjusting this parameter, the system can control the size and composition of operator groups, optimizing the balance between computation efficiency and scheduling complexity based on specific workload characteristics.
Data Source
AI summary
Provided are a method for processing data, an electronic device and a storage medium, which relate to the field of deep learning and data processing. The method may include: multiple target operators of a target model are acquired; the multiple target operators are divided into at least one operator group, according to an operation sequence of each of the multiple target operators in the target model, wherein at least one target operator in each of the at least one operator group is operated by the same processor and is operated within the same target operation period; and the at least one operator group is output.