Unified Instruction Queue for Data-Dependent Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning and big data processing systems face challenges in efficiently executing compute-intensive operations due to the limitations of conventional CPU computational power, leading to the need for specialized processors like GPUs and ASICs, which often require multiple stand-alone executing components and complex instruction management.
Innovation Solution
A method and apparatus for executing instructions that involve acquiring an instruction queue, determining instruction types, and routing arithmetic or blocking instructions to appropriate executing components, ensuring data dependence is handled by executing dependent instructions sequentially, and optimizing cache usage to improve execution efficiency and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Power
If multiple stand-alone executing components are provided in special purpose processors, then the computational power for specific operations (e.g., matrix computing, convolution, filtering, ranking) is improved, but the device complexity and instruction management complexity increase
Solution Approach 1:
The patent implements a universal instruction queue that can manage both arithmetic instructions and blocking instructions, serving multiple executing components (matrix computing unit, convolution unit, filtering unit, ranking unit) through a single unified interface. This multi-functional queue reduces the need for separate instruction management mechanisms for each executing component, thereby reducing device complexity while maintaining high computational power across different operations.
2Reliability
If data dependence is handled by sequential execution of dependent instructions, then the reliability of instruction execution is improved, but the execution time increases
Solution Approach 1:
The patent introduces blocking instructions that are generated in advance when data dependence is detected between arithmetic instructions. These blocking instructions are inserted into the instruction queue before execution, serving as pre-prepared synchronization barriers. This preliminary action allows the system to maintain sequential execution for data-dependent instructions (ensuring reliability) while providing a structured mechanism that can be optimized by the executing components, potentially reducing the overall execution time compared to unoptimized sequential processing.
3Reliability
If blocking instructions are inserted to handle data dependence, then the correctness of instruction execution order is improved, but the instruction queue length and memory usage increase
Solution Approach 1:
The patent extracts the data dependence management function into a separate blocking instruction mechanism, which is then inserted into the unified instruction queue only where needed. This extraction allows the system to handle data dependence explicitly through dedicated blocking instructions rather than requiring the entire instruction queue to be managed sequentially. The blocking instructions are selectively inserted only when data dependence is detected, minimizing the impact on instruction queue size while ensuring execution correctness.
Data Source
AI summary
A method and apparatus for executing an instruction are provided. In the method, an instruction queue is first generated, and an instruction from the instruction queue in preset order is acquired. Then, a sending step including: determining a type of the acquired instruction; determining, in response to determining that the acquired instruction is an arithmetic instruction, an executing component for executing the arithmetic instruction from an executing component set; and sending the arithmetic instruction to the determined executing component is executed. Last, in response to determining that the acquired instruction is a blocking instruction, a next instruction is acquired after receiving a signal for instructing an instruction associated with the blocking instruction being completely executed.