Neural Network Processor Asynchronous Task Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data processing techniques for neural network processing face latency issues due to data dependencies and the inability to overlap execution of jobs, particularly in fetching neural network processing data structures and resolving dependencies before starting new tasks.
Innovation Solution
The proposed solution involves a data processing system with a processor configured to perform neural network processing, allowing for early fetching of neural network processing data structures and the overlap of job execution even if dependencies have not been resolved. This is achieved through asynchronous processing tasks with dependency flags that indicate whether execution can proceed with unresolved dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data dependencies are resolved before starting new tasks, then processing reliability is improved, but processing time increases due to stalls and inability to overlap execution
Solution Approach 1:
The patent applies preliminary action by allowing the neural engine to fetch metadata and program data structures in advance before all dependencies are resolved. The system can start executing tasks once critical path dependencies are met, while non-critical dependencies continue to resolve in the background, eliminating execution stalls and enabling overlapping of independent tasks.
Solution Approach 2:
The patent segments dependency resolution into critical and non-critical paths. Critical dependencies must be resolved before task execution begins, while non-critical dependencies can resolve asynchronously in the background. This segmentation allows the system to proceed with task execution without waiting for all dependencies to resolve, reducing overall processing time while maintaining reliability through proper dependency tracking.
2Ease of manufacture
If metadata and program data structures are fetched before execution, then task preparation is improved, but latency increases due to blocking on dependency resolution
Solution Approach 1:
The system performs preliminary fetching of metadata and program data structures before task execution begins. The neural engine can retrieve and prepare necessary data structures in advance, then start executing tasks once critical dependencies are satisfied, rather than blocking and waiting for all dependencies to resolve sequentially.
Solution Approach 2:
The patent introduces dynamic dependency resolution where the system can adaptively determine when to start task execution based on critical path dependency status. The system dynamically transitions from a blocking dependency model to a non-blocking model where execution can proceed once critical dependencies are met, while non-critical dependencies resolve asynchronously in the background.
3Productivity
If job execution is overlapped without resolving dependencies, then productivity is improved, but data dependency conflicts may occur
Solution Approach 1:
The patent segments dependencies into critical and non-critical categories. Critical dependencies are resolved before task execution begins, ensuring data consistency for tasks that require them. Non-critical dependencies continue to resolve in the background without blocking execution, enabling overlapping of independent tasks and improving throughput while maintaining data consistency where required.
Solution Approach 2:
The system implements feedback mechanisms through dependency tracking that monitor when dependencies are resolved and provide signals to the neural engine. This feedback allows the system to dynamically adjust task execution timing, ensuring that tasks wait for critical dependencies while allowing non-critical tasks to execute in parallel, thus maintaining both productivity and data consistency.
Data Source
AI summary
A data processing system, the data processing system comprising a processor that is configured to perform neural network processing, the processor comprising: at least one execution unit configured to perform processing operations for neural network processing; and a control circuit configured to distribute processing tasks to the at least one execution unit to cause the at least one execution unit to perform processing operations for neural network processing in response to a set of indications of neural network processing to be performed provided to the control circuit; wherein the processing tasks are asynchronous and comprise a dependency on at least one other processing task, the set of indications of neural network processing to be performed comprising an indication flag to indicate whether the execution unit can be caused to operate with a dependency on at least one other asynchronous processing task being unresolved.