MLA Compiler Routing Conflict Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning networks require significant computational resources and data handling, often necessitating cloud-based implementations, which can lead to latency and inefficiencies, especially in edge devices that require real-time processing and have limited resources.
Innovation Solution
A compiler generates statically scheduled computer programs for machine learning accelerators (MLAs) that implement machine learning networks, using default routing rules to manage data transfers and resolve routing conflicts, allowing for efficient execution on edge devices with limited resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If machine learning networks are implemented in cloud-based compute facilities, then computational resources and data handling capacity are improved, but latency and processing efficiency deteriorate due to remote data transmission
Solution Approach 1:
The patent segments the machine learning network into multiple processing stages distributed across different processing elements within the edge device. This allows data processing to occur locally in stages rather than transmitting all data to a remote cloud facility, reducing latency while maintaining computational capability through distributed processing architecture.
Solution Approach 2:
The patent introduces an on-chip memory system as an intermediary between sensors and processing elements within the edge device. This intermediary enables efficient local data buffering and processing without requiring continuous remote communication, thereby reducing transmission latency while supporting substantial computational workloads.
2Productivity
If machine learning networks are embedded on edge devices, then latency and processing efficiency are improved, but device complexity increases
Solution Approach 1:
The patent employs a mesh network topology where processing elements can serve multiple functions: they act as compute units for machine learning operations, as memory when selected by the memory interface, and as routing nodes for data transfer. This multi-functionality reduces overall device complexity by eliminating the need for separate dedicated components for each function.
Solution Approach 2:
The patent implements dynamic memory allocation where the memory interface dynamically selects which processing element serves as memory based on current computational needs. This dynamic reconfiguration allows the same hardware resources to adapt to different computational requirements, reducing the need for fixed, complex memory architectures.
3Adaptability or versatility
If data transfers are routed through intermediate memories in an MLA, then routing flexibility is improved, but routing conflicts increase due to shared resource contention
Solution Approach 1:
The patent performs preliminary routing analysis during the compilation phase to identify and resolve potential routing conflicts before execution. The compiler generates conflict-free data transfer schedules and routing paths, preventing routing conflicts from occurring during runtime rather than merely managing them when they arise.
Solution Approach 2:
The patent implements continuous data flow through the mesh network where data transfers are scheduled to proceed without interruption or conflict. By carefully orchestrating data movement through intermediate memories and processing elements, the system maintains continuous useful action without routing conflicts that would cause idle time or data loss.
Data Source
AI summary
A compiler receives a description of a machine learning network (MLN) and generates a computer program that implements the MLN on a machine learning accelerator (MLA). To implement the MLN, the compiler generates compute instructions that implement computations of the MLN on different processing units (Tiles), and data transfer instructions that transfer data used in the computations. The compiler may statically schedule at least a portion of the instructions for execution by the Tiles according to fixed timing. The compiler may initially implement data transfers between non-adjacent Tiles (or external memories) by implementing a sequence of transfers through one or more intermediate Tiles (or external memories) in accordance with a set of default routing rules that dictates the data path. The computer program may then be simulated to identify routing conflicts. When routing conflicts are detected, the compiler updates the computer program in a manner that avoids the conflicts.