NoC Topology Generation with Reinforcement Learning and Deadlock Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training machine learning models to generate network-on-chip (NoC) topologies is challenging due to complex concepts like connectivity, packet routing, and deadlock avoidance, which require mathematical certainty rather than statistical models, and resetting the NoC to resolve deadlocks is undesirable.
Innovation Solution
Employing reinforcement learning on a fully routed simplistic NoC topology to identify optimal transformations through multiple training sessions, using a machine learning model to apply transformations, compute costs, and update policies iteratively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a machine learning model is trained to generate NoC topology, then topology generation capability is improved, but the model cannot ensure mathematical certainty for deadlock avoidance
Solution Approach 1:
The topology generation process is segmented into two distinct parts: (1) a machine learning model that generates candidate topologies to improve automation, and (2) a formal verification module that provides mathematical certainty for deadlock avoidance. This segmentation allows each component to specialize in its strength while the system as a whole achieves both automation and reliability.
Solution Approach 2:
A formal verification module acts as an intermediary between the machine learning model and the final topology output. This intermediary verifies candidate topologies using mathematical methods to ensure deadlock freedom, bridging the gap between statistical generation and deterministic guarantees.
2Manufacturing precision
If multiple iterations of NoC topology are generated to satisfy criteria, then topology quality is improved, but training complexity and time increase
Solution Approach 1:
The system performs preliminary action by pre-training the machine learning model on synthetic NoC topology data before actual topology generation. This pre-training accelerates convergence during actual use, reducing the number of iterations needed to generate high-quality topologies while maintaining improvement in topology quality.
Solution Approach 2:
The system uses synthetic copies of NoC topologies for training the machine learning model. These synthetic examples allow the model to learn from numerous iterations without requiring equivalent time for actual hardware validation, separating training time from deployment time and enabling rapid iteration.
3Adaptability or versatility
If a statistical model is used for NoC design, then adaptability is improved, but mathematical certainty for critical concepts is lost
Solution Approach 1:
The design system is segmented into a statistical machine learning component that provides adaptability and flexibility in generating diverse topology candidates, and a formal verification component that provides mathematical certainty for critical properties like deadlock avoidance. Each segment handles the aspects where it excels.
Solution Approach 2:
Formal verification methods serve as an intermediary that takes the flexible, adaptive output from the statistical model and applies deterministic mathematical checks to ensure critical reliability requirements are met, combining the strengths of both statistical and formal methods.
Data Source
AI summary
A computer-implemented method includes loading a simplistic network-on-chip (NoC) topology that is fully routed, and performing reinforcement learning on the NoC topology to identify a sequence of topology transformations that will produce a more optimal NoC topology. Performing the reinforcement learning includes running a plurality of training sessions. Running each training session includes using a machine learning model to apply a set of transformations to the NoC topology according to a policy, computing a cost of the NoC topology after the set of transformations has been applied, and updating the policy in response to the cost.