Reinforcement Learning Circuit Routing Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for determining connections between terminals of integrated circuits, especially on smaller technology nodes like 10 nm or less, are limited by strict design rules and require manual design efforts that are time-consuming and inefficient, often relying on assumptions that all design rules can be expressed as linear constraints for integer programming models.
Innovation Solution
A machine learning-based method using reinforcement learning (RL) optimizes routing connections by training an agent to play a routing game within a routing environment, where the agent receives observations and rewards, allowing it to learn optimal routing actions independently of the number of nets or circuit size, thus enabling automated and efficient determination of connections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual design methods are used for routing connections, then design accuracy can be maintained, but design time and efficiency deteriorate significantly
Solution Approach 1:
The patent replaces manual mechanical design processes with an automated reinforcement learning system. The RL agent learns optimal routing strategies through simulated interactions with a routing environment, substituting human designers with an intelligent algorithm that can process and evaluate routing options automatically, thereby maintaining accuracy while dramatically reducing design time
Solution Approach 2:
The routing system performs self-optimization through the reinforcement learning mechanism. The agent independently learns routing policies by interacting with the environment, receiving rewards for successful routing, and automatically improving its performance without requiring continuous manual intervention or adjustment, enabling the system to serve itself in optimizing routing connections
2Reliability
If traditional integer programming models are used for routing, then design rules can be enforced, but complexity increases due to linear constraint assumptions
Solution Approach 1:
The patent changes the fundamental parameters and approach of routing optimization from deterministic linear programming to probabilistic reinforcement learning. Instead of enforcing design rules through linear constraints, the RL agent learns to comply with design rules through environmental feedback and reward mechanisms, transforming the problem from a constrained optimization task to a sequential decision-making task that better handles the complexity of modern routing problems
3Productivity
If automated routing methods are implemented, then productivity increases, but manufacturing precision may deteriorate due to algorithmic approximations
Solution Approach 1:
The patent implements a feedback mechanism where the reinforcement learning agent receives reward signals from the routing environment based on the quality and validity of its routing decisions. This feedback loop allows the agent to learn from successful and unsuccessful routing attempts, continuously improving its precision while maintaining high productivity through automated operation
Solution Approach 2:
The system performs preliminary learning and training in a simulated routing environment before actual routing is performed. The RL agent pre-learns optimal routing strategies and policies through extensive simulation, which prepares it to execute accurate routing decisions efficiently in real design scenarios, thereby achieving both high productivity and precision
Data Source
AI summary
The disclosure provides a general solution for determining connections between terminals of various types of circuits using machine learning (ML). A ML method that uses reinforcement learning (RL), such as deep RL, to determine and optimize routing of circuit connections using a game process is provided. In one example a method of determining routing connection includes: (1) receiving a circuit design having known terminal groups, (2) establishing terminal positions for the terminal groups in a routing environment, and (3) determining, by the RL agent, routes of nets between the known terminal groups employing a model that is independent of a number of the nets of the circuit. A method of creating a model for routing nets using RL, a method of employing a game for training a RL agent to determine routing connections, and a RL agent for routing connections of a circuit are also disclosed.