A hotspot area layout automatic repair system and method based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's layout hotspot automatic repair system and method, belong to integrated circuit physical design field.The system includes hotspot extraction and state modeling module, action space construction module, reinforcement learning decision module, environmental feedback and reward signal module, strategy iteration and continuous optimization module.The application encodes local layout as state, and the repair action is output by intelligent agent, and the multidimensional reward function of manufacturability, performance, area and design rule is driven strategy optimization by fusing, realize the automatic elimination of hotspot and PPA collaborative optimization, improve design efficiency and chip yield.

Description

Technical Field

[0001] This invention belongs to the field of electronic design automation (EDA) and integrated circuit physical layout repair technology, and particularly relates to an automatic hotspot area layout repair system and method based on reinforcement learning. Background Technology

[0002] As semiconductor manufacturing processes advance to deep submicron and below technology nodes (such as 7nm, 5nm, and even more advanced processes), the geometric tolerance window in IC layouts is becoming increasingly narrow. During manufacturing processes such as photolithography and etching, certain specific pattern structures on the layout are highly susceptible to defects due to optical proximity effects, uneven chemical mechanical polishing, and other factors, leading to open circuits, short circuits, or performance drift. These defect-prone sensitive areas are high-risk areas, known in the industry as "hotspots." The presence of hotspots seriously threatens chip yield and reliability.

[0003] To ensure chip manufacturability, the industry commonly employs rule-based checking (DRC) and model-based hotspot detection tools in the post-physical design stage to identify these sensitive areas. However, existing technologies have significant bottlenecks: The repair process is highly dependent on manual labor: Current mainstream electronic design automation (EDA) tools mainly act as "detectors," identifying hotspot locations, but repair solutions still require design engineers to manually adjust the layout graphics based on experience (such as moving traces and adjusting spacing). This method is inefficient, and for complex designs containing thousands or even tens of thousands of hotspots, the time and cost of manually repairing each one is prohibitive.

[0004] Repair solutions are localized and suboptimal: Manual repairs typically aim to "eliminate current hotspots," employing conservative and formulaic local adjustments. These adjustments lack a global perspective and may inadvertently create new hotspots ("hotspot drift") or negatively impact key chip performance, power consumption, and area (collectively referred to as PPA).

[0005] Lack of closed-loop optimization and learning capabilities: Existing automated repair scripts or tools are mostly based on predefined fixed rules, lacking flexibility and unable to adapt to the differentiated requirements of different process nodes or design styles. The effectiveness of repair actions needs to be verified by rerunning time-consuming simulations, forming an open-loop process of "detection-manual repair-re-verification". The system itself does not have the ability to learn and evolve from historical successful and failed repair cases.

[0006] To address these challenges, recent research has attempted to apply machine learning to hotspot detection, such as using convolutional neural networks (CNNs) to classify image segments to identify potential hotspot patterns. However, these methods remain at the "recognition" level and have not yet formed a complete, autonomously optimizable decision-making loop with the "repair" action. How to construct an intelligent system capable of automatically making decisions, executing repair actions, and continuously improving itself based on multi-objective optimization has become a pressing technical problem in this field. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of the prior art and provide a reinforcement learning-based automatic hotspot region layout repair system and method. This invention encodes local layouts as states, with an agent outputting repair actions. By integrating a multi-dimensional reward function that considers manufacturability, performance, area, and design rules to drive strategy optimization, it achieves automatic hotspot elimination and PPA co-optimization, thereby improving design efficiency and chip yield.

[0008] This invention is implemented as follows: Firstly, it provides an automatic hotspot region map repair system based on reinforcement learning, comprising: The hotspot extraction and state modeling module is used to identify hotspot regions from the integrated circuit chip layout and encode them into state input vectors acceptable to the reinforcement learning agent. The action space construction module is used to define multiple layout repair operations as repair actions to construct the action space for reinforcement learning. The reinforcement learning decision-making module is used to train the agent to perform the optimal repair action on the identified hotspot areas using deep reinforcement learning algorithms. The environmental feedback and reward signal module is used to calculate the reward function for reinforcement learning and evaluate the effect of each repair action based on the reward function. The policy iteration and continuous optimization module is used to update and maintain the reinforcement learning policy network.

[0009] Preferably, the operation of the hotspot extraction and state modeling module in identifying hotspot areas from the integrated circuit chip layout includes: using the manufacturability design rule engine to identify layout areas that do not conform to manufacturing rules from the integrated circuit chip layout, extracting the corresponding center coordinates and surrounding layout information, forming local layout blocks, and using them as hotspot areas.

[0010] Preferably, the operation of the hotspot extraction and state modeling module in encoding the identified hotspot regions into state input vectors acceptable to the reinforcement learning agent includes: The hotspot region is encoded as a grayscale image or a multi-layer matrix. At least one feature is extracted from the wiring density, line width statistics, and metal orientation histogram to form a state vector, thus forming a state space. The state space is then used as the input to the agent.

[0011] Preferably, the layout repair operation in the action space construction module includes at least one of the following: increasing metal spacing, reducing line width, switching wiring direction, repositioning vias, and rerouting local wiring.

[0012] Preferably, the execution of the repair action is constrained by design rules.

[0013] Preferably, the reward function of the environmental feedback and reward signal module is: R = α·(1 - Hs) + β·ΔP + γ·ΔA + δ·(1 - DRCv) where Hs represents the hotspot score after repair, ΔP represents the performance change after repair, ΔA represents the change in hotspot area after repair, DRCv represents whether a new DRC error is generated, and α, β, γ, and δ represent weights, all of which are greater than 0.

[0014] Preferably, the optimization algorithm used in the reinforcement learning decision module is the near-end policy optimization algorithm.

[0015] Secondly, a method for automatic hotspot repair based on reinforcement learning is provided, implemented based on the system described in any one of claims 1-7, characterized in that the method includes: Identify hotspot regions in the integrated circuit chip layout and encode them into state input vectors acceptable to the reinforcement learning agent; Define multiple layout repair operations as repair actions to construct the action space for reinforcement learning; A deep reinforcement learning algorithm is used to train the agent to perform optimal repair actions on the identified hotspot areas; The reward function for reinforcement learning is calculated, and the effectiveness of each repair action is evaluated based on the reward function, thereby updating and maintaining the reinforcement learning policy network.

[0016] Thirdly, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that the program, when executed by a processor, implements the automatic hotspot repair method for the layout.

[0017] Fourthly, a layout repair device is provided, characterized in that it includes a processor and a memory, wherein the memory stores instructions, and the instructions, when executed by the processor, implement the automatic layout hotspot repair method.

[0018] Compared with the prior art, the technical solution provided by the present invention has the following significant advantages: This invention frees designers from tedious and repetitive manual hotspot repair work, achieving full-process automation and efficiency improvement, greatly shortening the layout delivery cycle and improving design efficiency.

[0019] This invention, through a carefully designed multidimensional reward function, guides the repair strategy to not only eliminate hotspots as the primary goal, but also actively consider the impact on performance, power consumption, and area. This results in a globally superior comprehensive repair solution that surpasses the limitations of traditional single-target repair.

[0020] This invention utilizes a reinforcement learning framework, enabling the system to learn from continuous interaction with its "environment" (i.e., layout design rules and PPA evaluator) and continuously optimize its repair strategy. Through training, the system can adapt to the specific requirements of different process nodes and diverse circuit design styles, demonstrating strong generalization ability.

[0021] The system design of this invention fully considers compatibility with existing industry standards. Its input and output interfaces can be connected to mainstream layout databases (GDSII, OpenAccess) and existing DFM engines and simulation modules. Attached Figure Description

[0022] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating the principle of the automatic hotspot area map repair system provided in this embodiment of the invention.

[0024] Figure 2 This is a schematic diagram of the reinforcement learning training process provided in an embodiment of the present invention, illustrating the interaction between the agent, environment, state, action, and reward in the training cycle.

[0025] Figure 3 This is a comparison image of the sub-regions of the layout before and after repair, provided in an embodiment of the present invention.

[0026] Terminology Explanation: Hotspot t (i.e., hotspot area): A high-risk manufacturing defect area on the map.

[0027] RL (Reinforcement Learning): Reinforcement learning.

[0028] DFM (Design for Manufacturability): Design for Manufacturability.

[0029] Layout Patch: A solution for patching layouts.

[0030] Reward Signal: The reward signal in reinforcement learning. Detailed Implementation

[0031] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0032] like Figure 1 As shown in the illustration, this invention provides an automatic hotspot region map repair system based on reinforcement learning. This system constructs an agent that interacts with the map design environment, simulating a design expert's decision-making process. The system comprises five functional modules, forming a complete closed-loop process of identification, repair, and optimization.

[0033] The hotspot extraction and state modeling module is responsible for obtaining complete IC layout data from GDS layout files or layout databases (such as OpenAccess) in EDA tools, locating and identifying hotspot regions, and encoding them into state input vectors acceptable to the reinforcement learning agent. Specifically, it includes: Hotspot extraction: Identify areas of the map that do not conform to the manufacturing rules using the existing DFM rule engine, and extract the center coordinates and surrounding layout information of the hotspots.

[0034] Local environment construction: Taking the hotspot coordinates as the center, extract the map sub-region containing the hotspot and its surrounding related geometry to form a local map (Patch), i.e., the hotspot region.

[0035] State representation: Encoding abstract local geometric information into state inputs acceptable to the reinforcement learning agent, used to construct the state space of reinforcement learning.

[0036] The state encoding method combines image encoding and feature vector encoding. Image encoding preserves the spatial topology and geometric distribution information of the local layout, while feature vector encoding extracts numerical statistical features, such as wiring density, linewidth distribution, and metal orientation histogram, thereby forming a multi-dimensional state description for the agent to make decisions and learn.

[0037] The image tensor after image encoding is fused with the numerical feature vector after feature vector encoding to form a comprehensive state description of the hotspot region, thus constructing the state space S ∈ ℝ for reinforcement learning. n This serves as the input to the intelligent agent. The fusion can include, but is not limited to, methods such as feature splicing, weighted combination, or multimodal fusion, to simultaneously retain information from local geometric and statistical features, thereby achieving a high-dimensional representation of hotspot regions on the map.

[0038] The action space construction module defines the set of layout repair operations (Action Space) that can be executed by the RL (Reinforcement Learning) system, and clarifies its granularity and constraints. Action design follows physical design rules to ensure operational feasibility. Typical actions include, but are not limited to: 1. Increase metal spacing (+ΔSpacing) 2. Decrease trace width (-ΔWidth, subject to DRC) 3. Change routing direction (horizontal / vertical switching) 4. Via relocation (preventing short circuits or bridging) 5. Local routing rerouting (automatically avoiding hotspot areas) Each action a i ∈ A corresponds to a specific geometric transformation function f: GDS → GDS', and an embedded constraint filter is used to ensure that existing design rules (such as DRC, LVS) are not violated, so that the action outputs a legal and manufacturable intermediate layout.

[0039] The reinforcement learning decision-making module is implemented using a deep reinforcement learning algorithm. The agent determines the decision based on the input current state s. t (Hotspot image + encoding), select an action a based on the current policy. t The environment returns a reward r after execution. t and new state s t ₊1. Figure 2 This is a schematic diagram of the reinforcement learning training process provided in an embodiment of the present invention, illustrating the interaction between the agent, environment, state, action, and reward in the training cycle. Specifically, it is as follows: Algorithm selection: Proximal Policy Optimization (PPO) is preferred because it performs well in terms of stability and sample efficiency, and is suitable for complex decision problems with continuous or discrete action spaces.

[0040] Network Architecture: The policy network in the reinforcement learning decision module preferably adopts an encoder-decoder structure to achieve multi-level feature extraction and action decision output for complex map states.

[0041] The encoder section receives and processes the state input from hotspot regions. This state input is a fused comprehensive state representation, containing joint features of a fused image tensor (used to describe the geometric and topological features of the local landscape) and numerical feature vectors (used to characterize statistical or physical properties), used to construct a high-dimensional state representation for reinforcement learning. The encoder can employ a convolutional neural network (CNN) structure to extract local spatial correlation features and multi-scale geometric features, thereby forming a high-dimensional state representation.

[0042] The decoder section is used to synthesize the high-level features output by the encoder, and can employ a multilayer perceptron (MLP) or other deep feedforward network structures. The decoder fuses the extracted local spatial correlation features and multi-scale geometric features, and outputs the policy result corresponding to the action space A.

[0043] Specifically, for the discrete action space, the decoder output is the probability distribution of each candidate repair action; for the continuous action space, the decoder output is the action parameters (such as the mean μ and variance σ), which are used for subsequent action sampling and execution.

[0044] Through the above structural design, the policy network can simultaneously understand the local geometric features and statistical properties of the graph under multimodal input conditions, realize the end-to-end mapping from complex graph state to repair decision action, and improve the accuracy and generalization ability of automatic repair strategy.

[0045] Training objective: Maximize the cumulative reward R = ∑ r t .

[0046] Decision-making process: During the reasoning (deployment) phase, the module receives state s. t Action a is obtained through network forward propagation. t It is then handed over to subsequent modules for execution.

[0047] The environmental feedback and reward signal module is responsible for evaluating the agent's actions. t The effect after that is to generate a quantified reward signal r. t This is used to guide the learning direction of the RL model. The reward function incorporates multi-dimensional optimization objectives: R = α·(1 - Hs) + β·ΔP + γ·ΔA + δ·(1 - DRCv) Where Hs represents the hotspot area score after repair, ΔP represents the performance change after repair, ΔA represents the change in hotspot area caused by repair, DRCv represents whether a new DRC design rule error is generated, and α, β, γ, and δ represent the weights set for each objective.

[0048] The policy iteration and continuous optimization module manages the training lifecycle of the policy network, specifically including: Training Mechanism: The system employs a combination of offline training and online fine-tuning. The initial training dataset can be derived from historical successful repair cases or generated through experimental design, showing Hotspot regions and their corresponding repaired regions. During operation, the system continuously stores sequences of (state, action, reward, new state) as experience samples in the replay buffer. Training using a GPU-based TensorFlow / PyTorch architecture is supported.

[0049] Policy update: Periodically use accumulated experience data to update the parameters of the policy network through the training process of algorithms such as PPO, so that it evolves in the direction of obtaining higher cumulative rewards.

[0050] Knowledge preservation and transfer: Supports saving strategy models for specific processes or design types. When dealing with similar tasks, pre-trained models can be loaded for rapid adaptation, enabling knowledge reuse.

[0051] Deployment method: It is integrated into the EDA tool interface as a plugin that allows for simultaneous inference and repair.

[0052] The above five modules constitute a complete autonomous optimization closed loop: identifying hotspot regions → constructing state vectors → reinforcement learning decision-making → executing actions to modify the landscape → multi-objective evaluation to generate rewards (or, DFM and PPA feedback) → using rewards to update the policy to optimize future decisions (i.e., policy optimization), see appendix. Figure 1 .

[0053] Figure 3 This diagram visually illustrates the layout changes before and after the system of this invention automatically repairs a typical manufacturing hotspot. In the left image, two adjacent metal lines belonging to different networks have a spacing that approaches or slightly violates the minimum design rule for the current process node, forming a high-risk short-circuit hotspot. This area is highly susceptible to bridging due to line proximity during photolithography, leading to circuit malfunction. Furthermore, a via connecting different layers is located close to the edge of the upper metal line, further reducing the effective insulation gap and exacerbating the process sensitivity and reliability risks in this area. From a layout perspective, the routing congestion in this area is high; the initial routing scheme sacrificed manufacturing margin in pursuit of routing completion. This structure may also introduce large parasitic coupling capacitance, potentially negatively impacting signal timing. Therefore, the left image exhibits at least issues such as excessively narrow linewidth, small metal spacing, and overlapping metal intersections. The right image demonstrates how the manufacturing hotspot is eliminated by increasing the spacing and changing the routing direction using the repair method of this invention. Furthermore, the entire modification process was ensured by the constraint filter in the action space construction module, without generating any new DRC violations.

[0054] This embodiment also provides a method for automatic hotspot repair based on the above system, the method including: Identify hotspot regions in the integrated circuit chip layout and encode them into state input vectors acceptable to the reinforcement learning agent; Define multiple layout repair operations as repair actions to construct the action space for reinforcement learning; A deep reinforcement learning algorithm is used to train the agent to perform optimal repair actions on the identified hotspot areas; The reward function for reinforcement learning is calculated, and the effectiveness of each repair action is evaluated based on the reward function, thereby updating and maintaining the reinforcement learning policy network.

[0055] This invention automates the entire process from hotspot identification to optimization and repair, significantly improving layout convergence efficiency. A multi-objective optimization mechanism is introduced during the repair process to ensure that hotspots are eliminated while simultaneously optimizing the chip's power performance (PPA). Furthermore, this invention aims to give the system self-learning and generalization capabilities, enabling it to adapt to different process design kits (PDKs) and circuit design styles.

[0056] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A hotspot region map automatic repair system based on reinforcement learning, characterized in that, include: The hotspot extraction and state modeling module is used to identify hotspot regions from the integrated circuit chip layout and encode them into state input vectors acceptable to the reinforcement learning agent. The action space construction module is used to define multiple layout repair operations as repair actions to construct the action space for reinforcement learning. The reinforcement learning decision-making module is used to train the agent to perform the optimal repair action on the identified hotspot areas using deep reinforcement learning algorithms. The environmental feedback and reward signal module is used to calculate the reward function for reinforcement learning and evaluate the effect of each repair action based on the reward function. The policy iteration and continuous optimization module is used to update and maintain the reinforcement learning policy network.

2. The system as described in claim 1, characterized in that, The hotspot extraction and state modeling module identifies hotspot areas in the integrated circuit chip layout by: using the manufacturability design rule engine to identify layout areas that do not conform to manufacturing rules in the integrated circuit chip layout, extracting the corresponding center coordinates and surrounding layout information, forming local layout blocks, and using them as hotspot areas.

3. The system as described in claim 1, characterized in that, The hotspot extraction and state modeling module performs the following operations to encode identified hotspot regions into acceptable state input vectors for the reinforcement learning agent: The hotspot region is encoded as a grayscale image or a multi-layer matrix. At least one feature is extracted from the wiring density, line width statistics, and metal orientation histogram to form a state vector, thus forming a state space. The state space is then used as the input to the agent.

4. The system as described in claim 1, characterized in that, The layout repair operations in the action space construction module include at least one of the following: increasing metal spacing, reducing line width, switching routing direction, repositioning vias, and rerouting local wiring.

5. The method as described in claim 3, characterized in that, The execution of the repair action is constrained by design rules.

6. The system as described in claim 1, characterized in that, The reward function of the environmental feedback and reward signal module is: R = α·(1 - Hs) + β·ΔP + γ·ΔA + δ·(1 - DRCv) Where Hs represents the hotspot region score after repair, ΔP represents the performance change, ΔA represents the area change, DRCv represents whether a new DRC error is generated, and α, β, γ, and δ represent the weights.

7. The system as described in claim 1, characterized in that, The optimization algorithm used in the reinforcement learning decision module is the near-end policy optimization algorithm.

8. A method for automatic hotspot repair based on reinforcement learning, implemented based on the system described in any one of claims 1-7, characterized in that, The method includes: Identify hotspot regions in the integrated circuit chip layout and encode them into state input vectors acceptable to the reinforcement learning agent; Define multiple layout repair operations as repair actions to construct the action space for reinforcement learning; A deep reinforcement learning algorithm is used to train the agent to perform optimal repair actions on the identified hotspot areas; The reward function for reinforcement learning is calculated, and the effectiveness of each repair action is evaluated based on the reward function, thereby updating and maintaining the reinforcement learning policy network.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in claim 8.

10. A layout repair device, characterized in that, It includes a processor and a memory, wherein the memory stores instructions that, when executed by the processor, implement the method as described in claim 8.

Images