Simulation ic netlist layout collaborative optimization method and system based on reinforcement learning algorithm

Abstract

The application discloses a simulation IC netlist layout cooperative optimization method and system based on a reinforcement learning algorithm, and the method comprises the following steps: generating an initial netlist of a simulation IC and inputting the initial netlist into a simulation platform; generating a simulation IC circuit layout according to the initial netlist of the simulation IC and simulating through the simulation platform; building a graph neural network based on a pre-constructed simulation IC netlist multi-edge heterogeneous graph, and an intelligent agent of reinforcement learning performs an optimization task to generate an optimized netlist of the simulation IC and sends the optimized netlist to the simulation platform; generating a new simulation IC circuit layout according to the optimized netlist of the simulation IC, simulating the optimized netlist of the simulation IC and the new simulation IC circuit layout through the simulation platform, and further optimizing the simulation IC circuit layout through the intelligent agent of reinforcement learning; and continuously interacting and optimizing the intelligent agent of reinforcement learning and the simulation platform until a final parameter combination meeting a performance index is obtained. The application greatly improves the efficiency and performance of circuit optimization.

Description

Technical Field

[0001] This invention relates to the field of EDA (electronic design automation), and in particular to a method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithms. Background Technology

[0002] With the rapid development of integrated circuit technology, analog ICs (integrated circuits) are increasingly playing a central role in fields such as communications, medical devices, automotive electronics, and consumer electronics. The performance of analog ICs often directly determines the quality and reliability of the entire system; therefore, their design process demands extremely high precision and efficiency in parameter optimization. In traditional analog IC design flows, circuit size generation and optimization primarily rely on the experience of design engineers or heuristic algorithms. However, experience-driven optimization methods not only require designers to possess extensive professional knowledge and practical experience but are also easily influenced by human factors, exhibiting drawbacks such as strong subjectivity, low repeatability, and difficulty in large-scale scalability. While heuristic algorithms can achieve a certain degree of parameter optimization, their essence lies in the pre-defined rules based on local search, lacking a global perspective and having limited ability to handle complex multi-objective optimization tasks. At deep submicron process nodes, the nonlinear effects and parasitic parameters of devices further increase design difficulty, causing traditional methods to face significant bottlenecks in optimization efficiency and effectiveness, making it difficult to meet the demands of modern analog ICs for high-performance, high-precision design.

[0003] Meanwhile, with the development of machine learning and artificial intelligence technologies, a large number of data-driven circuit design and optimization methods have emerged. However, these methods require a large amount of high-quality labeled data as a training set. The strong correlation and complexity between analog IC layouts and netlists make manual labeling an extremely time-consuming and error-prone process, not only inefficient but also potentially introducing human biases that affect model training performance. Especially in place-and-route collaborative design, the labeling of device coordinates, dimensions, and related information often involves multiple levels and data types, making it difficult to meet actual production needs by relying solely on manual tasks. Therefore, how to effectively combine intelligent algorithms in circuit optimization while developing efficient layout and netlist data labeling methods to achieve high-performance parameter optimization and rapid generation of large-scale datasets has become a key problem that urgently needs to be solved in the field of analog IC design. Summary of the Invention

[0004] The main objective of this invention is to provide a method and system for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithms, which improves the efficiency and performance of circuit optimization, while reducing reliance on manual intervention and significantly shortening the design cycle.

[0005] The technical solution adopted in this invention is:

[0006] A method for collaborative optimization of simulated IC netlist layout based on reinforcement learning algorithm is provided, characterized by the following steps:

[0007] Select a specified type of IC circuit and use an open-source schematic editor to generate and export the initial netlist of the analog IC;

[0008] The initial netlist of the analog IC is input into the pre-built simulation platform, and the simulation results of the netlist are input into the pre-built reinforcement learning agent. The reward function of the reinforcement learning agent is associated with the performance index of the analog IC.

[0009] The analog IC circuit layout is generated based on the initial netlist of the analog IC, and then simulated through a simulation platform. The simulation results of the layout are then input into the reinforcement learning agent.

[0010] The reinforcement learning agent builds a graph neural network based on a pre-constructed analog IC netlist heterogeneous graph to perform optimization tasks, generate an optimized analog IC netlist, and send it to the simulation platform. The analog IC netlist heterogeneous graph construction process is as follows: based on the initial analog IC netlist, different types of nodes in the heterogeneous graph are defined, and corresponding node features are extracted according to different node types to generate the nodes, node feature matrix, and edge type matrix of the analog IC netlist heterogeneous graph. Finally, an analog IC netlist heterogeneous graph based on signal flow recognition is generated.

[0011] A new analog IC circuit layout is generated based on the optimized analog IC netlist. The simulation platform simulates both the optimized analog IC netlist and the new analog IC circuit layout, and then outputs the generated simulation results to the reinforcement learning agent to further optimize the analog IC circuit layout.

[0012] The reinforcement learning agent continuously interacts and optimizes with the simulation platform until the final parameter combination that meets the performance indicators is obtained, and the optimized circuit parameters and corresponding netlist files are output for subsequent design and manufacturing.

[0013] Following the above technical solution, the optimization objective is determined based on the performance indicators, and combined with the constraints, the two are transformed into corresponding parameters and associated with the reward function of the reinforcement learning agent. The performance indicators include gain, bandwidth, phase margin, and common-mode rejection ratio, and the constraints include specific power consumption, area, and size.

[0014] Following the above technical solution, when building the simulation platform, the simulation environment settings are also performed, including simulation mode, simulation temperature, simulation parameters, and units of simulation output items.

[0015] Following the above technical solution, the analog IC circuit layout is specifically generated based on heuristic algorithms, including the simulated annealing placement algorithm and the A* routing algorithm. In the placement stage, the placement is optimized using the simulated annealing algorithm and the placement tree. In the routing stage, path search is performed based on the A* algorithm. By gradually exploring paths and selecting the path with the lowest cost, the connection scheme between devices is finally generated.

[0016] Following the above technical solution, the A* routing algorithm also dynamically adjusts the strategy according to the complexity of the routing to avoid path intersections and blockages, and optimize the signal transmission quality and layout density of the entire layout.

[0017] Following the above technical solution, the reinforcement learning agent specifically interacts repeatedly with the simulation platform, utilizes the Actor network to output the circuit parameter policy distribution, utilizes the Critic network to evaluate the policy quality, and combines random exploration and goal-oriented optimization strategies to continuously adjust the network weights and gradually converge to the optimal parameters.

[0018] Following the above technical solution, the node types of the heterogeneous graph include m nodes representing MOSFET transistors, c nodes representing capacitors, and v nodes representing voltages. Different parameters are extracted as node features for different types of nodes.

[0019] Following the above technical solution, the transistor size in the analog IC circuit layout is specifically optimized.

[0020] This invention also provides a simulated IC netlist layout collaborative optimization system based on reinforcement learning algorithm, comprising:

[0021] An open-source schematic editor for generating and exporting netlists of analog ICs based on the selected type of IC circuit.

[0022] Analog IC circuit layout generator, used to generate analog IC circuit layouts based on the netlist of analog ICs;

[0023] The analog IC netlist heterogeneous graph generator is used to define different types of nodes in the heterogeneous graph based on the netlist of the analog IC, extract the corresponding node features according to different node types, generate the node, node feature matrix and edge type matrix of the analog IC netlist heterogeneous graph, and finally generate the analog IC netlist heterogeneous graph based on signal flow recognition.

[0024] The simulator is used to pre-build a simulation platform and perform simulation verification. Specifically, it performs simulation verification of the netlist of the analog IC, generates netlist simulation results, and performs simulation verification of the circuit layout of the analog IC, generates layout simulation results. The simulator inputs all simulation results into the optimization module.

[0025] The optimization module is used to pre-build a reinforcement learning agent whose reward function is associated with the performance indicators of the analog IC. Based on the obtained netlist simulation results and layout simulation results, the reinforcement learning agent builds a graph neural network based on the pre-built heterogeneous graph of the analog IC netlist to perform optimization tasks, generating optimized analog IC parameters and corresponding netlist files. The reinforcement learning agent continuously interacts with the simulation platform to optimize until the final parameter combination that meets the performance indicators is obtained, and outputs the optimized circuit parameters and corresponding netlist files for subsequent design and manufacturing.

[0026] Following the above technical solution, the analog IC circuit layout generator specifically generates analog IC circuit layouts based on heuristic algorithms, including the simulated annealing placement algorithm and the A* routing algorithm. In the placement stage, the placement is optimized using the simulated annealing algorithm and the placement tree; in the routing stage, path search is performed based on the A* algorithm. By gradually exploring paths and selecting the path with the lowest cost, the interconnection scheme between devices is finally generated.

[0027] The present invention also provides a computer storage medium storing a computer program executable by a processor, the computer program executing the analog IC netlist layout collaborative optimization method based on reinforcement learning algorithm described in the above technical solution.

[0028] The beneficial effects of this invention are as follows: By introducing a reinforcement learning agent-based approach to optimize the generated analog IC circuit layout, this invention does not rely on human experience or preset rules, and can achieve efficient layout optimization in complex optimization scenarios with multiple objectives and constraints. Furthermore, through two simulations of the netlist and the layout, the simulation results serve as a component of the feedback obtained by the reinforcement learning agent. The simulation platform and the reinforcement learning agent dynamically interact, continuously optimizing the layout and narrowing the difference between the two simulation results. This collaborative optimization improves the integration and performance of the circuit design, ensuring that the circuit layout meets process constraints while effectively reducing wiring complexity and power consumption, and minimizing the impact of parasitic effects on chip performance.

[0029] Furthermore, by constructing heterogeneous graphs through multi-terminal device nodes and signal flow identification, the circuit topology characteristics and signal transmission rules are accurately captured, effectively improving the optimization algorithm's ability to predict circuit performance and its convergence speed. The reinforcement learning agent, through dynamic interaction with the simulation platform, continuously updates and optimizes the weights of the policy network and the evaluation network, and can automatically learn the optimal design parameters applicable to different circuit topologies and performance requirements.

[0030] Furthermore, this invention also offers significant advantages in circuit layout generation. The reinforcement learning-based placement and routing optimization strategy not only automatically optimizes circuit dimensions but also generates circuit layouts that match these dimensions. In the placement phase, simulated annealing is used to optimize device positions, ensuring minimum spacing and a reasonable layout between devices. In the routing phase, the A* algorithm is used to optimize wiring paths, ensuring efficient signal transmission and reducing signal interference. Compared to traditional heuristic methods, this invention possesses stronger global search and generalization capabilities, significantly improving the efficiency and performance of circuit optimization while reducing reliance on manual intervention and significantly shortening the design cycle. It provides an efficient and intelligent optimization solution for analog IC circuit design.

[0031] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1A This is a flowchart of the simulated IC netlist layout collaborative optimization method based on reinforcement learning algorithm according to an embodiment of the present invention;

[0034] Figure 1B This is a schematic diagram of the framework of the analog IC netlist layout collaborative optimization method based on reinforcement learning algorithm according to an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram illustrating the construction of a multi-sided heterogeneous graph neural network simulating an IC netlist according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the layout generated by the layout generator based on the heuristic algorithm in an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram illustrating the interaction between the reinforcement learning agent and the environment in an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0039] It should be noted that the illustrations provided in the embodiments of the present invention are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0040] In this invention, it should also be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used only for descriptive and distinguishing purposes and should not be construed as indicating or implying relative importance.

[0041] Furthermore, it should be noted that the features of the various embodiments of the present invention can be combined or integrated in whole or in part, and as those skilled in the art will understand, they can interact and operate in different ways. Each embodiment can be implemented independently of each other or in association with one another.

[0042] like Figure 1A As shown, the simulated IC netlist layout collaborative optimization method based on reinforcement learning algorithm in this embodiment of the invention includes the following steps:

[0043] S1. Select the specified type of IC circuit, use the open-source schematic editor to generate and export the initial netlist of the analog IC;

[0044] S2. Input the initial netlist of the analog IC into the pre-built simulation platform, and input the netlist simulation results into the pre-built reinforcement learning agent. The reward function of the reinforcement learning agent is associated with the performance index of the analog IC.

[0045] S3. Generate the analog IC circuit layout based on the initial netlist of the analog IC, and simulate it through the simulation platform. Input the simulation results of the layout into the reinforcement learning agent.

[0046] S4. The reinforcement learning agent builds a graph neural network based on a pre-constructed analog IC netlist heterogeneous graph to perform optimization tasks, generate an optimized analog IC netlist, and send it to the simulation platform. The analog IC netlist heterogeneous graph construction process is as follows: define different types of nodes in the heterogeneous graph according to the initial analog IC netlist, extract corresponding node features according to different node types, generate the nodes, node feature matrix, and edge type matrix of the analog IC netlist heterogeneous graph, and finally generate an analog IC netlist heterogeneous graph based on signal flow recognition.

[0047] S5. Generate a new analog IC circuit layout based on the optimized analog IC netlist. The simulation platform simulates both the optimized analog IC netlist and the new analog IC circuit layout, and outputs the generated simulation results to the reinforcement learning agent to further optimize the analog IC circuit layout.

[0048] S6. The reinforcement learning agent continuously interacts and optimizes with the simulation platform until the final parameter combination that meets the performance indicators is obtained, and outputs the optimized circuit parameters and corresponding netlist files for subsequent design and manufacturing.

[0049] As can be seen, this embodiment of the present invention optimizes the generated analog IC circuit layout by introducing a reinforcement learning agent, without relying on human experience and preset rules. It can achieve efficient layout optimization in complex optimization scenarios with multiple objectives and constraints. Furthermore, through two simulations of the netlist and the layout, the simulation results serve as a component of the feedback obtained by the reinforcement learning agent. The simulation platform and the reinforcement learning agent interact dynamically, continuously optimizing the layout and narrowing the difference between the two simulation results. This collaborative optimization improves the integration and performance of the circuit design, ensuring that the circuit layout meets process constraints while effectively reducing wiring complexity and power consumption, and minimizing the impact of parasitic effects on chip performance.

[0050] When setting up a simulation platform and environment for an analog IC netlist, it is necessary to select a process design kit (PDK) for a specific design process and a specific circuit type. An initial netlist is generated and exported using the open-source schematic editor xschem. For example, in this use case, the PDK is a commercial 130nm PDK, and the circuit type is an OTA circuit. Each device in the netlist needs to be selected from the process design library according to the circuit design requirements. For example, in this case, devices include PMOS / NMOS types, and each device is selected from the PDK process library. Next, based on the device selection and circuit design requirements, a customized simulation platform is built for each analog IC netlist. In this use case, the gate voltage rating of the selected NMOS / PMOS devices is 5V, and the drain voltage rating is 10.5V, suitable for high-voltage circuit design. Therefore, the highest bit voltage VDD in the simulation environment is set to 5V. Simultaneously, for the OTA circuit, four key performance indicators are designed: gain, CMRR (common-mode rejection ratio), bandwidth, and PM (phase margin). In the application case of this invention, an external circuit for simulation is built, and a load capacitor is connected. Through the above steps, the simulation process of each generated circuit parameter solution with the above four key indicators is fully realized during the optimization process, that is, the simulation platform construction and simulation environment settings of the analog IC netlist are completed.

[0051] The analog IC netlist-based heterogeneous graph generation based on signal flow identification mainly combines the analog IC netlist to define different types of nodes for the heterogeneous graph. In the selected case, the node types include m (MOSFET transistor), c (capacitor), and v (voltage). Different parameters are extracted as node features for different types of nodes. Further explanation based on the above rules: In multi-terminal devices, where transistors have four ports and capacitors have two ports, each port is defined as a node, and the node type is determined by the device type. In this case, a total of 13 node features were extracted, which can form a node observation matrix of the number of nodes * 13. The features extracted from m-type nodes include: id (drain current) / gm (transconductance) / vth (threshold voltage) / vdsat (saturation voltage) / vds (drain-source voltage) / vgs (gate-source voltage) / gds (drain conductance). The features extracted from c-type nodes include: capacity (capacitance value) / w (capacitor width) / l (capacitor length) / m (number of capacitors in parallel). The features extracted from v-type nodes include the voltage and current values ​​at that point. Further integration of the above information completes the establishment of the nodes and node feature matrix of the heterogeneous diagram of the analog IC netlist.

[0052] The generation of heterogeneous graphs of analog IC netlists based on signal flow identification mainly builds upon the previously established nodes and node feature matrices of the heterogeneous graph of the analog IC netlist by further constructing the edges and edge feature matrices of the heterogeneous graph. First, based on the analog IC netlist, nodes with connections are recorded. For example, in the case selected in this invention, node 0 is the drain of transistor MM0, and node 6 is the source of transistor MM1. The netlist shows a connection between nodes 0 and 6, and the signal path connecting the two nodes is net6. Therefore, this edge needs to be recorded as [0,6],[6,0]. From this rule, the connections between all nodes can be written out. In the OTA circuit selected in this invention, there are a total of 33 nodes and 48 edges. Further integrating the above information completes the edge set construction of the heterogeneous graph of the analog IC netlist. Combining simulation results, signal path analysis is performed on each edge sequentially to determine whether the path carries DC signals or AC information in the AC simulation environment. The two results are numbered separately to establish an edge type matrix. In the selected case of this invention, the 48 edges are judged in sequence, the DC signal path is marked as 1 and the AC signal route is marked as 0, and an edge type matrix containing 48 0 / 1 is established.

[0053] When building a heuristic algorithm-based analog IC circuit layout generator, as in the case used in this invention, an OTA (Over-The-Air) commercial 130nm PDK is employed. This invention uses a heuristic algorithm to generate analog IC circuit layouts, specifically including a simulated annealing placement algorithm and an A* routing algorithm. First, in the placement phase, the inputs include a circuit netlist, device type and size constraints, design rules (such as minimum spacing, power requirements, etc.), and process parameters provided in the process design kit (PDK) (such as device size, gate oxide thickness, etc.). Based on these inputs, placement optimization is performed using the simulated annealing algorithm and a B-tree (placement tree). A B-tree is a bottom-up tree-like data structure used to manage the location and placement relationships of devices. In the placement phase, each device is a node in the B-tree, where node characteristics include device type (such as NMOS, PMOS, capacitor, etc.), device size, and location constraints (such as minimum spacing). The simulated annealing algorithm gradually finds the globally optimal placement scheme by randomly adjusting the device node positions and calculating a cost function. The cost function includes factors such as minimum spacing between devices, power and ground distribution, and circuit layout compactness. The output of the placement phase is the final location of the devices on the chip, ensuring compliance with all design rules. Next, in the routing phase, the inputs are the optimized device placement, connection information between nodes (power and signal paths), and design rule constraints (such as routing width, number of routing layers, path congestion, etc.). The routing algorithm uses the A* algorithm for path search, with the cost function of each path comprehensively considering factors such as connection length, routing density, signal interference, and minimum distance between devices. In the A* algorithm, the start and end points are the device ports to be connected, and the heuristic function calculates the expected cost from the current node to the target node. The A* algorithm explores paths step-by-step and selects the path with the lowest cost, ultimately generating the interconnection scheme between devices. The output of the routing phase is the final interconnection path, ensuring that each port is correctly connected and the path is optimal, while also meeting process rule requirements (such as minimum width, minimum spacing, shortest path, etc.). Furthermore, the A* algorithm dynamically adjusts its strategy based on the routing complexity to avoid path intersections and blockages, optimizing the signal transmission quality and placement density of the entire layout. By combining simulated annealing layout and A* routing algorithm, this invention can automatically generate optimal IC layouts while meeting electrical performance requirements (such as gain, bandwidth, common-mode rejection ratio, and phase margin) and ensuring the rationality of physical layout and process compatibility. After implementing the simulated IC circuit layout generation part, it is linked with the simulated IC netlist simulation platform and simulation environment settings in the previous step to ensure that each generated layout can be correctly simulated with Spice. The simulation results will be used as part of the feedback obtained by the reinforcement learning agent.

[0054] When performing automatic optimization of analog IC netlist circuit parameters, firstly, the optimization objectives (such as gain, bandwidth, phase margin, and common-mode rejection ratio) and constraints (such as power consumption, area, and sizing legalization range) are determined based on the performance requirements of the analog IC circuit. These objectives and constraints are parameterized, and the reward function of the reinforcement learning agent is set. The reward function mainly consists of three parts: netlist simulation results, layout simulation results, and the difference between the simulations before and after. Next, a circuit simulation environment based on a real SPICE simulation platform is built. By coupling the simulation platform with the reinforcement learning framework, the agent can interact with the simulation environment. Two circuit simulations are performed between every two interactions, and circuit performance feedback is obtained in the form of the reward function. Then, based on SAC (Software-Assisted Acoustics), the optimization is performed on the netlist simulation results. The Actor-Critic algorithm constructs a reinforcement learning agent. Through repeated interaction with the simulation platform, the agent utilizes the Actor network to output the circuit parameter policy distribution, while the Critic network evaluates the policy quality. Combining random exploration and goal-oriented optimization, the network weights are continuously adjusted until convergence to the optimal parameters. Simultaneously, a heterogeneous graph neural network for multi-terminal device nodes and signal flow recognition is used for circuit topology optimization modeling. This network, through its structure containing different types of nodes (e.g., transistor nodes, power supply nodes) and edges (e.g., DC signal edges, AC signal edges), effectively captures the circuit's topological characteristics and signal transmission patterns, extracting the implicit relationships between circuit parameters and performance to further guide the agent's policy optimization. Finally, a new circuit netlist is generated based on the agent's output parameter policy and verified through simulation. The verification results are fed back to the reinforcement learning model for continuous optimization until the final parameter combination that meets performance requirements is obtained. The optimized circuit parameters and corresponding netlist file are then output for subsequent design and manufacturing.

[0055] For the reward function:

[0056] reward = α * performance_score post-target +β*gap_score post-pre

[0057]

[0058] α and β are weights. The Reward must ensure that the layout performance metrics meet the standards and that the difference from the previous simulation is as small as possible. (performance_score) post-target This is a performance metric, and obviously, the higher the better. gap_score post-per This term represents the difference between the previous and subsequent simulations. Ideally, it should be as close to 0 as possible. However, the actual performance of the subsequent simulation is unlikely to exceed that of the previous simulation, meaning this term is likely to be negative. Therefore, we want it to be as large as possible, and the reward for this optimization should be as large as possible.

[0059] The reason this invention chooses SAC (Soft Actor-Critic reinforcement learning algorithm) for transistor parameter optimization is primarily to balance stability and efficiency. SAC, as a reinforcement learning algorithm, possesses advantages in both stability and efficiency. Compared to traditional algorithms, SAC introduces an entropy regularization term, allowing the policy to maintain a certain degree of randomness during the optimization process, avoiding premature convergence to local optima and thus ensuring the breadth of the exploration process. This is particularly important for transistor size parameter optimization involved in OTA circuit optimization, as circuit performance is affected by various factors, and prematurely getting trapped in local optima may cause other potential optimal solutions to be missed. Through continuous exploration and optimization, SAC can ensure that the globally optimal parameter configuration is found. SAC exhibits superior convergence, demonstrating excellent convergence when facing complex high-dimensional continuous action spaces. It controls the entropy of the policy through a temperature parameter, balancing exploration and convergence speed, thereby enabling the algorithm to converge to the optimal solution more smoothly. In transistor size optimization for over-the-air (OTA) circuits, the complexity and nonlinearity of the parameter space require algorithms to converge stably and quickly. SAC, with its superior convergence performance, effectively avoids slow convergence or unstable training, ensuring the optimal solution is found within a limited number of training steps. SAC integrates well with graph neural networks (GNNs) and is highly compatible with handling analog circuit problems. By incorporating GNNs, SAC enables more efficient modeling of the relationships between components in the circuit during optimization. Transistors and other components in an OTA circuit can be viewed as a graph structure. GNNs can extract the complex interactions between components through multiple convolutional layers, providing more accurate feedback for the optimization process. This allows SAC to more efficiently adjust transistor size parameters, thereby optimizing circuit performance. Furthermore, SAC is a powerful algorithm for handling continuous action space problems. In OTA transistor optimization, transistor dimensions (such as width W and length L) are continuous, which traditional reinforcement learning algorithms may struggle with. SAC (Search Engine Accelerator) can finely adjust parameters in a continuous space, ensuring that every action in the optimization process effectively improves circuit performance, which is particularly crucial for transistor size adjustment in OTA (Over-The-Air) circuits. SAC also boasts excellent multi-objective optimization capabilities. In OTA circuit optimization, not only gain needs to be considered, but also multiple performance metrics such as bandwidth, PSRR (Power Supply Rejection Ratio), and phase margin need to be optimized simultaneously. SAC excels in multi-objective optimization by adjusting the weights of different objectives to achieve a performance balance. This allows SAC to ensure that multiple objectives are reasonably optimized when optimizing transistor size, rather than being limited to a single objective. SAC avoids local optima and overfitting; its entropy regularization mechanism effectively prevents the risk of prematurely falling into local optima and overfitting.Optimizing OTA circuits requires meticulous adjustments across a large parameter space. If the algorithm is overly biased towards a fixed solution, it may lead to poor optimization results. SAC's policy update method ensures that the algorithm maintains sufficient exploratory activity during training, avoiding this situation and thus improving the robustness of the optimization.

[0060] This invention introduces a reinforcement learning-based circuit size optimization method, utilizing an innovative design combining the SAC (Soft Actor-Critic) algorithm and heterogeneous graph neural networks. It does not rely on human experience or preset rules, enabling efficient circuit parameter search and optimization in complex optimization scenarios with multiple objectives and constraints. By constructing a heterogeneous graph through multi-terminal device node and signal flow identification, it accurately captures circuit topology characteristics and signal transmission patterns, effectively improving the optimization algorithm's predictive ability and convergence speed for circuit performance. The reinforcement learning agent, through dynamic interaction with the simulation platform, continuously updates and optimizes the weights of the policy network and evaluation network, automatically learning the optimal design parameters suitable for different circuit topologies and performance requirements. Compared with traditional optimization methods, this invention also has significant advantages in circuit layout generation. The reinforcement learning-based placement and routing optimization strategy not only automatically optimizes circuit size but also generates circuit layouts that match the size optimization. In the placement stage, simulated annealing is used to optimize device positions, ensuring minimum spacing and reasonable layout between devices; in the routing stage, the A* algorithm is used to optimize connection paths, ensuring efficient signal transmission and reducing signal interference. By employing collaborative optimization, this invention improves the integration and performance of circuit design, ensuring that the circuit layout meets process constraints while effectively reducing routing complexity and power consumption, and minimizing the impact of parasitic interactions on chip performance. Compared to traditional heuristic methods, this invention possesses stronger global search and generalization capabilities, significantly improving the efficiency and performance of circuit optimization while reducing reliance on manual intervention and significantly shortening the design cycle. It provides an efficient and intelligent optimization solution for analog IC circuit design.

[0061] Transistor size has a significant impact on circuit performance. The dimensions of a transistor (such as width W, length L, and gain M) directly determine the circuit's operating characteristics, including important performance indicators such as gain, power consumption, bandwidth, and common-mode rejection ratio (CMRR). For example, for a common-emitter amplifier circuit, the gain is directly determined by the formula: gm*RL / (1+gm*RL). By adjusting the transistor size, these performance indicators can be effectively adjusted to meet design goals. Furthermore, transistor size adjustment is flexible and a common and flexible method in circuit design. For example, transistor size optimization can adapt to different process nodes (such as 180nm, 90nm, 45nm, etc.). The legal range of transistor size differs significantly at different process nodes. By modifying the width and length of the transistor, the circuit performance can be adjusted under different process nodes and operating conditions, especially during design optimization. Reinforcement learning can gradually adjust the transistor size based on actual simulation feedback, bringing the circuit's performance indicators closer to the target.

[0062] Reference Figure 1B The method for optimizing the size of analog IC circuits takes the initial netlist of the analog IC as input. The circuit size parameters at this point should be the default parameters from the device model in the technology library used when drawing the circuit diagram. Additionally, the input should include the performance objectives to be achieved, which consist of two parts: circuit performance objectives and objectives that do not violate device size legitimacy. For example, for the current OTA circuit, four performance metrics are set as the performance objectives of the optimization algorithm, and the difference between the pre- and post-simulation metrics is made as close to zero as possible, while incorporating the legitimacy range of the selected device model as the optimization objective, as follows:

[0063]

[0064] By constructing a multi-sided heterogeneous graph neural network based on the simulated IC netlist to extract feature vectors, and by having the SAC agent interact with the simulation platform-based environment, the optimal solution for device size is found, ultimately realizing a method for optimizing the size of simulated IC circuits.

[0065] Where Gain is the gain, which measures the circuit's ability to amplify signals. The calculation formula is: The formula for calculating differential circuits is:

[0066] CMRR: Common-Mode Rejection Ratio, measures a circuit's ability to suppress common-mode signals. A higher CMRR indicates that the circuit can effectively suppress common-mode noise and improve signal quality. The formula for calculating a differential circuit is:

[0067]

[0068] PM: Phase Margin. Phase margin refers to the difference between the system's phase and -180° at the open-loop gain crossover frequency point. The calculation method for differential circuits is as follows: 1. Measure the frequency point when the gain is 1 (0dB). 2. Find the phase at that frequency point. 3. Add 180 to the phase to get the PM value.

[0069] Bandwidth: Bandwidth is the frequency range within which a circuit can effectively amplify a signal. It is usually defined as the frequency range from when the gain drops to -3dB of the maximum gain. The calculation method for differential circuits is as follows: 1) Find the frequency point where the gain is at its maximum as the upper limit; 2) Find the frequency point where the gain is -3dB as the lower limit; 3) The distance between the upper and lower limits is the bandwidth value.

[0070] Reference Figure 2The construction of a heterogeneous graph neural network for a simulated IC netlist is shown in the figure. For example, the current OTA circuit includes 7 transistors and 1 capacitor. The nodes are divided into three types: m-type nodes (transistors), c-type nodes (capacitors), and v-type nodes (power supplies). For multi-terminal devices like transistors and capacitors, each port of the device is treated as a separate node, with the node type matching the device type, resulting in 33 nodes including the three types. Next, the node features of different types of nodes are defined as observation values. m-type nodes have 7 features (id / gm / gds / vth / vdsat / vds / vgs), c-type nodes have 4 features (capacitance / w / l / m), and v-type nodes have 2 features (v / i). Therefore, a 33*13 feature matrix is ​​constructed, with each row representing a node and each row containing 13 parameters, including the feature values ​​of the three types of nodes. Next, a heterogeneous edge and edge feature matrix are constructed for the current OTA circuit. The connection relationships contained in the initial netlist are read. Taking the specific case selected in this invention as an example, there are nodes 0 and 6 in the circuit structure of this case. Node 0 is the drain of transistor MM0, while node 6 is the source of transistor MM1. Through in-depth analysis and interpretation of the analog IC netlist, it can be clearly seen that there is a specific connection relationship between nodes 0 and 6. Further investigation shows that the signal path formed by the connection of the two nodes is marked as net6. This signal path is of great significance in the signal transmission and functional implementation of the circuit. Given the importance of this connection relationship in circuit analysis, it needs to be recorded. Specifically, this edge to be recorded is presented in the form of node pairs, namely [0,6] and [6,0]. This recording method can clearly reflect the bidirectional connection characteristics between the two nodes, providing accurate data support for subsequent circuit topology analysis, fault diagnosis, and other related circuit research. According to this rule, all the connection relationships between the nodes in the initial netlist are read, totaling 48. After recording the node connection relationships in the simulated IC netlist, further in-depth analysis is needed based on the simulation results. Specifically, a detailed signal path analysis is required for each recorded edge. This analysis is conducted in an AC (alternating current) simulation environment to accurately determine whether the signal path corresponding to each edge carries DC or AC signals. To clearly distinguish and manage these two different types of signal paths, they are assigned specific numbers. This numbering method facilitates subsequent data processing and analysis for both DC and AC signal paths. Based on this, an edge type matrix is ​​established, which serves as an important data structure for storing and displaying the signal type information of each edge. Taking the case selected in this invention as an example, there are a total of 48 edges in this specific case.These 48 edges need to be carefully evaluated sequentially. During this evaluation, if the signal path of an edge carries a DC signal, it is marked as 1; if the signal path carries an AC signal, it is marked as 0. In this way, an edge type matrix containing 48 elements (0 or 1) can be constructed. This edge type matrix comprehensively and intuitively reflects the signal type characteristics of all edges in an AC simulation environment, providing crucial data for subsequent in-depth research on the circuit's performance, operating status, and potential problems under different signal types. This helps to further optimize circuit design and related workflows.

[0071] refer to Figure 3 This layout was generated for an OTA circuit containing 7 transistors and 1 capacitor. The circuit netlist is consistent with the netlist in the aforementioned problem, containing 7 transistors (m-type nodes), 1 capacitor (c-type node), and the corresponding power supply (v-type node). The PDK used is a commercial 130nm PDK, which provides multiple layer definitions, covering process constraints from metal layers to ground layers and isolation layers. During the layout generation process, the number of layers and the definitions of each layer strictly adhere to the PDK requirements. The layout generation process follows a series of heuristic algorithms. During the placement process, this invention automatically extracts the matching constraint relationships of each device in the circuit to ensure that the generated layout meets the performance requirements and design rules of the circuit. Specifically, the system automatically identifies and extracts matching constraint relationships by analyzing the relative positions and electrical characteristics of transistors and other devices in the netlist. For example, in this invention, there are specific matching relationships between transistors M0 and M1, and transistors M2 and M4, etc. That is, their size, layout, and electrical performance need to meet certain constraints, such as the relative matching of width-to-length ratio (W / L), to ensure the consistency of key performance indicators such as circuit gain and CMRR. During the routing phase, the A* algorithm is used to plan the connections between transistors, between transistors and capacitors, and between power supplies and other parts of the circuit. By dynamically adjusting the routing path and connection sequence, the signal transmission path is optimized, ensuring the reliability of signal transmission in the circuit and minimizing noise interference. In this stage, the PDK metal layer design rules are also followed to ensure that the connection width, spacing, etc., conform to the design rules, avoiding short circuits and excessive resistance losses.

[0072] refer to Figure 4 The optimization of analog IC circuit dimensions is achieved through the interaction between a Smart Agent (SAC) and a specific environment constructed based on a simulation platform. To ensure the scientific validity and effectiveness of the simulation platform, the ngspice simulator was chosen as a crucial component. The ngspice simulator features high precision and wide applicability, enabling it to accurately simulate the operating states of circuits under various conditions.

[0073] Meanwhile, this invention also includes a powerful simulation result analyzer. The main task of this analyzer is to comprehensively and deeply analyze the simulation results of four circuit performance parameters during both netlist simulation and layout simulation. These four circuit performance parameters cover key indicators such as signal gain, frequency response, power consumption, and noise. Through the analyzer's detailed analysis of these indicators, reliable data support and decision-making basis can be provided for circuit size optimization, thereby further improving the performance and reliability of analog IC circuits. This is achieved through the interaction between the SAC agent and the simulation platform-based environment. The ngspice simulator is selected as an important component of the simulation platform, and a simulation result analyzer is built to analyze the simulation results of the four circuit performance parameters. Finally, the performance results are used to calculate the current reward according to the reward formula: reward = α * performance_score. post-target +β*gap_score post-pre

[0074]

[0075] α and β are weights, and the calculation results are passed into the agent as input.

[0076] A reinforcement learning agent is constructed based on the advanced SAC (Soft Actor-Critic) algorithm, playing a central role in the entire optimization process. It engages in repeated and in-depth interactions with the simulation platform, a crucial step in achieving circuit optimization. In this process, the Actor network within the agent is responsible for outputting the policy distribution of circuit parameters, while the Critic network accurately evaluates the policy quality, providing a basis for parameter adjustment. Simultaneously, the agent cleverly combines stochastic exploration and goal-oriented optimization strategies. Stochastic exploration uncovers potentially hidden better solutions; the goal-oriented optimization strategy ensures that optimization always progresses towards meeting performance requirements. Through the synergistic effect of these two strategies, the agent continuously fine-tunes the network weights, and in this iterative process, the parameters gradually converge to the optimal state. Furthermore, to further optimize the circuit, a heterogeneous graph neural network for multi-terminal device nodes and signal flow recognition is used to model the circuit topology. This network has a unique structure, containing various types of nodes, such as transistor nodes and power supply nodes. It also includes edges of different types, such as DC signal edges and AC signal edges, which together constitute a complete information transmission network. Through this unique structure, heterogeneous graph neural networks can effectively capture the complex topological characteristics and signal transmission patterns of circuits. Furthermore, they can uncover deeply hidden relationships between circuit parameters and performance, providing valuable information to powerfully guide the agent's strategy optimization. Finally, a new circuit netlist is generated based on the parameter strategy output by the agent. This netlist is then verified through pre-simulation, followed by layout generation using a simulated circuit layout generator, and subsequent simulation verification. The results of these two verifications are fed back to the reinforcement learning model, forming a continuous optimization loop. In this cycle, the model is continuously adjusted and improved until a final parameter combination that fully meets performance requirements is obtained. At this point, the optimized circuit parameters, corresponding netlist files, and layout files can be output. These results provide a solid foundation for subsequent circuit design and manufacturing, ensuring high quality and efficiency throughout the entire design and manufacturing process.

[0077] To implement the optimization method described in the above embodiments, the present invention also provides a simulated IC netlist layout collaborative optimization system based on reinforcement learning algorithm, comprising:

[0078] An open-source schematic editor for generating and exporting netlists of analog ICs based on the selected type of IC circuit.

[0079] Analog IC circuit layout generator, used to generate analog IC circuit layouts based on the netlist of analog ICs;

[0080] The analog IC netlist heterogeneous graph generator is used to define different types of nodes in the heterogeneous graph based on the netlist of the analog IC, extract the corresponding node features according to different node types, generate the node, node feature matrix and edge type matrix of the analog IC netlist heterogeneous graph, and finally generate the analog IC netlist heterogeneous graph based on signal flow recognition.

[0081] The simulator is used to pre-build a simulation platform and perform simulation verification. Specifically, it performs simulation verification of the netlist of the analog IC, generates netlist simulation results, and performs simulation verification of the circuit layout of the analog IC, generates layout simulation results. The simulator inputs all simulation results into the optimization module.

[0082] The optimization module is used to pre-build a reinforcement learning agent whose reward function is associated with the performance indicators of the analog IC. Based on the obtained netlist simulation results and layout simulation results, the reinforcement learning agent builds a graph neural network based on the pre-built heterogeneous graph of the analog IC netlist to perform optimization tasks, generating optimized analog IC parameters and corresponding netlist files. The reinforcement learning agent continuously interacts with the simulation platform to optimize until the final parameter combination that meets the performance indicators is obtained, and outputs the optimized circuit parameters and corresponding netlist files for subsequent design and manufacturing.

[0083] Furthermore, the analog IC circuit layout generator specifically generates analog IC circuit layouts based on heuristic algorithms, including the simulated annealing placement algorithm and the A* routing algorithm. In the placement stage, the placement is optimized using the simulated annealing algorithm and the placement tree; in the routing stage, path search is performed based on the A* algorithm. By gradually exploring paths and selecting the path with the lowest cost, the interconnection scheme between devices is finally generated.

[0084] Each module or structure is mainly for implementing the various steps in the method implementation, and will not be described in detail here.

[0085] This application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program, and the program implements corresponding functions when executed by a processor. The computer-readable storage medium of this embodiment implements the method of analog IC netlist layout collaborative optimization based on reinforcement learning algorithm in the implementation embodiment.

[0086] In summary, this invention proposes a method for optimizing analog IC dimensions based on the SAC (Soft Actor-Critic) algorithm. This method automates the optimization of device size selection in analog IC design through reinforcement learning and integrates it with the layout generation process for collaborative optimization. This approach significantly improves optimization efficiency, reduces the time and effort required for traditional manual adjustments, and adaptively adjusts over multiple iterations to find the optimal size configuration, thereby enhancing design efficiency. In this process, the algorithm not only considers circuit performance metrics (such as bandwidth, gain, CMRR, and phase margin) but also achieves close collaboration between the netlist and layout design, ensuring consistency between size optimization and layout, thus avoiding negative impacts on layout caused by size adjustments.

[0087] Through the SAC algorithm based on reinforcement learning, each optimization in the design process is based not only on feedback from circuit performance but also on the generation of the layout and the circuit performance, providing real-time feedback on the optimization results. This strategy of co-optimizing the netlist and the layout allows for simultaneous adjustments to the circuit size optimization and layout, ensuring that the optimized circuit meets performance requirements while the layout can be successfully generated, avoiding layout conflicts or mismatches caused by size adjustments in traditional design processes.

[0088] This method avoids subjective biases and errors inherent in manual design, ensuring consistency and accuracy in the optimization process. It also possesses strong generalization ability, adapting to the design requirements of different circuits. By combining size optimization with layout generation, this invention provides an efficient and intelligent optimization scheme for analog IC design, significantly shortening the design cycle, improving design quality, and demonstrating broad application prospects.

[0089] It should be noted that, depending on the implementation needs, the various steps / components described in this application can be broken down into more steps / components, or two or more steps / components or parts of the operation of steps / components can be combined into new steps / components to achieve the purpose of this invention.

[0090] The sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm, characterized in that, Includes the following steps: Select a specified type of IC circuit and use an open-source schematic editor to generate and export the initial netlist of the analog IC; The initial netlist of the analog IC is input into the pre-built simulation platform, and the simulation results of the netlist are input into the pre-built reinforcement learning agent. The reward function of the reinforcement learning agent is associated with the performance index of the analog IC. The analog IC circuit layout is generated based on the initial netlist of the analog IC, and then simulated through a simulation platform. The simulation results of the layout are then input into the reinforcement learning agent. The reinforcement learning agent builds a graph neural network based on a pre-constructed analog IC netlist heterogeneous graph to perform optimization tasks, generate an optimized analog IC netlist, and send it to the simulation platform. The analog IC netlist heterogeneous graph construction process is as follows: based on the initial analog IC netlist, different types of nodes in the heterogeneous graph are defined, and corresponding node features are extracted according to different node types to generate the nodes, node feature matrix, and edge type matrix of the analog IC netlist heterogeneous graph. Finally, an analog IC netlist heterogeneous graph based on signal flow recognition is generated. A new analog IC circuit layout is generated based on the optimized analog IC netlist. The simulation platform simulates both the optimized analog IC netlist and the new analog IC circuit layout, and then outputs the generated simulation results to the reinforcement learning agent to further optimize the analog IC circuit layout. The reinforcement learning agent continuously interacts and optimizes with the simulation platform until the final parameter combination that meets the performance indicators is obtained, and the optimized circuit parameters and corresponding netlist files are output for subsequent design and manufacturing.

2. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, Specifically, the optimization objective is determined based on performance indicators, and combined with constraints, the two are transformed into corresponding parameters and associated with the reward function of the reinforcement learning agent. Among them, performance indicators include gain, bandwidth, phase margin and common-mode rejection ratio, and constraints include specific power consumption, area and size.

3. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, When setting up the simulation platform, the simulation environment should be configured simultaneously, including simulation mode, simulation temperature, simulation parameters, and units of simulation output items.

4. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, The analog IC circuit layout is generated based on heuristic algorithms, including the simulated annealing placement algorithm and the A* routing algorithm. In the placement stage, the placement is optimized using the simulated annealing algorithm and placement tree. In the routing stage, the path search is performed based on the A* algorithm. By exploring paths step by step and selecting the path with the lowest cost, the connection scheme between devices is finally generated.

5. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, The A* routing algorithm also dynamically adjusts its strategy based on the complexity of the routing to avoid path intersections and blockages, thereby optimizing the signal transmission quality and layout density of the entire layout.

6. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, The reinforcement learning agent interacts repeatedly with the simulation platform, utilizes the policy distribution of the output circuit parameters of the Actor network, evaluates the policy quality using the Critic network, and combines random exploration and goal-oriented optimization strategies to continuously adjust the network weights and gradually converge to the optimal parameters.

7. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to claim 1, characterized in that, The node types of the heterogeneous graph include m nodes representing MOSFET transistors, c nodes representing capacitors, and v nodes representing voltages. Different parameters are extracted as node features for different types of nodes.

8. The method for collaborative optimization of analog IC netlist layout based on reinforcement learning algorithm according to any one of claims 1-7, characterized in that, Specifically, the transistor size in the analog IC circuit layout is optimized.

9. A simulated IC netlist layout collaborative optimization system based on reinforcement learning algorithm, characterized in that, include: An open-source schematic editor for generating and exporting netlists of analog ICs based on the selected type of IC circuit. Analog IC circuit layout generator, used to generate analog IC circuit layouts based on the netlist of analog ICs; The analog IC netlist heterogeneous graph generator is used to define different types of nodes in the heterogeneous graph based on the netlist of the analog IC, extract the corresponding node features according to different node types, generate the node, node feature matrix and edge type matrix of the analog IC netlist heterogeneous graph, and finally generate the analog IC netlist heterogeneous graph based on signal flow recognition. The simulator is used to pre-build a simulation platform and perform simulation verification. Specifically, it performs simulation verification of the netlist of the analog IC, generates netlist simulation results, and performs simulation verification of the circuit layout of the analog IC, generates layout simulation results. The simulator inputs all simulation results into the optimization module. An optimization module is used to pre-build a reinforcement learning agent whose reward function is correlated with the performance metrics of the analog IC; Based on the obtained netlist simulation results and layout simulation results, the reinforcement learning agent builds a graph neural network to perform optimization tasks based on the pre-constructed analog IC netlist heterogeneous graph, generating the optimized analog IC parameters and corresponding netlist files. The reinforcement learning agent continuously interacts and optimizes with the simulation platform until the final parameter combination that meets the performance indicators is obtained, and the optimized circuit parameters and corresponding netlist files are output for subsequent design and manufacturing.

10. The analog IC netlist layout collaborative optimization system based on reinforcement learning algorithm according to claim 9, characterized in that, The analog IC circuit layout generator is specifically based on heuristic algorithms to generate analog IC circuit layouts. The heuristic algorithms include the simulated annealing placement algorithm and the A* routing algorithm. In the placement stage, the placement is optimized by using the simulated annealing algorithm and the placement tree. In the routing stage, the path search is performed based on the A* algorithm. By gradually exploring paths and selecting the path with the lowest cost, the interconnection scheme between devices is finally generated.

Images