Chip thermal layout method based on reinforcement learning
By employing a reinforcement learning-based chip thermal layout method, combined with finite element analysis and policy network optimization, the problems of thermal distribution influence and convergence difficulties in existing technologies are solved, achieving global optimization of chip performance and simultaneous optimization of heat dissipation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NAT INNOVATION INST OF DEFENSE TECH PLA ACAD OF MILITARY SCI
- Filing Date
- 2026-03-12
- Publication Date
- 2026-06-02
AI Technical Summary
Existing mask placement algorithms do not consider the impact of thermal distribution on chip performance, leading to local overheating caused by dense placement, increasing leakage current and temperature gradient, and affecting chip performance; traditional metaheuristic algorithms have difficulty converging in large-scale chip placement and are difficult to achieve global optimization.
A reinforcement learning-based chip hot layout method is adopted. By obtaining the chip's register transfer level description file, static memory replacement and logic synthesis are performed. Combined with finite element analysis and policy network optimization, the optimal hot layout is determined, and heat dissipation and line length are optimized.
It effectively solves the problem of local overheating, ensures chip performance, avoids the local optima and convergence difficulties of traditional algorithms, and achieves global optimization.
Smart Images
Figure CN122133404A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of AI algorithm optimization technology for chip back-end component layout, and particularly relates to a chip hot layout method based on reinforcement learning. Background Technology
[0002] With the development of large-scale integrated circuits, placement is a crucial task that directly affects chip performance, such as speed and energy consumption. In placement tasks, more than 100 macrocells and more than 100,000 standard cells are placed in appropriate locations to meet design specifications, such as line length, wireability, timing, power consumption, maximum temperature, and manufacturability. As chip size continues to increase, manual design is difficult to meet various design specifications simultaneously.
[0003] Currently, a large number of AI algorithms are being used in chip design. For example, reinforcement learning can quickly and efficiently complete automatic chip placement and achieve placement results that surpass those of manual placement. Existing mask placement algorithms avoid component stacking by adding hard constraints to the feedback function and use half-perimeter line length as the reward function, thereby reducing the half-perimeter line length while avoiding component stacking and optimizing the timing design of the chip.
[0004] However, existing mask placement algorithms only use half-perimeter line length as optimization indicators, without considering the impact of thermal distribution on chip performance. This can easily lead to local overheating caused by dense placement, resulting in thermal reliability issues such as increased leakage current and increased temperature gradient. In particular, the chip's highest temperature and large temperature gradient will have an adverse effect on chip performance. Large temperature gradient will increase the clock offset in the clock distribution network, and the positive feedback between high temperature and increased leakage current in semiconductor devices will further aggravate local overheating, seriously affecting chip performance.
[0005] Furthermore, traditional metaheuristic algorithms suffer from convergence difficulties, low parallel computing efficiency, and difficulty in achieving global optimization when laying out large-scale chips, such as those with more than 100 macrocells and more than 100,000 standard cells. Summary of the Invention
[0006] To address some or all of the technical problems existing in the prior art, this invention provides a chip hot layout method based on reinforcement learning.
[0007] The chip hot placement method based on reinforcement learning provided by this invention includes: Obtain the register transfer level description file of the chip and perform static memory replacement, determine the standard cell and macro cell of the chip, set constraints and perform logic synthesis, and obtain the output file of the electronic design automation tool; The three-dimensional thermal field of the chip is mapped onto a two-dimensional plane. Based on the boundary conditions of the chip preset in the output file of the electronic design automation tool, the temperature distribution of the chip is preliminarily analyzed and the rectangular shape of each macrocell and the heat source intensity function are determined. The thermal field distribution of the chip is further determined by combining the finite element analysis method and key optimizations are performed. The chip layout process is modeled, the mask is processed based on a policy network, macrocell placement actions are obtained, and the optimal hot layout of the chip is determined.
[0008] Furthermore, in the aforementioned reinforcement learning-based chip hot placement method, the register transfer level description file of the chip is obtained and static memory replacement is performed; the standard cells and macrocells of the chip are determined; constraints are set and logic synthesis is performed; and the output file of the electronic design automation tool is obtained, including: The chip design configuration is performed using hardware language scripts through the chip field open-source software, generating the chip's register transfer level description file. The static memory in the register transfer level description file is replaced using the memory technology library, and the register transfer level description file after static memory replacement is obtained. At this time, the register transfer level description file contains both standard cells and macro cells. Using the register transfer level description file as a template, set the chip's constraints; The register-transfer level description file after static memory replacement and the chip's constraints are input into the electronic design automation (EDA) tool for logic synthesis to obtain the EDA tool's output file. The EDA tool's output file includes: the chip's gate-level netlist structure, the chip's area, and the chip's power consumption.
[0009] Furthermore, in the aforementioned reinforcement learning-based chip thermal layout method, the three-dimensional thermal field of the chip is mapped to a two-dimensional plane. Based on the boundary conditions of the chip preset in the output file of the electronic design automation tool, the temperature distribution of the chip is initially analyzed, and the rectangular shape of each macrocell and the heat source intensity function are determined. The finite element analysis method is then used to further determine the thermal field distribution of the chip and perform key optimizations, including: An efficient algorithm based on Green's function maps the three-dimensional thermal field of the chip to a two-dimensional plane; The heat conduction equation of a chip is obtained in a two-dimensional plane based on the output file of an electronic design automation tool. Preset the boundary conditions of the chip; Based on the Laplace equation in the heat conduction equation of the chip and the boundary conditions of the chip, the general solution is derived by ignoring the time condition and implemented in the integral function, and the temperature distribution of the chip is preliminarily analyzed. By approximating each macrocell of the chip as a rectangle and introducing a level set function, the geometric description function of each macrocell of the chip is determined. Based on the unit step function, the geometric description function of each macrocell of the chip is projected onto a density field to determine the heat source intensity function of the chip; The structured quadrilateral finite element method is introduced to divide the chip and determine the chip mesh. The heat source intensity function of the chip is used to assign heat source values to the mesh center or mesh nodes. The equilibrium relationship between each grid and its adjacent grids is established based on the equilibrium equations of the elements, and then solved using finite element analysis to further determine the thermal field distribution of the chip.
[0010] Furthermore, in the above reinforcement learning-based chip thermal layout method, the chip is divided into (M, N) networks according to the accuracy of temperature distribution analysis.
[0011] Furthermore, in the above reinforcement learning-based chip thermal layout method, the chip boundary conditions are preset, wherein it is assumed that the four sides of the chip are isolated from the environment, and heat will not flow to the walls in the boundary direction in the two-dimensional plane, but only to the heat sink at the top of the chip or to the printed circuit board at the bottom for heat dissipation.
[0012] Furthermore, in the aforementioned reinforcement learning-based chip hot layout method, the chip's boundary conditions remain unchanged.
[0013] Furthermore, in the aforementioned reinforcement learning-based chip hot placement method, key optimizations include: Calculate the boundary conditions in advance and store them as a parameter; Save the geometric description function of the macrocell, place the components one by one, and incrementally update the geometric description function of the macrocell after each component is placed. The matrix solving process in finite element analysis is ported from the CPU to the GPU.
[0014] Furthermore, in the aforementioned reinforcement learning-based chip hot placement method, the chip placement process is modeled, the mask is processed based on a policy network to obtain macrocell placement actions, and the optimal hot placement of the chip is determined, including: The chip layout process is modeled and transformed into a Markov decision process; A reinforcement learning model is constructed based on a policy network and a value network. The policy network adopts an encoder-decoder structure, takes chip layout distribution as input, and outputs chip layout actions. The optimization metrics for the reinforcement learning model are set based on the line length and the chip's maximum temperature. Set layout constraints for the reinforcement learning model, where each position on the chip canvas is occupied by at most one macrocell, and the routing congestion at each position is less than a fixed threshold. Macrocells are sorted and placed sequentially on the chip canvas from largest to smallest. During each placement process, a policy network receives four masks as input to the reinforcement learning model and outputs the macrocell placement action. The four masks include: view mask, position mask, line length mask, and hot mask. The reward function for chip layout is determined based on four masks. It is then passed back to the neural network through a reinforcement learning algorithm to correct the neuron weights and finally obtain the optimal hot layout of the chip.
[0015] The main advantages of the technical solution of this invention are as follows: This invention provides a reinforcement learning-based chip thermal layout method. It obtains the chip's register-transfer level description file and performs static memory replacement to determine the chip's standard cells and macrocells. Constraints are set and logic synthesis is performed to obtain the output file from an electronic design automation (EDA) tool. The chip's three-dimensional thermal field is mapped to a two-dimensional plane. Boundary conditions are preset based on the EDA tool output file. The chip's temperature distribution is preliminarily analyzed, and the rectangular shape and heat source intensity function of each macrocell are determined. Finite element analysis is then used to further determine the chip's thermal field distribution and perform key optimizations. The chip layout process is modeled, and the mask is processed based on a policy network to obtain macrocell placement actions, determining the optimal thermal layout of the chip. This invention effectively ensures chip performance by simultaneously optimizing heat dissipation and line length, and avoids the local optima and convergence problems of traditional algorithms through reinforcement learning algorithms. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only for further understanding of the embodiments of the present invention and constitute a part of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart illustrating the chip hot placement method based on reinforcement learning provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the chip thermal layout optimization process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the reinforcement learning model provided in an embodiment of the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0018] refer to Figure 1 The reinforcement learning-based chip hot placement method provided in this invention is applicable to hot placement optimization in the chip design process, and includes the following steps: S101: Obtain the register transfer level description file of the chip and perform static memory replacement, determine the standard cell and macro cell of the chip, set constraints and perform logic synthesis processing, and obtain the output file of the electronic design automation tool; S102: Map the three-dimensional thermal field of the chip to a two-dimensional plane, preset the chip boundary conditions based on the output file of the electronic design automation tool, preliminarily analyze the temperature distribution of the chip and determine the rectangular shape of each macrocell and the heat source intensity function, and further determine the thermal field distribution of the chip and perform key optimizations by combining the finite element analysis method. S103: Model the chip layout process, process the mask based on the policy network, obtain the macrocell placement action, and determine the optimal hot layout of the chip.
[0019] In this embodiment of the invention, the register transfer level description file of the chip, i.e. the chip's RTL file, is used for digital circuit design and verification.
[0020] In this embodiment of the invention, the standard unit of a chip refers to the basic logic unit or storage unit in the chip design process, which is the basis of digital circuit design; the macro unit refers to the unit preset in the chip design process to implement a specific function.
[0021] In this embodiment of the invention, constraints are set, wherein constraints refer to conditions used in the chip design process to ensure the normal operation of the chip, such as clock constraints, which are used to ensure that the circuits in the chip operate normally at a specified clock frequency.
[0022] In this embodiment of the invention, the output file of the electronic design automation tool, i.e., the EDA output file, includes the gate-level netlist structure and area, power consumption, and other statuses.
[0023] In this embodiment of the invention, the three-dimensional thermal field of the chip is mapped onto a two-dimensional plane. Based on the boundary conditions of the chip preset in the output file of the electronic design automation tool, the temperature distribution of the chip is initially analyzed and the rectangular shape of each macrocell and the heat source intensity function are determined. The thermal field distribution of the chip is further determined by combining the finite element analysis method and key optimizations are performed, namely the chip layout process, and the chip is processed in two dimensions.
[0024] In this embodiment of the invention, the chip is divided into (M, N) networks according to the accuracy of temperature distribution analysis. The larger M and N are, the more detailed the analysis of the chip temperature layout is.
[0025] In this embodiment of the invention, the chip's boundary conditions are preset, wherein it is assumed that all four sides of the chip are isolated from the environment, and heat will not flow to the walls in the boundary direction in the two-dimensional plane, but will only flow to the heat sink at the top of the chip or to the printed circuit board at the bottom for heat dissipation.
[0026] In this embodiment of the invention, the boundary conditions of the chip remain unchanged.
[0027] In this embodiment of the invention, key optimizations include: pre-calculating boundary conditions and storing them as a parameter; saving the geometric description function of macro elements, placing components sequentially, and incrementally updating the geometric description function of macro elements after each component placement; and porting the matrix solution in finite element analysis from the CPU to the GPU for execution.
[0028] In this embodiment of the invention, the three-dimensional thermal field refers to the temperature distribution of the chip in three-dimensional space.
[0029] In this embodiment of the invention, each macrocell is approximated as a rectangle.
[0030] In this embodiment of the invention, the macrocell placement action refers to the action of placing macrocells on the chip canvas during the chip design process.
[0031] In this embodiment of the invention, the optimal thermal layout of the chip refers to the chip layout determined after optimizing the line length and heat dissipation simultaneously and performing reinforcement learning.
[0032] In step S101, the register transfer level description file of the chip is obtained and static memory replacement is performed. The standard cells and macro cells of the chip are determined, constraints are set and logic synthesis is performed. The output file of the electronic design automation tool is obtained, including: The chip design configuration is performed using hardware language scripts through the chip field open-source software, generating the chip's register transfer level description file. The static memory in the register transfer level description file is replaced using the memory technology library, and the register transfer level description file after static memory replacement is obtained. At this time, the register transfer level description file contains both standard cells and macro cells. Using the register transfer level description file as a template, set the chip's constraints; The register-transfer level description file after static memory replacement and the chip's constraints are input into the electronic design automation (EDA) tool for logic synthesis to obtain the EDA tool's output file. The EDA tool's output file includes: the chip's gate-level netlist structure, the chip's area, and the chip's power consumption.
[0033] In embodiments of the present invention, such as Figure 2 The diagram illustrates the chip hot placement optimization process, which includes: Chipyard open-source software; Chisel configuration, which configures the chip design using hardware language scripts via Chipyard open-source software; RISC-V, an open-source instruction set architecture based on the Reduced Instruction Set Computing principle; Verilog, a hardware description language used for circuit implementation; (seq std)RTL files, i.e., time standard RTL files; RTL files, i.e., register transfer level description files; SRAM mapping, i.e., static memory mapping; (SRAM)RTL files, i.e., RTL files after static memory replacement; Synopsys, an electronic design automation tool; and RL (reinforcement learning) models, i.e., reinforcement learning networks.
[0034] In this embodiment of the invention, the output file of the electronic design automation tool is used as the input file in the layout stage for subsequent chip two-dimensional layout design and reinforcement learning of chip layout design.
[0035] In step S102, the three-dimensional thermal field of the chip is mapped onto a two-dimensional plane. Based on the boundary conditions of the chip preset in the output file of the electronic design automation tool, the temperature distribution of the chip is preliminarily analyzed and the rectangular shape of each macrocell and the heat source intensity function are determined. The thermal field distribution of the chip is further determined using the finite element analysis method, and key optimizations are performed, including: An efficient algorithm based on Green's function maps the three-dimensional thermal field of the chip to a two-dimensional plane; The heat conduction equation of a chip is obtained in a two-dimensional plane based on the output file of an electronic design automation tool. Preset the boundary conditions of the chip; Based on the Laplace equation in the heat conduction equation of the chip and the boundary conditions of the chip, the general solution is derived by ignoring the time condition and implemented in the integral function, and the temperature distribution of the chip is preliminarily analyzed. By approximating each macrocell of the chip as a rectangle and introducing a level set function, the geometric description function of each macrocell of the chip is determined. Based on the unit step function, the geometric description function of each macrocell of the chip is projected onto a density field to determine the heat source intensity function of the chip; The structured quadrilateral finite element method is introduced to divide the chip and determine the chip mesh. The heat source intensity function of the chip is used to assign heat source values to the mesh center or mesh nodes. The equilibrium relationship between each grid and its adjacent grids is established based on the equilibrium equations of the elements, and then solved using finite element analysis to further determine the thermal field distribution of the chip.
[0036] In this embodiment of the invention, the heat conduction equation of the chip is obtained in a two-dimensional plane based on the output file of the electronic design automation tool; ; in, Indicates the chip in coordinates Location, Time The temperature at that time; Indicates standard deviation; Indicates the chip's time Temperature gradient over time; Represents the gradient operator; Indicates the thermal conductivity of the chip; Indicates the chip in coordinates Location, Time The heat power density per unit area (or unit volume) at that time; This indicates the coordinate range of the chip.
[0037] In this embodiment of the invention, the boundary conditions of the chip are preset; ; in, Indicates the chip's x-axis dimension; Indicates the chip's y-axis dimension; Indicates the chip's z-axis dimension; This represents the temperature gradient at coordinate r along the x-axis of the chip at time t. This represents the temperature gradient at coordinate r on the y-axis of the chip at time t. Indicates the chip's z-axis is Temperature gradient at coordinate r at time t; This represents the temperature gradient at coordinate r when the chip's z-axis is 0, and at time t. This indicates the main heat flow to the chip's heat sink; This indicates the secondary heat flow to the printed circuit board of the chip; Indicates the chip in coordinates Temperature at location and time t; Indicates the chip in coordinates Temperature at location and time t; This indicates the thermal conductivity of the chip.
[0038] In this embodiment of the invention, based on the Laplace equation in the heat conduction equation of the chip and the boundary conditions of the chip, the general solution is derived by ignoring the time condition and implemented in the integral function, which is used to approximate the temperature distribution in the chip, thereby achieving a preliminary analysis of the temperature distribution of the chip.
[0039] In this embodiment of the invention, during the calculation of temperature distribution, the solution in the z-axis direction is independent of the solutions in the x-axis and y-axis directions. Therefore, dimensionality reduction processing in the z-axis direction is beneficial to facilitating the thermal analysis of the temperature distribution of the chip.
[0040] In this embodiment of the invention, a three-dimensional finite element analysis method is used to analyze the heat distribution on the chip.
[0041] In this embodiment of the invention, each macrocell of the chip is approximated as a rectangle, and a level set function is introduced to determine the geometric description function of each macrocell of the chip. ; in, The geometric description function representing the macrocell; Integer values representing the number of macrocells; Indicates the semi-major axis length of the macrocell; Indicates the semi-minor axis length of the macrocell; Represents the coordinates of the macrocell; This represents the coordinates of the geometric center corresponding to the macrocell.
[0042] In this embodiment of the invention, the geometric description function of each macrocell of the chip is projected onto a density field based on the unit step function to determine the heat source intensity function of the chip; ; in, The function representing the heat source intensity of the chip; Indicates the chip number The intensity distribution function of a heat source; Indicates the chip number The geometric description function of each heat source corresponds to a macrocell; This indicates that the chip's first step function is based on the unit step function. The geometric description function of a macrocell corresponding to a heat source is projected onto a density field to determine the value. This indicates the number of heat sources on the chip.
[0043] In this embodiment of the invention, a unit step function is used; ; When the unit step function is equal to 1, the corresponding region is the area occupied by each macrocell of the chip projected into a density field, and heat sources are distributed in this region.
[0044] In this embodiment of the invention, the balance equation of the elements; ; in, Represents the element heat transfer matrix; Represents the temperature vector of an element node; This represents the hot load vector of the equivalent element node.
[0045] In this embodiment of the invention, the thermal field distribution of the chip refers to the distribution of heat within the chip.
[0046] In this embodiment of the invention, the thermal field distribution of the chip can be used as a thermal mask input in subsequent models.
[0047] In this embodiment of the invention, the chip thermal layout process refers to the process of planning the layout of the chip's heat sources. Components need to be placed one by one. During each placement process, thermal analysis of the thermal field distribution of the chip thermal layout is required. The efficiency of the chip thermal analysis directly affects the efficiency of the chip thermal layout. Therefore, key optimization of the chip's thermal field distribution is necessary.
[0048] In this embodiment of the invention, the heat source of the chip refers to the source of heat generated by the chip during operation, such as the various components in the chip.
[0049] In this embodiment of the invention, key optimizations are performed on the thermal field distribution of the chip, including: pre-calculating the boundary conditions and storing them as a parameter; saving the geometric description function of the macrocell, placing components one by one, and incrementally updating the geometric description function of the macrocell after each component is placed; and porting the matrix solution in finite element analysis from the CPU to the GPU for execution.
[0050] In this embodiment of the invention, the boundary conditions of the chip remain unchanged. Therefore, the boundary conditions are calculated in advance and stored as a parameter, which effectively avoids the inefficiency caused by repeated calculation of boundary conditions during thermal analysis.
[0051] In this embodiment of the invention, the geometric description function of the macrocell is saved, and the components are placed one by one. After each component is placed, the geometric description function of the macrocell is incrementally updated, which avoids the need to traverse all components for each thermal analysis process and optimizes the time complexity.
[0052] In this embodiment of the invention, the geometric description function of the macrocell is stored, and components are placed sequentially. After each component is placed, the geometric description function of the macrocell is incrementally updated, reducing the time complexity from... Reduce to ,in, Indicates the number of components.
[0053] In this embodiment of the invention, time complexity refers to the efficiency of calculating the geometric description function of a macrocell.
[0054] In this embodiment of the invention, the chip thermal layout process involves matrix solving. Traditional methods typically use CPUs for matrix solving, while GPUs are more efficient at matrix solving than CPUs. Therefore, the matrix solving is ported from the CPU to the GPU for computation, thereby improving the efficiency of chip thermal analysis processing.
[0055] For example, the boundary conditions of the chip are calculated in advance and stored as parameter A. When component B1 is placed, parameter A is directly called to perform thermal analysis of the thermal field distribution. When component B2 is placed, parameter A is again directly called to perform thermal analysis of the thermal field distribution.
[0056] For example, during the chip hot layout process, components B1, B2, and B3 are placed sequentially.
[0057] For example, before placing component B1, the geometric description function C0 of the macrocell is obtained; when placing component B1, the geometric description function C1 of the macrocell is obtained by incremental update based on the geometric description function C0 of the macrocell; when placing component B2, the geometric description function C1 of the macrocell is updated by incremental update.
[0058] For example, before key optimizations, the time complexity of placing components B1, B2, and B3 sequentially is . After key optimizations, the time complexity of placing components B1, B2, and B3 sequentially is... .
[0059] For example, when performing chip thermal layout, the thermal analysis processing time without key optimization is no less than 0.1 seconds, and the thermal analysis processing time after key optimization is controlled within 0.1 seconds.
[0060] In step S103, the chip layout process is modeled, the mask is processed based on the policy network, macrocell placement actions are obtained, and the optimal hot layout of the chip is determined, including: The chip layout process is modeled and transformed into a Markov decision process; A reinforcement learning model is constructed based on a policy network and a value network. The policy network adopts an encoder-decoder structure, takes chip layout distribution as input, and outputs chip layout actions. The optimization metrics for the reinforcement learning model are set based on the line length and the chip's maximum temperature. Set layout constraints for the reinforcement learning model, where each position on the chip canvas is occupied by at most one macrocell, and the routing congestion at each position is less than a fixed threshold. Macrocells are sorted and placed sequentially on the chip canvas from largest to smallest. During each placement process, a policy network receives four masks as input to the reinforcement learning model and outputs the macrocell placement action. The four masks include: view mask, position mask, line length mask, and hot mask. The reward function for chip layout is determined based on four masks. It is then passed back to the neural network through a reinforcement learning algorithm to correct the neuron weights and finally obtain the optimal hot layout of the chip.
[0061] In this embodiment of the invention, the Markov decision process refers to the process by which a policy network makes decisions in the context of an environment, and is used for reinforcement learning.
[0062] In this embodiment of the invention, modeling the chip layout process and transforming it into a Markov decision process means converting the chip layout into an optimization problem, which minimizes the chip temperature distribution and satisfies the constraints by adjusting the position of each macrocell in the chip.
[0063] In this embodiment of the invention, the reinforcement learning model refers to a model that uses reinforcement learning algorithms to perform reinforcement learning on the chip hot layout process in order to obtain the optimal chip hot layout.
[0064] In this embodiment of the invention, the policy network refers to a network used to output corresponding actions based on the current environmental state; the value network refers to a network used to predict whether the result is good or bad based on the current environmental state.
[0065] In this embodiment of the invention, the line length refers to the length of the chip, which can be approximately obtained based on the half-perimeter of the chip and determined by calculating the sum of the half-perimeters of the rectangles of the macrocells in the chip.
[0066] In this embodiment of the invention, the highest temperature refers to the highest temperature obtained during the chip thermal layout process. The power density of each macrocell in the chip is obtained through chip power to determine the thermal field of the chip, and the highest temperature is extracted from it.
[0067] In this embodiment of the invention, chip placement action refers to the actions performed by the chip during the placement process.
[0068] In this embodiment of the invention, chip layout distribution refers to the distribution of each component in the chip.
[0069] In this embodiment of the invention, the optimization metric refers to the metric used by the reinforcement learning model to optimize the hot layout of the chip.
[0070] In this embodiment of the invention, layout constraints refer to the constraints that the reinforcement learning model needs to satisfy when performing hot chip layout, in order to avoid overlap between macrocells.
[0071] In this embodiment of the invention, the chip canvas refers to a preset area for chip thermal layout, used in chip thermal layout design.
[0072] In this embodiment of the invention, the optimization index of the reinforcement learning model is set according to the line length and the highest temperature of the chip, and the layout constraints of the reinforcement learning model are set. ; in, Indicates the line length of the macrocell; Indicates the temperature of the macrocell; This indicates taking the minimum value; This indicates taking the maximum value; This represents the balance parameter between line length and maximum temperature. Indicates the amount of overlap between macrocells; This indicates routing congestion between macrocells; A fixed threshold representing routing congestion between macrocells; Indicates the length of the component; Indicates the width of the component.
[0073] In embodiments of the present invention, such as Figure 3 As shown, a reinforcement learning model is constructed based on a policy network and a value network, wherein, Indicates pixel-level module representation; arrows indicate the propagation process; Indicates the state of the macrocell; Represents a chip canvas; Indicates the location mask; Indicates the line length mask; Indicates a hot mask; Indicates encoder; Indicates decoder; Represents the reward function; Indicates a merger; Represents a value network; Represents a policy network; This is a reward calculator; This indicates positional embedding.
[0074] In this embodiment of the invention, a position mask is used to avoid macrocell overlap; a view mask is used to record chip layout; a line length mask is used to constrain line length; and a thermal mask is used to constrain temperature and avoid local overheating.
[0075] For example, the position mask avoids the overlap of macrocells B1, B2, and B3; the view mask is used to record the layout of macrocells at time t and time t-1; the line length mask is used to constrain the line length of macrocells to not exceed c1; and the thermal mask constrains the temperature of macrocells to not exceed T1.
[0076] In this embodiment of the invention, the reward calculator calculates the reward function by weighting the increment of the line length and the highest temperature.
[0077] In this embodiment of the invention, macrocells are placed from largest to smallest.
[0078] In this embodiment of the invention, a reward function for chip layout is determined based on four masks; ; in, Represents the reward function; Indicates time The length of the line at that time; Indicates time The length of the line at that time; Indicates time The highest temperature at that time; Indicates time The highest temperature at that time; This represents the balance parameter between line length and maximum temperature.
[0079] In this embodiment of the invention, after the reward calculation is completed, it is back-propagated to the neural network through a reinforcement learning algorithm to correct the neuron weights. Through multiple iterations, the optimal hot layout of the chip is finally obtained.
[0080] The optimal thermal layout of a chip refers to the layout with the minimum overall line length and the lowest temperature obtained through a reinforcement learning model.
[0081] In summary, this invention provides a reinforcement learning-based chip thermal layout method. It obtains the chip's register-transfer level description file and performs static memory replacement, determines the chip's standard cells and macrocells, sets constraints and performs logic synthesis, and obtains the output file of an electronic design automation (EDA) tool. The three-dimensional thermal field of the chip is mapped to a two-dimensional plane. Based on the EDA tool output file, the chip's boundary conditions are preset, and the chip's temperature distribution is initially analyzed to determine the rectangular shape of each macrocell and the heat source intensity function. The finite element method is then used to further determine the chip's thermal field distribution and perform key optimizations. The chip layout process is modeled, and the mask is processed based on a policy network to obtain macrocell placement actions, determining the optimal thermal layout of the chip. This invention effectively ensures chip performance by simultaneously optimizing heat dissipation and line length, and avoids the local optima and convergence problems of traditional algorithms through reinforcement learning algorithms.
[0082] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Additionally, the terms "front," "back," "left," "right," "upper," and "lower" in this document refer to the placement shown in the accompanying drawings.
[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A chip hot placement method based on reinforcement learning, characterized in that, include: Obtain the register transfer level description file of the chip and perform static memory replacement, determine the standard cell and macro cell of the chip, set constraints and perform logic synthesis, and obtain the output file of the electronic design automation tool; The three-dimensional thermal field of the chip is mapped onto a two-dimensional plane. Based on the boundary conditions of the chip preset in the output file of the electronic design automation tool, the temperature distribution of the chip is preliminarily analyzed and the rectangular shape of each macrocell and the heat source intensity function are determined. The thermal field distribution of the chip is further determined by combining the finite element analysis method and key optimizations are performed. The chip layout process is modeled, the mask is processed based on a policy network, macrocell placement actions are obtained, and the optimal hot layout of the chip is determined.
2. The chip hot placement method based on reinforcement learning according to claim 1, characterized in that, Obtain the chip's register-transfer level description file and perform static memory replacement; determine the chip's standard cells and macrocells; set constraints and perform logic synthesis; obtain the output files from the electronic design automation tool, including: The chip design configuration is performed using hardware language scripts through the chip field open-source software, generating the chip's register transfer level description file. The static memory in the register transfer level description file is replaced using the memory technology library, and the register transfer level description file after static memory replacement is obtained. At this time, the register transfer level description file contains both standard cells and macro cells. Using the register transfer level description file as a template, set the chip's constraints; The register-transfer level description file after static memory replacement and the chip's constraints are input into the electronic design automation (EDA) tool for logic synthesis to obtain the EDA tool's output file. The EDA tool's output file includes: the chip's gate-level netlist structure, the chip's area, and the chip's power consumption.
3. The chip hot placement method based on reinforcement learning according to claim 1, characterized in that, The three-dimensional thermal field of the chip is mapped onto a two-dimensional plane. Based on the boundary conditions preset in the output file of the electronic design automation tool, the temperature distribution of the chip is initially analyzed, and the rectangular shape of each macrocell and the heat source intensity function are determined. The thermal field distribution of the chip is further determined using the finite element analysis method, including: An efficient algorithm based on Green's function maps the three-dimensional thermal field of the chip to a two-dimensional plane; The heat conduction equation of a chip is obtained in a two-dimensional plane based on the output file of an electronic design automation tool. Preset the boundary conditions of the chip; Based on the Laplace equation in the heat conduction equation of the chip and the boundary conditions of the chip, the general solution is derived by ignoring the time condition and implemented in the integral function, and the temperature distribution of the chip is preliminarily analyzed. By approximating each macrocell of the chip as a rectangle and introducing a level set function, the geometric description function of each macrocell of the chip is determined. Based on the unit step function, the geometric description function of each macrocell of the chip is projected onto a density field to determine the heat source intensity function of the chip; The structured quadrilateral finite element method is introduced to divide the chip and determine the chip mesh. The heat source intensity function of the chip is used to assign heat source values to the mesh center or mesh nodes. The equilibrium relationship between each grid and its adjacent grids is established based on the equilibrium equations of the elements, and then solved using finite element analysis to further determine the thermal field distribution of the chip.
4. The chip hot placement method based on reinforcement learning according to claim 3, characterized in that, Based on the accuracy of temperature distribution analysis, the chip is divided into (M, N) networks.
5. The chip hot placement method based on reinforcement learning according to claim 3, characterized in that, The chip's boundary conditions are preset, assuming that all four sides of the chip are isolated from the environment, and that heat will not flow to the walls in the boundary direction in the two-dimensional plane, but will only flow to the heat sink at the top of the chip or to the printed circuit board at the bottom for heat dissipation.
6. The chip hot placement method based on reinforcement learning according to claim 5, characterized in that, The boundary conditions of the chip remain unchanged.
7. The chip hot placement method based on reinforcement learning according to claim 3, characterized in that, Key optimizations include: Calculate the boundary conditions in advance and store them as a parameter; Save the geometric description function of the macrocell, place the components one by one, and incrementally update the geometric description function of the macrocell after each component is placed. The matrix solving process in finite element analysis is ported from the CPU to the GPU.
8. The chip hot placement method based on reinforcement learning according to claim 1, characterized in that, The chip placement process is modeled, and the mask is processed based on a policy network to obtain macrocell placement actions, thereby determining the optimal hot layout of the chip, including: The chip layout process is modeled and transformed into a Markov decision process; A reinforcement learning model is constructed based on a policy network and a value network. The policy network adopts an encoder-decoder structure, takes chip layout distribution as input, and outputs chip layout actions. The optimization metrics for the reinforcement learning model are set based on the line length and the chip's maximum temperature. Set layout constraints for the reinforcement learning model, where each position on the chip canvas is occupied by at most one macrocell, and the routing congestion at each position is less than a fixed threshold. Macrocells are sorted and placed sequentially on the chip canvas from largest to smallest. During each placement process, a policy network receives four masks as input to the reinforcement learning model and outputs the macrocell placement action. The four masks include: view mask, position mask, line length mask, and hot mask. The reward function for chip layout is determined based on four masks. It is then passed back to the neural network through a reinforcement learning algorithm to correct the neuron weights and finally obtain the optimal hot layout of the chip.