Method and apparatus for placement of cores in a thermal optimization oriented integrated circuit

Abstract

The application provides a chip grain layout method and device for thermal optimization in an integrated circuit, and belongs to the field of integrated circuits. In the process of gradually placing the chip grain, the current partial layout state is converted into an image representation containing position, wiring and thermal information after each action is executed, and an instant reward signal is constructed based on the state, so that the conversion from sparse reward to dense reward is realized. Further, the method constructs a thermal mask based on a thermal interaction potential, and forms a fusion mask in combination with a line length mask; local and global layout features are extracted through a double-branch mask coding network, and an action probability distribution is generated by using a guided soft mask mechanism with feasibility rollback. In this way, effective feedback information can be continuously provided in the decision-making process, which helps to guide the strategy to evolve in a better layout direction, thereby improving the training efficiency and convergence performance.

Description

Technical Field

[0001] This application relates to the field of integrated circuits, and more particularly to a method and apparatus for laying out chips in integrated circuits for thermal optimization. Background Technology

[0002] As Moore's Law slows its pace of growth, traditional two-dimensional monolithic integrated circuits are gradually showing limitations in performance improvement, power consumption control, and heterogeneous integration. As an important technology for advanced packaging, 2.5D integrated circuits (2.5D-ICs) achieve high-density integration of multiple functional chips through silicon interposers or redistribution layers. While maintaining relatively mature manufacturing processes, they also offer high interconnect bandwidth, low interconnect power consumption, and strong system integration flexibility, thus showing broad application prospects in high-performance computing, artificial intelligence, and data centers.

[0003] like Figure 1 As shown, in 2.5D integrated circuits, various functional modules such as computing, storage, and input / output are typically divided into multiple relatively independent and optimized chips. The chips are fixed to an interposer layer 11a via microbumps 12a. High-density through-silicon vias (TSVs) 13a are formed in the interposer layer 11a. The microbumps interconnect the chips with the organic substrate through the interposer wiring structure in the interposer layer 11a, the TSVs 13a, and C4 bumps 14a located on the opposite side of the interposer layer 11a relative to the chips. The opposite side of the chip relative to the interposer layer 11a is in contact with a thermal interface material 15a. A heat spreader 16a and a heat sink 17a are sequentially stacked on the surface of the thermal interface material 15a. Compared to traditional chip-level layout problems, chip layout not only needs to consider interconnect length, space utilization, and manufacturability, but also must address the thermal coupling problem caused by the aggregation of high-power chips within a limited interposer layer. If multiple high-power chips are arranged too close together, local hot spots can easily form, which in turn affects the system's thermal balance, reliability, and operational stability.

[0004] Therefore, how to simultaneously consider routing costs and heat distribution during the placement phase and construct an automatic placement optimization method applicable to 2.5D-IC scenarios has become an important research problem in the field of chiplet physical design. Summary of the Invention

[0005] This application provides a method and apparatus for laying out chips in integrated circuits with thermal optimization, aiming to achieve automated placement of chips in 2.5D integrated circuits.

[0006] In a first aspect, this application provides a layout method for chips in an integrated circuit for thermal optimization, comprising: The design information of the integrated circuit is obtained, and the chip layout process is modeled as a thermally sensitive Markov decision process based on the design information of the integrated circuit. Determine the placement order of the core particles and use this order as the execution order for subsequent step-by-step layout decisions; The process involves acquiring the layout state features of the core particles and constructing a reinforcement learning model. The reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features. The current core particle to be placed is then placed according to the action probability matrix, and the immediate reward for the placement action of the current core particle is calculated. This immediate reward is calculated based on routing cost and thermal coupling cost. After updating the layout state features based on the placed core particles, the above steps are repeated to place the next core particle. After all core particles in the current round are placed, the next iteration is performed. During the iteration process, the reinforcement learning model is updated every few iterations until the reinforcement learning model converges, completing the core particle placement. The Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle placement trajectory at the end of each iteration, and then summing the immediate rewards from the current time to the termination time in that trajectory according to a discount factor.

[0007] Optionally, the wiring cost is determined based on the increment of the half-circumference line length.

[0008] Optionally, the thermal coupling cost is calculated based on the following formula:

[0009] In the formula: This represents the thermal coupling cost between the chip to be placed and the chips already placed. Indicates the first The power consumption of each placed chip. The power consumption of each chip to be placed. Indicates the first The Euclidean distance between the geometric centers of the already placed core particles and the core particle to be placed. To prevent small positive numbers with unstable values.

[0010] Optionally, the reinforcement learning model includes: Strategy networks and value networks; The policy network includes: Local branches, global branches, and merged modules; The local branch is used to extract local features around the core to be placed based on the layout state features, which is used to reflect overlapping constraints, local wiring effects and short-range thermal coupling information. The global branch is used to extract global features based on the layout state characteristics, which are used to reflect the overall distribution trend of the placed core particles, global wiring and thermal coupling information; The fusion module is used to fuse the local features and the global features to calculate the action probability matrix, which is used for core layout. The value network outputs the state value of the current layout state based on the position embedding of the current placement step, or based on the position embedding of the current placement step and global encoding features, in order to estimate the expected cumulative reward that can be obtained in the subsequent layout process starting from the current layout state.

[0011] Optionally, the layout state features include: position mask, line length mask, thermal mask, view mask, and fusion mask; The position mask is used to indicate the legal placement area for the current core particle to be placed and the next core particle to be placed; The line length mask is used to represent the half-circumference line length increment caused when the current core particle to be placed or the next core particle to be placed is placed in different candidate positions; The thermal mask is used to represent the increase in thermal interaction potential caused when the current core particle to be placed or the next core particle to be placed is placed at different candidate positions; The view mask is used to represent the overall layout state of the core particles that have been placed. The fusion mask is obtained by linearly combining line length mask and thermal mask according to weights, and is used to uniformly represent wiring information and thermal coupling information.

[0012] Optionally, the process by which the fusion module fuses the local features and the global features to calculate the action probability matrix includes: By fusing the local features and the global features, a logits graph is obtained; An adaptive threshold is constructed based on a guided image containing wiring costs and / or thermally related costs, along with an adjustable soft coefficient. Generate a guidance inhibition mask based on an adaptive threshold; Generate a physical feasibility mask based on the layout legality constraints of the current core particles to be deployed; A soft screening mask is obtained based on the guidance inhibition mask and the physical feasibility mask. The candidate layout actions of the logits graph are soft-screened based on the soft screening mask. During the soft screening process, if there is at least one feasible candidate position in the soft screening mask, the logits graph is masked using the soft screening mask. If the soft screening mask is empty, the logits graph is masked using the physical feasibility mask. Based on the filtered logits graph, the motion probability distribution of the current core particles to be deployed is calculated.

[0013] Optionally, the loss function of the policy network is:

[0014] In the formula: Represents the expectation function, This indicates the objective function used by the policy network. For policy entropy, The entropy regularization coefficient;

[0015] In the formula: This is the cutting factor. It represents the probability ratio of the placement actions on the same core particle in any two adjacent iterations over several iterations; Indicates range clipping, used to... Limited to and between; This represents the normalized advantage value of the particle placement strategy during a certain round of iteration; the unnormalized advantage value is determined based on the state value of the particle placement strategy output by the value network and the Monte Carlo reward calculated based on the reward of each particle placement process in the particle placement strategy.

[0016] Optionally, the loss function of the value network is:

[0017] In the formula: Represents the expectation function, This represents the smoothed L1 loss function; This represents the state value calculated by the value network based on a certain round of core particle placement strategy; This represents the Monte Carlo cumulative reward calculated based on the reward for each core placement process in the core placement strategy.

[0018] Optionally, during the iteration process of the reinforcement learning model, the PPO method can be used to iteratively train the reinforcement learning model.

[0019] Secondly, this application provides a layout apparatus for chips in thermally optimized integrated circuits, comprising: The modeling module acquires the design information of the integrated circuit and models the chip layout process as a thermally sensitive Markov decision process based on the design information of the integrated circuit. The placement order determination module determines the placement order of the core particles and uses this order as the execution order for subsequent step-by-step layout decisions; The layout module acquires the layout state features of the core particles and constructs a reinforcement learning model. The reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features. It then places the current core particle according to the action probability matrix and calculates the immediate reward for the placement action of the current core particle. The immediate reward is calculated based on the routing cost and thermal coupling cost. After updating the layout state features based on the placed core particles, the above steps are repeated to place the next core particle. After all core particles in the current round are placed, the next iteration is performed. During the iteration process, the reinforcement learning model is updated every few iterations until the reinforcement learning model converges, completing the core particle layout. The Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle layout trajectory at the end of each iteration, and then summing the immediate rewards from the current time to the termination time in that layout trajectory according to a discount factor.

[0020] Compared with the prior art, the present invention has the following beneficial effects: First, this invention models the chiplet placement process as a thermally sensing Markov decision process, enabling the agent to consider the effects of wiring costs and thermal coupling effects simultaneously during the step-by-step placement process, rather than optimizing only a single metric, thereby facilitating a comprehensive balance between line length and thermal distribution.

[0021] Second, this invention proposes a thermal interaction potential (TIP) model, which approximates the thermal coupling effect between high-power chiplets with lower computational cost, avoiding frequent calls to complex thermal simulation processes during training, thereby improving layout optimization efficiency and reducing computational overhead.

[0022] Third, this invention uses various state representation methods such as position mask, line length mask, thermal mask, view mask and fusion mask to uniformly encode layout legality, wiring information and thermal information, enabling the policy network to more accurately identify potential hot spots and bad layout trends, thereby improving decision quality.

[0023] Fourth, the GSM-FF mechanism proposed in this invention can softly suppress obviously suboptimal actions, while ensuring that the action space is always non-empty through a feasible backoff mechanism, thereby improving the training stability and convergence robustness under large-scale discrete action spaces.

[0024] Fifth, this invention employs fixed advantage estimation, entropy regularization, and an early stopping strategy based on KL divergence to further enhance the training stability of reinforcement learning algorithms in high-dimensional discrete layout decision problems.

[0025] Sixth, the method of the present invention has obtained better or similar Pareto solution distributions in multiple test designs, and exhibits a strong line length-thermal coupling trade-off search capability under the same thermal sensing agent target.

[0026] Seventh, in post-layout physical evaluation, the method of the present invention can maintain good thermal performance while reducing wiring length, indicating that the optimization of the proxy target by the method can be effectively converted into actual physical design benefits. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0028] Figure 1 A schematic diagram of a 2.5D integrated circuit structure provided for related technologies; Figure 2 (a) is a flowchart of a core placement method based on simulated annealing algorithm provided by related technologies; Figure 2 (b) in the figure is a flowchart of the core placement method based on the enumeration algorithm provided by the relevant technology; Figure 3 (a) in the figure is a flowchart of a core particle layout method based on reinforcement learning provided by related technologies; Figure 3 (b) is a flowchart of a reinforcement learning-based chip placement method provided in an embodiment of this application; Figure 4 A flowchart illustrating a thermally optimized chip placement method for integrated circuits according to an embodiment of this application; Figure 5 (a) is a top view of an inter-chip interconnection structure provided in an embodiment of this application; Figure 5 (b) is a cross-sectional view of the inter-chip interconnect structure provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a core layout model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a global branch provided in an embodiment of this application; Figure 8 (a) is a schematic diagram of an action screening process provided in an embodiment of this application; Figure 8 (b) is a schematic diagram of a strategy optimization process provided in an embodiment of this application; Figure 9 A structural block diagram of a thermally optimized integrated circuit chip layout device provided in one embodiment of this application; Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0029] The attached figures are labeled as follows: 11a: Intermediate layer; 12a: Microbump; 13a: Through-silicon via; 14a: C4 bump; 15a: Thermal interface material; 16a: Heat sink; 17a: Heat sink; 11: Policy Network; 111: Local Branch; 112: Global Branch; 113: Fusion Module; 12: Value Network; 21: Modeling module; 22: Placement order determination module; 23: Layout module; 31: Processor; 32: Memory. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] Existing automatic chip placement methods for 2.5D integrated circuits can be broadly classified into three categories: heuristic optimization methods, combinatorial search methods, and reinforcement learning methods.

[0032] In terms of heuristic optimization methods, existing research largely uses simulated annealing as the core solution framework. For example, based on simulated annealing and B... The multi-chip collaborative planning method with a tree structure can simultaneously handle the layout problems of chips, macromodules, and input / output buffers. Furthermore, some studies achieve cross-layer collaborative optimization by jointly considering network topology and layout factors during simulated annealing. Other methods embed the thermal simulation process into the layout optimization flow, thereby achieving joint optimization of line length and thermal distribution. These methods are relatively mature in engineering implementation, but essentially still belong to the search mode based on local perturbations. Their algorithm flow is as follows: Figure 2 As shown in (a) of the diagram.

[0033] In terms of combinatorial search methods, existing research has employed enumeration, network flow, and constraint satisfaction to model and solve chip placement problems. For example, the placement problem can be transformed into a combinatorial optimization problem or a constraint satisfaction problem, and then solved using techniques such as branch and bound. These methods can accurately represent complex constraints, but their solution complexity typically increases significantly with design scale. Their algorithm flow is as follows: Figure 2 As shown in (b) of the diagram.

[0034] In reinforcement learning methods, existing research typically models the core placement process as a stepwise decision-making problem. This involves encoding the placement states using graph structures or image representations, allowing an agent to sequentially determine the placement position or order of each module. For example... Figure 3 As shown in (a), this type of method typically evaluates the overall layout through wiring estimation and thermal simulation after all cores have been placed, thereby obtaining a reward signal for policy updates.

[0035] However, the rewards obtained by the above methods are usually only calculated after the complete layout is generated, which is a typical form of sparse reward. This sparse feedback mechanism is difficult to reflect the quality of intermediate layout states in a timely manner, which is not conducive to the stable training of reinforcement learning strategies and can easily lead to slow convergence speed or even unstable training.

[0036] In response to the above problems, such as Figure 3 As shown in (b), during the step-by-step placement of the core, this invention converts the current partial layout state into a graphical representation containing position, wiring, and thermal information after each step, and constructs an instantaneous reward signal based on this state, thereby achieving the transformation from sparse reward to dense reward. In this way, effective feedback information can be continuously provided during the decision-making process, which helps guide the strategy to evolve towards a better layout, thereby improving training efficiency and convergence performance.

[0037] In addition, during the layout phase, to avoid calling the detailed routing process in each iteration, existing methods usually use a surrogate model with half-perimeter line length as the routing cost to approximately reflect the routing quality of the layout result.

[0038] Figure 4 A flowchart illustrating a thermally optimized chip placement method for integrated circuits according to an embodiment of this application. See also... Figure 4 ,include: S101. Obtain the design information of the integrated circuit, and model the chip layout process as a thermally sensitive Markov decision process based on the design information of the integrated circuit.

[0039] In some examples, the design information includes at least the size information of several chips, power consumption information, interconnection information, interposer size information, and layout constraint information.

[0040] In this embodiment, the intermediary layer is divided into: The discrete grid is used, and the core placement problem is modeled as a thermally sensitive Markov decision process by defining the core placement action as an action.

[0041] It should be noted that in the thermally sensing Markov decision process, the reward for each action is determined based on the layout cost and thermal coupling cost of the core particle layout action, so that the core particle layout process can comprehensively consider the layout cost and thermal coupling cost.

[0042] In some examples, the heat-sensing Markov decision process is defined by the state space, action space, state transition function, reward function, and termination condition.

[0043] In this application, the state space is the set of all possible partial layout states during the chip-by-chip layout process. For the first... Step-by-step layout decision, current state This represents the current partial layout state, including at least the elements that have already been placed. The information includes the location of each chip, the chip to be placed, the next chip to be placed, chip size information, power consumption information, interconnection relationship information, and the legality of the current layout.

[0044] Preferably, the current state The layout state is represented by pixel-level features, which include a position mask, a line length mask, a thermal mask, a view mask, and a fusion mask. The position mask represents the legal placement area for the current and next core particles to be placed; the line length mask represents the increment in half-perimeter line length caused by placing the current or next core particle at different candidate positions; the thermal mask represents the increment in thermal interaction potential caused by placing the current or next core particle at different candidate positions; the view mask represents the overall layout state of the completed core particle placement; and the fusion mask is obtained by a weighted linear combination of the line length mask and the thermal mask, used to uniformly represent routing information and thermal coupling information.

[0045] The action space is the set of candidate placement positions for the current core particle on the discretized intermediate layer mesh. For the first... Step-by-step planning and decision-making, actions This indicates that a grid position is selected from the set of candidate placement positions as the current placement location for the core. The action probability matrix output by the policy network is not the action space itself, but rather the probability distribution of the policy network in the action space. Each element in the action probability matrix represents the probability that the current core will be placed in the corresponding candidate grid position.

[0046] In performing the action Then, place the currently selected core particle into the action position. The corresponding candidate positions are determined, and the already placed core set, position mask, line length mask, thermal mask, view mask, and fusion mask are updated to obtain the next state. Once all the core particles to be laid out have been placed, one round of the layout process is complete.

[0047] A state transition function is a function that updates the layout state after the current placement of the core particle is completed; it can also be understood as a state update operation.

[0048] In some examples, a reward function is used to describe the quality of a particle placement action. In this embodiment, after each particle placement action is completed, a reward is calculated for that action to obtain the reward for the current particle to be placed. The calculated reward can be used to optimize the particle placement strategy.

[0049] In some examples, the termination condition is that all the cores of the current wheel have been placed, which means that the termination condition of the current wheel is met.

[0050] In some examples, the Markov decision process for heat sensing is as follows: Assumptions include The design information of a 2.5D integrated circuit is obtained from each chip to be laid out. for:

[0051] In the formula: Indicates the side length of the intermediate layer; Indicates the first The width of each core particle, Indicates the first The height of each core particle, Indicates the first The power consumption of each chip; This indicates the interconnection information between the core particles. ,like Indicates the first The first core and the first Information on the interconnection relationships between individual core particles.

[0052] In some examples, the interconnection information of the core components includes communication requirements or connection strength.

[0053] During the placement of the core particles, the AI ​​determines the candidate placement positions of the current core particles based on their current layout state. Initially, if no core particles are placed on the intermediary layer, the intermediary layer will be divided into... Each of the discrete grid points corresponds to a candidate placement position for a core particle. After each core particle is placed on the intermediate layer, the current layout state of the core particle is updated. When placing the core particle next time, the candidate placement position of the core particle is determined based on the updated current layout state.

[0054] For the One core particle to be placed, and the current layout state of the core particles. From the already placed The information consists of the position of each core particle, the current core particle to be placed, the next core particle to be placed, the legal layout area, the line length increment information, and the thermal coupling increment information.

[0055] For the A core particle to be placed, layout action. This indicates selecting one position from all candidate placement positions as the first position. The target placement position of the core to be placed (specifically, the target placement position can be set as the lower left corner coordinate or the center reference coordinate of the core).

[0056] During the layout action After that, the first Each core particle is placed at the target placement position, and the current layout state is checked. Update to obtain the next state. Regarding the current layout state During the update, the placed core set, feasible core placement areas, line length mask, and thermal mask are updated.

[0057] When all When all core particles have been placed, the core particle placement process for the current round ends, and the set of placement coordinates for all core particles is output: ,in, Indicates the first The x-coordinate of each core particle, Indicates the first The ordinate of each core particle.

[0058] In some examples, the immediate reward for each placement action of the core is determined by both the wiring cost and the thermal coupling cost.

[0059] In some examples, wiring cost refers to the cost of interconnecting the core chips.

[0060] See Figure 5 The diagram illustrates the routing method between chips. The chip layout process needs to take into account the routing method between chips; otherwise, the routing process between chips will become complicated, which is detrimental to the performance of integrated circuits.

[0061] In this application, considering the computationally high cost of obtaining the chip routing length by calling the detailed router during the iterative optimization of chip placement, this application uses half-perimeter length (HPWL) to construct a routing cost proxy model to simplify the calculation process of chip routing length. For each chip placement network, the routing cost of the chip placement network is obtained by calculating the minimum bounding box half-perimeter surrounding all pins of the chip placement network, and the half-perimeter lengths (HPWL) of all chip placement networks are summarized as the routing optimization target in the chip placement stage.

[0062] In some examples, the thermal coupling cost of the chip refers to the thermal effect between chips. If only the wiring cost between chips is considered during the chip placement process, without considering the thermal effect between chips, the final chip placement scheme may lead to local overheating of the integrated circuit, which will be detrimental to the performance of the integrated circuit.

[0063] In some examples provided in this application, the thermal coupling cost is calculated by constructing a thermal interaction potential model. Under steady-state heat conduction conditions, each core particle is approximated as a concentrated heat source located at its geometric center. As can be seen from the linear superposition relationship of the thermal Green's function, the temperature field can be obtained by superimposing the contributions of each heat source.

[0064] For quasi-two-dimensional packaging structures such as the interposer layer of 2.5D integrated circuits, considering that the accurate heat core is related to the packaging stack structure, boundary conditions, material parameters and heat dissipation path, this application adopts a monotonic distance-weighted approximation that is inversely proportional to the square of the distance during the layout optimization stage to characterize the trend of enhanced thermal coupling when the high-power chips are close together.

[0065] In some examples, the thermal coupling cost is calculated based on the following formula:

[0066] In the formula: This represents the thermal coupling cost between the chip to be placed and the chips already placed. Indicates the first The power consumption of each placed chip. The power consumption of each chip to be placed. Indicates the first The Euclidean distance between the geometric centers of the already placed core particles and the core particle to be placed. To prevent small positive numbers with unstable values.

[0067] The heat-related reward is obtained by taking the negative of the heat coupling cost, i.e.:

[0068] in, This represents the thermally relevant reward for the layout action corresponding to the currently pending core particle.

[0069] Based on the above discussion, in this application, the reward for each chip placement action is... It can be expressed based on the increments of the half-circumference length (HPWL) and the thermal interaction potential (TIP) as follows:

[0070] In the formula: This is a trade-off coefficient between the target line length and the target thermal performance. This indicates the half-cycle long-term increment introduced by the current core particle layout action. This represents the increase in thermal interaction potential introduced by the current core particle layout action.

[0071] In this way, the agent can receive real-time feedback at each step of the core placement process, thereby transforming the sparse reward of evaluating the overall core placement action only after all cores have been placed into a dense reward of gradually providing feedback on the core placement action.

[0072] S102. Determine the placement order of the core particles to be placed, and use this order as the execution order for subsequent layout decisions.

[0073] In some examples, the placement order is determined according to the connectivity of each chiplet, prioritizing chiplets that are connected to more other chiplets.

[0074] In subsequent steps, the agent can make layout decisions for each chiplet in this order until a complete layout is completed.

[0075] S103. Obtain the layout state features of the core particles and construct a reinforcement learning model; the reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features; the current core particle to be placed is placed according to the action probability matrix, and the immediate reward of the placement action of the current core particle is calculated, the immediate reward is calculated based on the wiring cost and thermal coupling cost; after updating the layout state features based on the placed core particles, the above steps are repeated to place the next core particle; after all core particles in the current round are placed, the next round of iteration is performed; during the iteration process, the reinforcement learning model is updated every few rounds until the reinforcement learning model converges and the core particle placement is completed; wherein, the Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle placement trajectory at the end of each iteration, and then summing the immediate rewards of each step from the current time to the termination time in the placement trajectory according to the discount factor.

[0076] In order to enable the agent to simultaneously perceive the layout legality, line length information and thermal information of the core particles during the decision-making process, this application extracts multiple pixel-level feature maps as layout state features (also the current layout state).

[0077] In some examples, the layout state features include: a position mask, a line length mask, a thermal mask, a view mask, and a fusion mask; the position mask is used to represent the legal placement area of ​​the current core to be placed and the next core to be placed; the line length mask is used to represent the half-perimeter line length increment caused by placing the current core to be placed or the next core to be placed in different candidate positions; the thermal mask is used to represent the thermal interaction potential increment caused by placing the current core to be placed or the next core to be placed in different candidate positions; the view mask is used to represent the overall layout state of the cores that have been placed; the fusion mask is obtained by linearly combining the line length mask and the thermal mask according to weights, and is used to uniformly characterize the routing information and thermal coupling information.

[0078] See Figure 6 In some examples, the reinforcement learning model includes: Policy network 11 and value network 12; The policy network 11 includes: Local branch 111, global branch 112, fusion module 113; The local branch 111 is used to extract local features around the core to be placed based on the layout state features, which are used to reflect overlapping constraints, local wiring effects and short-range thermal coupling information. The global branch 112 is used to extract global features based on the layout state features, which are used to reflect the overall distribution trend of the placed core particles, global wiring and thermal coupling information; The fusion module 113 is used to fuse the local features and the global features to calculate the action probability matrix, which is used for core layout. The value network 12 outputs the state value of the current layout state based on the position embedding of the current placement step, or based on the position embedding of the current placement step and global encoding features, in order to estimate the expected cumulative reward that can be obtained in the subsequent layout process starting from the current layout state.

[0079] In some examples, the input to the reinforcement learning model is a state tensor formed by concatenating multiple two-dimensional feature maps. .by Taking the layout mesh as an example, each mask is represented as a size of A two-dimensional matrix, the input state tensor Represented as:

[0080] In the formula: , These are the position masks for the current core and the next core, respectively. , These are the line length masks for the current core and the next core, respectively. , These are the thermal masks for the current core and the next core, respectively. A view mask for the core particles that have been placed; , These are the fusion masks for the current core and the next core, respectively.

[0081] In some examples, the fusion mask Normalized line length mask and thermal mask After performing a linear combination, we get:

[0082] In the formula: This represents the coefficients of the linear combination.

[0083] In some examples, the output of a reinforcement learning model consists of two parts: Part 1: Action probability matrix output by policy network 11 , Action probability matrix Each element in the matrix represents the probability of placing the current core at the corresponding grid position. Each grid position represents a candidate placement position for a core, and each candidate placement position has an action probability value. All action probabilities are represented by the action probability matrix. To express.

[0084] Part Two: State Values ​​Output by Value Network 12 State value It is a scalar used to estimate the expected cumulative reward of the current layout state in subsequent layout processes.

[0085] In some examples, local branch 111 takes a location mask and a fusion mask as input and extracts local spatial features through shallow convolution to reflect overlap constraints, local wiring effects, and short-range thermal coupling information. Specifically, the input to local branch 111 is the current location mask. , With fusion mask , The splicing result, with an input size of .

[0086] Local branch 111 is used to extract local spatial features around the core to be placed. Local branch 11 extracts overlapping constraints, line length increment changes, and short-range thermal coupling changes in the local neighborhood through shallow two-dimensional convolution.

[0087] In some examples, local branch 111 includes a lightweight channel fusion convolution module, specifically including: The system consists of three sequentially connected 2D convolutional layers. The first convolutional layer uses a 1×1 kernel to map the 4-channel input features to an 8-channel feature map, maintaining the same spatial size. The output of the first convolutional layer is activated by a ReLU function and then fed into the second convolutional layer. The second convolutional layer also uses a 1×1 kernel and outputs an 8-channel feature map, again maintaining the same spatial size. The output of the second convolutional layer is activated by a ReLU function and then fed into the third convolutional layer. The third convolutional layer uses a 1×1 kernel and outputs a 1-channel feature map. The third convolutional layer does not perform any pooling operations, and the feature map spatial scale is not compressed.

[0088] Local branch 111 ultimately outputs a single-channel feature map with the same size as the input space, completing only channel-dimensional feature fusion and dimensionality compression.

[0089] In some examples, global branch 112 takes the view mask and fusion mask as input, extracts coarse-grained global context information through a lightweight global encoder-decoder network, and then recovers a full-resolution feature map aligned with the layout grid through upsampling. The encoder-decoder branch of the global mask is used to extract the overall distribution trend of placed chiplets and global wiring and thermal coupling information. The input of the global branch is the view mask. With fusion mask , The splicing result.

[0090] by For example, global branch 112 consists of an encoder, a fully connected compression layer, and a decoder. The network structure of the global branch is as follows: Figure 7 As shown.

[0091] The encoder portion of global branch 112 consists of three consecutive 2D convolutional layers, with each 2D convolutional layer followed by a pooling layer: The first convolutional layer outputs a 64×64 feature map with 16 channels, which is then pooled to obtain 32×32 scale features. The second convolutional layer outputs a 32-channel feature map, which is pooled again to obtain 16×16 scale features. The third convolutional layer outputs a 64-channel feature map, which is compressed into a 64×4×4 global feature map through pooling or adaptive pooling. The global feature map is then flattened into a 1024-dimensional vector, compressed into a 256-dimensional vector through a fully connected layer, and then reshaped into a 16×4×4 low-resolution feature block.

[0092] The decoder part of the global branch 112 gradually restores the spatial resolution through multi-level two-dimensional transposed convolutions, successively restoring feature maps of scales 8×8, 16×16, 32×32, and 64×64, and finally obtaining a global feature map consistent with the layout grid size. The global branch 112 can preserve the global layout trend with low model complexity, enabling the policy network 11 to not only focus on local legitimacy, but also perceive the overall layout compactness, potential hotspot regions, and global connection trends.

[0093] To introduce wiring and thermal information guidance into a large-scale discrete action space while avoiding excessive compression of the action space, this invention proposes a Guide-based Soft Masking with Feasibility Fallback (GSM-FF) mechanism to soft-screen candidate actions of the policy network 11. In some examples, the GSM-FF mechanism uses a guide image (logits map obtained from global and local features) containing line length and thermal information as input to soft-screen candidate actions of the policy network 11. This Guide-based Soft Masking mechanism is applied to the fusion process of the fusion module 113.

[0094] In some examples, the fusion module 113 is used to fuse the local features and the global features, and the process of calculating the action probability matrix includes: Step 1: Fuse the local features and the global features to obtain the logits graph.

[0095] In this embodiment, the local feature map output by local branch 111 and the global feature map output by global branch 112 Having the same space size By concatenating the two along the channel dimension, a joint feature map is obtained. :

[0096] Subsequently, the local feature map is processed by 1×1 convolution. With global feature map Merge to form the final state representation This also forms the logits graph (unnormalized layout action score), which is used in subsequent steps. express.

[0097]

[0098] The 1×1 convolution is used to learn the weight relationship between local feasibility information and global layout trends, enabling the network to simultaneously consider local overlap constraints, short-range hot coupling, global chiplet distribution, and long-range connection trends. This structure can reduce model complexity while taking into account both local decision-making ability and global layout trend perception.

[0099] Step 2: Construct an adaptive threshold based on the guide image containing wiring costs and / or thermally related costs, and an adjustable soft coefficient.

[0100] like Figure 8 As shown in (a) above, given a boot image containing thermally dependent or wiring-dependent costs, assuming the boot image is... Guide image Each element in This indicates that the current chiplet will be placed in the first position. The guiding value corresponding to each candidate grid position. It can be obtained from line-length masks, thermal masks, or a fusion of both, with the fused mask used as the guiding image. The fusion process of the fusion mask is as follows:

[0101] First, according to the guide image Minimum value and adjustable soft coefficient Constructing an adaptive threshold .

[0102] First, calculate the guide image. The minimum generation value is calculated, and a soft screening coefficient is added to this minimum generation value. To obtain the adaptive threshold :

[0103] In the formula: This is the soft screening coefficient, used to control the range of candidate positions retained. When... When the value is small, only positions close to the current optimal guiding value are retained; when... When the value is large, more candidate actions are allowed to enter the policy selection range.

[0104] Step 3: Generate a guidance inhibition mask based on the adaptive threshold.

[0105] Based on adaptive threshold Generate a guidance inhibition mask Soft suppression is applied to candidate positions that are clearly suboptimal. For the first... If a candidate position has a guiding cost greater than the adaptive threshold, then... If the candidate position is deemed significantly suboptimal under the current line length or thermal coupling conditions, it is marked as needing suppression. Otherwise, the candidate position is retained.

[0106] In some examples, a guidance inhibition mask is used. The generation process can be represented as:

[0107] in, This indicates that the candidate position was suppressed by the guidance information. This indicates that the candidate position was not suppressed. Suppression does not mean permanently prohibiting the action, but rather performing soft filtering on obviously undesirable positions based on line length and thermal information in the current state.

[0108] Step 4: Generate a physical feasibility mask based on the layout legality constraints of the current core particles to be deployed.

[0109] Subsequently, a physically feasible mask is generated based on the current layout legality constraints. For the current chip to be placed... If its width and height are respectively , Candidate positions The corresponding coordinates are Then the candidate position It is necessary to satisfy both boundary constraints and non-overlapping constraints.

[0110] In some examples, boundary constraints are represented as: , , ,

[0111] In some examples, the non-overlapping constraint is expressed as follows: after the current chiplet is placed at a candidate position, its rectangular region must not intersect with the rectangular regions of any already placed chiplets. Let the occupied region of the current chiplet be... , No. The occupied area of ​​each placed chip is Then the following is required:

[0112] If the spacing between micro-bumps or the wiring clearance is taken into account, a preset safety margin can be added to the outer edge of the chiplet rectangular area during the validity check. Then, the intersection is determined using the expanded rectangle.

[0113] Based on the above discussion, a physically feasible mask is defined as follows:

[0114] in, This indicates that the candidate location is physically feasible. This indicates that the candidate position is not feasible.

[0115] Step 5: Obtain a soft screening mask based on the guidance inhibition mask and the physical feasibility mask, and perform soft screening on the candidate layout actions of the logits graph based on the soft screening mask; during the soft screening process, if there is at least one feasible candidate position in the soft screening mask, the logits graph is masked using the soft screening mask; if the soft screening mask is empty, the logits graph is masked using the physical feasibility mask.

[0116] After obtaining the guidance inhibition mask and physical feasibility mask Then, the two are logically combined to obtain a set of soft screening actions.

[0117] because This indicates that candidate positions are suppressed by guiding information, while Indicating that the candidate location is physically feasible, the soft-screen mask is defined as follows:

[0118] In other words, a candidate location will only enter the set of soft screening actions if it is physically feasible and not suppressed by guiding information.

[0119] In some examples, to avoid the possibility of empty selectable actions due to overly strong guidance filtering, this invention further introduces a feasibility fallback mechanism.

[0120] like Then the final action mask is: .

[0121] like If the current guidance screening is too strict, then we should revert to the physical set of available actions. .

[0122] By combining the above-mentioned mechanisms and backtracking, it is possible to suppress suboptimal actions by utilizing line length and thermal information, while ensuring that there is at least one physically feasible action for each placement decision.

[0123] In some examples, after flattening the 2D logits, a mask is applied to the locations where motion is not possible:

[0124] Therefore, the probability of a position being excluded by the final action mask is 0, and the probability of retaining a position is determined by the logits output by the policy network.

[0125] Step 6: Calculate the motion probability distribution of the current core particles to be deployed based on the filtered logits graph.

[0126] In some examples, the processed Performing softmax yields the action probability distribution:

[0127] In the formula, This indicates the first step after the final action masking process. Unnormalized action scores for each candidate layout position; This indicates the first step after the final action masking process. Unnormalized action scores for each candidate layout position.

[0128] During the training phase, the agent can sample according to this probability distribution, and during the inference phase, it can select the candidate position with the highest probability as the target placement position of the current core particle.

[0129] In some examples, this application employs a stable and enhanced proximal policy optimization (PPO) method to train the layout policy of the policy network 11.

[0130] After each iteration, a complete core particle layout trajectory is obtained by sampling (i.e., the trajectory composed of all core particle layout positions in the current round), and the core particle layout trajectory at each time step is calculated. The Monte Carlo reward and the current state value are calculated, and the advantage value of the current wheel core particle layout strategy is calculated based on the current state value and the Monte Carlo reward. .

[0131] The current state value is calculated based on the embedding of the current placement step of the core particle, or based on the embedding of the current placement step of the core particle and the global state features. The Monte Carlo reward is calculated based on the immediate reward of each core particle placement action.

[0132] Every few iterations, the reinforcement learning model undergoes a policy network update and a value network update. During the policy network update, the probability ratio of the action probability distribution calculated by the policy network when performing a placement operation on the same core particle is calculated for each iteration. The probability is higher than This can reflect the differences between the old and new deployment strategies of the policy network. Subsequently, the policy network is based on probability ratios. Advantages of current wheel core layout strategies The objective function used in the PPO algorithm is updated, and then the policy network is updated.

[0133] like Figure 8 As shown in (b), after obtaining the core particle layout trajectory (the core particle layout trajectory or layout scheme composed of all core particles after a certain round of layout operations is completed) according to the current core particle layout strategy, the layout state of each core particle in the core particle layout trajectory and the Monte Carlo reward of each core particle layout action pair are calculated first. Then, the state value is estimated using the value network 12, and the advantage value is calculated based on the Monte Carlo reward and the state value.

[0134] In some examples, the value network 12 is used to estimate the expected cumulative reward of the current layout state. Unlike the policy network, which outputs an action probability matrix, the value network 12 outputs a scalar state value.

[0135] The state value is used to estimate the expected cumulative reward that can be obtained in the subsequent core placement process starting from the current layout state.

[0136] In one example provided in this application, the value network 12 uses the current placement step number The position is embedded to obtain the current state value.

[0137]

[0138] In the formula: Indicates a fully connected mapping. This indicates the position embedding corresponding to the current placement step number. The position embedding is used to indicate which core placement stage we are currently in.

[0139] In another example provided in this application, the value network 12 takes the globally encoded features extracted from the global branch 112 as its main input, and combines them with the current placement step. The position is embedded to obtain the current state value.

[0140] The global state features reflect the overall spatial distribution, line length trend, and thermal coupling trend of the placed core particles, while the position embedding indicates the current core particle placement stage. The two are concatenated and input into the fully connected layer to obtain the current state value. :

[0141] in, This represents the state characteristics output by the global encoder. This indicates the embedding position corresponding to the current placement step number. Indicates a fully connected mapping. This indicates feature splicing.

[0142] After obtaining a complete layout trajectory through sampling, the first step is to calculate the time step at each moment. Monte Carlo cumulative returns:

[0143] in, This represents the termination step number of the current wheel core particle layout trajectory. As a discount factor, For the first The reward obtained from the step-by-step core layout action.

[0144] Subsequently, the advantage value is calculated based on the cumulative returns and value network output. :

[0145] Among them, the dominance value Used to measure actual actions performed. The cumulative returns obtained afterward are better or worse than the expected returns of the value network. When this occurs, it indicates that the action is better than the current value network expectation, and the probability of this action should be increased; when If the action is worse than expected, the probability of that action should be reduced.

[0146] Then, regarding the aforementioned advantage value Normalization is performed within the trajectory range, that is, the mean and standard deviation of all dominant values ​​in the current sampled trajectory or trajectory buffer are calculated, and each dominant value is then normalized. Standardized as:

[0147] in, and These are the mean and standard deviation of the dominant value within the trajectory or trajectory buffer, respectively. To prevent small positive numbers from being divided by zero, the normalized dominant value is used. The scale is more stable, which can reduce the variance of policy gradient estimation and improve the stability of PPO updates.

[0148] In some examples, the policy network 11 updates by pruning the alternative objective function to control the difference between the old and new policies; at the same time, an entropy regularization term is added to the objective function to enhance the early exploration capability of training.

[0149] When updating the policy, first calculate the probability ratio of the old and new policies for the same action. :

[0150] This ratio deviates too much from 1, indicating that the new policy differs too much from the old policy, which may lead to training instability. Therefore, PPO uses pruning as an alternative objective function to limit the policy update magnitude.

[0151] in, This represents the clipping factor. If the advantage value is positive, increasing the probability of this action is beneficial for optimization, but when the probability ratio exceeds [a certain threshold], it becomes less desirable. After that, the objective function no longer encourages increasing the probability; if the advantage value is negative, then reducing the probability of that action is beneficial for optimization, but when the probability ratio is lower than... Afterward, the objective function no longer encourages a decrease in probability. This pruning mechanism prevents the policy from changing too drastically in a single update.

[0152] In some examples, the loss function of policy network 11 is:

[0153] In the formula: Represents the expectation function, The objective function of the strategy is represented. For policy entropy, is the entropy regularization coefficient. The entropy regularization term is used to enhance the exploration ability in the early stages of training and prevent the policy from converging to a local optimum too early.

[0154] In some examples, the loss function of value network 12 is:

[0155] In the formula: Represents the expectation function, This represents the smoothed L1 loss function; This represents the value estimate given by the value network in its current state; This indicates the cumulative return in Monte Carlo.

[0156] In addition, to further stabilize the training process, the KL divergence between the current policy and the old policy is monitored during each mini-batch update:

[0157] when Exceeding the preset threshold In such cases, the current round of policy updates is terminated early to avoid overly aggressive policy updates. Value Network 12 updates by minimizing the smoothed L1 loss between the predicted state value and the Monte Carlo reward.

[0158] In this application, the overall network update process of the reinforcement learning model is as follows: First, use the current policy network (Actor) to sample the core placement location according to the action probability distribution processed by GSM-FF, and collect data. one transition ( (The total number of cores is given, and the action probability of each core corresponds to 256 possible core placement positions). Then, the Monte Carlo reward for each state-action pair is calculated from back to front. The value network (Critic) is used to obtain the state value and calculate the normalized fixed advantage. Then, within 3 PPO epochs, the pruning target is constructed using the probability ratio of the new and old policies and the entropy regularization is added to update the policy network (Actor). At the same time, the value network (Critic) is fitted with the Monte Carlo reward using Smooth L1 loss, and the training stability is enhanced by KL thresholding and gradient pruning.

[0159] When the policy network and value network converge, the training of the reinforcement learning model is completed, and the final core layout model is obtained. When the core layout model is obtained, all the cores to be placed have been placed synchronously.

[0160] Figure 9 This is a structural block diagram of a thermally optimized chip placement device for integrated circuits, provided as an embodiment of this application. See also... Figure 9 ,include: Modeling module 21 is used to acquire the design information of the integrated circuit and model the chip layout process as a thermally sensitive Markov decision process based on the design information of the integrated circuit. The placement order determination module 22 is used to determine the placement order of the core particles and to arrange the core particles based on the placement order; The layout module 23 acquires the layout state features of the core particles and constructs a reinforcement learning model. The reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features. It then lays out the current core particle according to the action probability matrix and calculates the immediate reward for the layout action of the current core particle. The immediate reward is calculated based on the routing cost and thermal coupling cost. After updating the layout state features based on the laid-out core particles, the above steps are repeated to lay out the next core particle. After all core particles in the current round are laid out, the next iteration is performed. During the iteration process, the reinforcement learning model is updated every few iterations until the reinforcement learning model converges, completing the core particle layout. The Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle layout trajectory at the end of each iteration, and then summing the immediate rewards from the current time to the termination time in that layout trajectory according to a discount factor.

[0161] Figure 10 This is a structural block diagram of an electronic device provided according to an embodiment of this application. See also... Figure 10 Electronic devices may include Figure 9 The aforementioned layout arrangement of chips in a thermally optimized integrated circuit. Typically, the electronic device includes a processor 31 and a memory 32. The processor 31 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 31 may be implemented using at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), or PLA (Programmable Logic Array). The processor 31 may also include a main processor and a coprocessor. The main processor is used to process data in the wake-up state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. The memory 32 may include one or more computer-readable storage media, which may be non-transitory. The memory 32 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage medium in memory 32 is used to store at least one instruction, which is executed by processor 31 to implement the thermally optimized chip layout method for integrated circuits performed by an electronic device provided in the method embodiments of this application.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for laying out chips in an integrated circuit for thermal optimization, characterized in that, include: The design information of the integrated circuit is obtained, and the chip layout process is modeled as a thermally sensitive Markov decision process based on the design information of the integrated circuit. Determine the placement order of the core particles and use this order as the execution order for subsequent step-by-step layout decisions; The process involves acquiring the layout state features of the core particles and constructing a reinforcement learning model. The reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features. The current core particle to be placed is then placed according to the action probability matrix, and the immediate reward for the placement action of the current core particle is calculated. This immediate reward is calculated based on routing cost and thermal coupling cost. After updating the layout state features based on the placed core particles, the above steps are repeated to place the next core particle. After all core particles in the current round are placed, the next iteration is performed. During the iteration process, the reinforcement learning model is updated every few iterations until the reinforcement learning model converges, completing the core particle placement. The Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle placement trajectory at the end of each iteration, and then summing the immediate rewards from the current time to the termination time in that trajectory according to a discount factor.

2. The chip layout method for thermally optimized integrated circuits according to claim 1, characterized in that, The wiring cost is determined based on the increment of the half-circumference line length.

3. The chip layout method for thermally optimized integrated circuits according to claim 2, characterized in that, The thermal coupling cost is calculated based on the following formula: In the formula: This represents the thermal coupling cost between the chip to be placed and the chips already placed. Indicates the first The power consumption of each placed chip. The power consumption of each chip to be placed. Indicates the first The Euclidean distance between the geometric centers of the already placed core particles and the core particle to be placed. To prevent small positive numbers with unstable values.

4. The chip layout method for thermally optimized integrated circuits according to claim 1, characterized in that, The reinforcement learning model includes: Strategy networks and value networks; The policy network includes: Local branches, global branches, and merged modules; The local branch is used to extract local features around the core to be placed based on the layout state features, which is used to reflect overlapping constraints, local wiring effects and short-range thermal coupling information. The global branch is used to extract global features based on the layout state characteristics, which are used to reflect the overall distribution trend of the placed core particles, global wiring and thermal coupling information; The fusion module is used to fuse the local features and the global features to calculate the action probability matrix, which is used for core layout. The value network outputs the state value of the current layout state based on the position embedding of the current placement step, or based on the position embedding of the current placement step and global encoding features, in order to estimate the expected cumulative reward that can be obtained in the subsequent layout process starting from the current layout state.

5. The chip layout method for thermally optimized integrated circuits according to claim 1, characterized in that, The layout state features include: position mask, line length mask, thermal mask, view mask, and fusion mask; The position mask is used to indicate the legal placement area for the current core particle to be placed and the next core particle to be placed; The line length mask is used to represent the half-circumference line length increment caused when the current core particle to be placed or the next core particle to be placed is placed in different candidate positions; The thermal mask is used to represent the increase in thermal interaction potential caused when the current core particle to be placed or the next core particle to be placed is placed at different candidate positions; The view mask is used to represent the overall layout state of the core particles that have been placed. The fusion mask is obtained by linearly combining line length mask and thermal mask according to weights, and is used to uniformly represent wiring information and thermal coupling information.

6. The chip layout method for thermally optimized integrated circuits according to claim 1, characterized in that, The fusion module is used to fuse the local features and the global features to calculate the action probability matrix. The process includes: By fusing the local features and the global features, a logits graph is obtained; An adaptive threshold is constructed based on a guided image containing wiring costs and / or thermally related costs, along with an adjustable soft coefficient. Generate a guidance inhibition mask based on an adaptive threshold; Generate a physical feasibility mask based on the layout legality constraints of the current core particles to be deployed; A soft screening mask is obtained based on the guidance inhibition mask and the physical feasibility mask. The candidate layout actions of the logits graph are soft-screened based on the soft screening mask. During the soft screening process, if there is at least one feasible candidate position in the soft screening mask, the logits graph is masked using the soft screening mask. If the soft screening mask is empty, the logits graph is masked using the physical feasibility mask. Based on the filtered logits graph, the motion probability distribution of the current core particles to be deployed is calculated.

7. The chip layout method for thermally optimized integrated circuits according to claim 4, characterized in that, The loss function of the policy network is: In the formula: Represents the expectation function, This indicates the objective function used by the policy network. For policy entropy, The entropy regularity coefficient; In the formula: This is the cutting factor. It represents the probability ratio of the placement actions on the same core particle in any two adjacent iterations over several iterations; Indicates range clipping, used to... Limited to and between; This represents the normalized advantage value of the particle placement strategy during a certain round of iteration; the unnormalized advantage value is determined based on the state value of the particle placement strategy output by the value network and the Monte Carlo reward calculated based on the reward of each particle placement process in the particle placement strategy.

8. The chip layout method for thermally optimized integrated circuits according to claim 4, characterized in that, The loss function of the value network is: In the formula: Represents the expectation function, This represents the smoothed L1 loss function; This represents the state value calculated by the value network based on a certain round of core particle placement strategy; This represents the Monte Carlo cumulative reward calculated based on the reward for each core placement process in the core placement strategy.

9. The method for laying out chips in a thermally optimized integrated circuit according to any one of claims 1 to 8, characterized in that, During the iteration process of the reinforcement learning model, the PPO method is used to iteratively train the reinforcement learning model.

10. A layout apparatus for chips in an integrated circuit for thermal optimization, characterized in that, include: The modeling module acquires the design information of the integrated circuit and models the chip layout process as a thermally sensitive Markov decision process based on the design information of the integrated circuit. The placement order determination module determines the placement order of the core particles and uses this order as the execution order for subsequent step-by-step layout decisions; The layout module acquires the layout state features of the core particles and constructs a reinforcement learning model. The reinforcement learning model determines the action probability matrix of the current core particle based on the input layout state features. It then places the current core particle according to the action probability matrix and calculates the immediate reward for the placement action of the current core particle. The immediate reward is calculated based on the routing cost and thermal coupling cost. After updating the layout state features based on the placed core particles, the above steps are repeated to place the next core particle. After all core particles in the current round are placed, the next iteration is performed. During the iteration process, the reinforcement learning model is updated every few iterations until the reinforcement learning model converges, completing the core particle layout. The Monte Carlo cumulative reward of the reinforcement learning model is obtained by sampling a complete core particle layout trajectory at the end of each iteration, and then summing the immediate rewards from the current time to the termination time in that layout trajectory according to a discount factor.

Images