Chip layout model training method, chip layout method and related device

Abstract

This application discloses a training method for a chip layout model, a chip layout method, and related apparatus. The method includes: calculating first to third instantaneous rewards in parallel based on the layout state data of the target layout region in the current training cycle; calculating the corresponding temporal difference target and temporal difference error using the first to third instantaneous rewards, and updating the parameters of the density evaluation network, space utilization evaluation network, and position compliance evaluation network in parallel; outputting first to third expected rewards using the parameter-updated density evaluation network, space utilization evaluation network, and position compliance evaluation network; calculating the advantage function based on the first to third expected rewards and the corresponding first to third instantaneous rewards; and updating the parameters of the policy network based on the advantage function. This application improves the learning efficiency, training stability, and final layout quality of reinforcement learning models in complex layout tasks.

Description

Technical Field

[0001] This application relates to the field of semiconductor manufacturing technology, and in particular to a method for training a chip layout model, a chip layout method, and related apparatus. Background Technology

[0002] In the integrated circuit design flow, chip placement is a critical step in the physical design phase, directly impacting chip performance, power consumption, area, and manufacturability. Traditional chip placement methods primarily rely on heuristic algorithms or rule-based manual adjustments. These methods often struggle to find high-quality placement solutions within limited timeframes when facing increasingly complex modern chip designs, especially when dealing with large-scale macrocell placements. They suffer from low optimization efficiency, insufficient placement compactness, and difficulty in accommodating multiple design constraints. In recent years, with the development of deep reinforcement learning technology, it has been gradually applied to chip placement optimization. By constructing Markov decision process models, the placement process is modeled as a sequential decision-making problem of an intelligent agent in an environment, effectively improving the global optimization capability and automation level of placement solutions. However, in multi-project wafer (MPW) placement scenarios, design goals are often highly diverse and even conflicting. For example, improving space utilization may sacrifice placement compactness, while meeting strict physical compliance constraints may limit the space optimization space.

[0003] Against this backdrop, traditional reinforcement learning architectures employing a single Critic network face significant challenges. Because a single Critic network must simultaneously perform joint value estimation on multiple coupled and potentially competitive objectives, it struggles to accurately capture the independent contributions of each objective dimension, leading to large learning biases in the value function and difficulties in convergence. More critically, single Critic networks are prone to drastic fluctuations in value estimation during training: on the one hand, they need to predict future cumulative rewards based on the current state; on the other hand, their own output is used to construct temporal difference (TD) objectives, creating a self-itergencing dependency loop that causes objective value drift and a "target-chasing" phenomenon. This instability during training further propagates to the policy network (Actor), causing inaccurate policy updates, reduced learning efficiency, and even training divergence. Therefore, a novel network architecture and training mechanism that can decouple multi-objective value evaluation and improve training stability is urgently needed to achieve efficient and high-quality optimization of complex chip layout problems. Summary of the Invention

[0004] In view of the above problems, the purpose of this application is to provide a training method for a chip layout model, a chip layout method and related devices, which aims to improve the learning efficiency, training stability and final layout quality of reinforcement learning models in complex layout tasks by constructing a multi-dimensional parallel evaluation network architecture to achieve decoupled modeling and collaborative optimization of multiple optimization objectives in the chip layout process.

[0005] According to a first aspect of the embodiments of this application, a method for training a chip layout model is provided, the chip layout model including a policy network and parallel compactness evaluation networks, space utilization evaluation networks, and location compliance evaluation networks, comprising: Based on the layout state data of the target layout region in the current training cycle, the first instant reward related to layout density, the second instant reward related to space utilization, and the third instant reward related to chip position compliance are calculated in parallel. Using the first instant reward, the second instant reward, and the third instant reward, the corresponding time difference target and time difference error are calculated respectively, and the parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network are updated in parallel. Using the updated density evaluation network, space utilization evaluation network, and location compliance evaluation network, the first expected reward, the second expected reward, and the third expected reward are output respectively. Based on the first expected reward, the second expected reward, the third expected reward, and the corresponding first instant reward, the second instant reward, and the third instant reward, a dominance function is calculated, and the policy network parameters are updated based on the dominance function.

[0006] Optionally, before calculating in parallel the first instant reward related to layout density, the second instant reward related to space utilization, and the third instant reward related to chip location compliance based on the layout state data of the target layout region in the current training cycle, the training method further includes: Based on the layout state data of the target layout region in the previous training cycle, the policy network selects the target chip to be laid out in the target layout region in the current training cycle. The target chip is laid out using a layout engine to obtain the layout position of the target chip in the target layout area, and the layout state data is updated accordingly as the layout state data for the current training cycle.

[0007] Optionally, the step of calculating the advantage function based on the first expected reward, the second expected reward, the third expected reward, and the corresponding first instant reward, the second instant reward, and the third instant reward, and updating the parameters of the policy network based on the advantage function includes: The first expected reward, the second expected reward, and the third expected reward are weighted and summed to obtain the comprehensive expected reward. The first instant reward, the second instant reward, and the third instant reward are weighted and summed to obtain the comprehensive instant reward. The corresponding time difference error is calculated using the comprehensive instant reward and the comprehensive expected reward, and is used as the advantage function; Based on the aforementioned advantage function, the gradient of the policy network parameters is calculated, and an adaptive learning rate optimizer is used to perform a gradient ascent update on the policy network parameters. After the update is completed, the next training cycle begins.

[0008] Optionally, the step of using the first instant reward, the second instant reward, and the third instant reward to calculate the corresponding time difference target and time difference error, respectively, and updating the parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network in parallel includes: For any evaluation network: input the layout state data of the next training cycle into the corresponding evaluation network to obtain the expected reward for the next training cycle; The time difference objective is formed by adding the discount factor to the product of the immediate reward for the current training cycle and the expected reward for the next training cycle. Input the layout state data of the current training cycle into the corresponding evaluation network to obtain the expected reward of the current training cycle. Subtract the expected reward of the current training cycle from the corresponding time difference objective to obtain the time difference error. The gradient of the corresponding evaluation network parameters is calculated using the square of the time difference error as the loss, and gradient descent is performed to update them using an adaptive learning rate optimizer. The density evaluation network, space utilization evaluation network, and location compliance evaluation network each complete the above steps independently and in parallel to achieve synchronous parameter updates. After the update is completed, the next training cycle begins.

[0009] Optionally, the first instant reward for the current training cycle is calculated based on the quotient of the area of ​​the currently laid-out chip and the area of ​​the minimum outer border of the currently laid-out chip. The second immediate reward for the current training cycle is calculated based on the quotient of the area of ​​the currently deployed chip and the area of ​​the target deployment region; The third instant reward for the current training cycle is a score set based on whether the chip layout position meets the preset compliance constraints.

[0010] Optionally, the target chip selected by the policy network for the current training cycle in the target layout region based on the layout state data of the target layout region in the previous training cycle includes: The layout state data from the previous training cycle is input into the policy network, and the policy network outputs the probability that each chip in the remaining chips to be laid out is preferentially selected for layout. Randomly select one chip from the remaining chips to be laid out and determine it as the target chip for the current cycle of the target layout area.

[0011] According to a second aspect of the embodiments of this application, a chip layout method is provided, the chip layout method being used for a chip layout model trained by the training method, comprising: The initial layout state data of the target layout region is input into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out. The layout engine is invoked to lay out the chip to be laid out, and the layout position of the chip to be laid out in the target layout area is obtained. Perform a compliance check on the current layout result, and fine-tune it using a mixed-integer linear programming method if it fails to meet the requirements.

[0012] Optionally, the step of inputting the initial layout state data of the target layout region into a trained policy network, and having the policy network output the probability of each chip in the chips to be laid out being preferentially selected for layout, to determine the layout order of the chips to be laid out, includes: The initial layout state data is input into the policy network, and the policy network outputs the probability that each chip in the chips to be laid out is preferentially selected for layout. The order in which each chip in the chips to be laid out is determined by the probability of it being preferentially selected for placement.

[0013] According to a third aspect of the embodiments of this application, a training apparatus for a chip layout model is provided, the chip layout model including a policy network and parallel compactness evaluation networks, space utilization evaluation networks, and location compliance evaluation networks, comprising: The instant reward calculation unit is used to calculate in parallel the first instant reward related to the layout density, the second instant reward related to the space utilization, and the third instant reward related to the chip position compliance, based on the layout state data of the target layout region in the current training cycle. The evaluation network parameter update unit is used to calculate the corresponding time difference target and time difference error using the first instant reward, the second instant reward and the third instant reward respectively, and update the parameters of the density evaluation network, the space utilization evaluation network and the location compliance evaluation network in parallel. The expected reward calculation unit is used to output the first expected reward, the second expected reward, and the third expected reward respectively using the updated parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network. The policy network parameter update unit is used to calculate the advantage function based on the first expected reward, the second expected reward, the third expected reward and the corresponding first instant reward, the second instant reward and the third instant reward, and update the parameters of the policy network based on the advantage function.

[0014] According to a fourth aspect of the embodiments of this application, a chip layout apparatus is provided, the chip layout apparatus being used for a chip layout model trained according to the training method, comprising: The layout order determination unit is used to input the initial layout state data of the target layout area into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out. The layout position determination unit is used to call the layout engine to lay out the chip to be laid out, and to obtain the layout position of the chip to be laid out in the target layout area. The compliance check unit is used to perform compliance checks on the current layout result and to fine-tune it using a mixed-integer linear programming method when it fails to meet the requirements.

[0015] According to a fifth aspect of the embodiments of this application, an electronic device is provided, comprising: One or more processors; A memory for storing executable instructions that, when executed by the one or more processors, cause the electronic device to perform the method.

[0016] The unexpected technical effect of this application is: Based on the layout state data of the target layout region in the current training cycle, the first immediate reward related to layout density, the second immediate reward related to space utilization, and the third immediate reward related to chip position compliance are calculated in parallel. Using the first, second, and third immediate rewards, the corresponding temporal difference objectives and temporal difference errors are calculated respectively. The parameters of the density evaluation network, space utilization evaluation network, and position compliance evaluation network are updated in parallel. Using the parameter-updated density evaluation network, space utilization evaluation network, and position compliance evaluation network, the first expected reward, second expected reward, and third expected reward are output respectively. Based on the first expected reward, second expected reward, and third expected reward, as well as the corresponding first, second, and third immediate rewards, the advantage function is calculated, and the parameters of the policy network are updated based on the advantage function. In this way, by decomposing multiple layout optimization objectives into independent and parallel evaluation networks, a refined and professional value estimation of key indicators such as density, space utilization, and position compliance is achieved, avoiding the value function distortion problem caused by weight conflicts or gradient interference in the context of multi-objective coupling in traditional single Critic networks. Each evaluation network focuses on a specific optimization dimension, enabling it to more accurately capture the state change trend of the corresponding target, significantly improving the target stability and convergence speed during the temporal difference learning process. Simultaneously, because the evaluation networks update in parallel without interfering with each other, variance fluctuations during training are effectively reduced, alleviating the "target chasing" phenomenon. This allows the policy network to be updated under the guidance of a more reliable advantage function, thereby improving the robustness and directional consistency of the overall policy learning.

[0017] Furthermore, this application constructs a comprehensive advantage function by weightedly fusing the expected rewards and immediate rewards output by each evaluation network. This enables policy updates to not only rely on a single value signal but also to comprehensively consider the relative gains of multiple objective dimensions, thus realizing a dynamic and adaptive multi-objective trade-off mechanism. Attached Figure Description

[0018] The above and other objects, features and advantages of this application will become clearer from the following description of embodiments with reference to the accompanying drawings, in which: Figure 1 The diagram shown is a schematic diagram of the chip layout structure in the target layout area in the related technology; Figure 2 The diagram shown is a schematic frame of an exemplary chip layout model according to an embodiment of this application. Figure 3 The diagram shown is a schematic flowchart of an exemplary chip layout model training method according to an embodiment of this application. Figure 4 The diagram shown is a schematic flowchart of an exemplary chip layout method according to an embodiment of this application. Figure 5 The diagram shown is a schematic representation of an exemplary chip layout model training device according to an embodiment of this application. Figure 6 The diagram shown is a schematic structural diagram of an exemplary chip layout apparatus according to an embodiment of this application; Figure 7 The diagram shown is a schematic diagram of an exemplary chip layout structure in a target layout area according to an embodiment of this application; Figure 8 The diagram shown is a schematic representation of an exemplary electronic device according to an embodiment of this application. Detailed Implementation

[0019] The present application will now be described in more detail with reference to the accompanying drawings. In the various drawings, the same elements are indicated by similar reference numerals. For clarity, the various parts in the drawings are not drawn to scale. Furthermore, some well-known parts may not be shown.

[0020] This application may be presented in various forms, some of which will be described below.

[0021] In the text: Chip layout is the process in integrated circuit (IC) manufacturing where, after the chip design is completed, the circuit patterns corresponding to multiple chips are arranged on a mask.

[0022] MPW (Multi-Project Wafer) is a manufacturing process that integrates multiple different design projects onto the same wafer for fabrication.

[0023] The target layout area refers to the wafer area that can be covered by a single exposure (shot) of the photolithography equipment, and may contain one or more chip layout units. Within this area, several chip units to be processed must be arranged according to process rules, while reserving necessary dicing paths and alignment mark space. In actual layout, the utilization rate of the target layout area directly affects wafer production efficiency and manufacturing costs.

[0024] Figure 1 The diagram shown illustrates the chip layout structure in the target layout area of ​​a related technology. For example... Figure 1 The chip layout shown is a manually placed scheme in a multi-project wafer (MPW) layout scenario. It has low density and irregular spacing, resulting in significant space waste. This layout method cannot meet the industrial demands for high integration and efficient use of mask area. Furthermore, manual layout is inefficient and prone to introducing human error.

[0025] Meanwhile, in multi-project wafer (MPW) layout scenarios, the traditional reinforcement learning architecture using a single criterion network for chip layout optimization also has many limitations.

[0026] Based on this, embodiments of this application provide a training method for a chip layout model, a chip layout method, and related apparatus. By constructing a multi-dimensional parallel evaluation network architecture, decoupled modeling and collaborative optimization of multiple optimization objectives in the chip layout process are achieved, thereby improving the learning efficiency, training stability, and final layout quality of the reinforcement learning model in complex layout tasks.

[0027] Figure 2 The diagram shown is a schematic frame of an exemplary chip layout model according to an embodiment of this application. Figure 2 As shown, the chip layout model 200 includes a strategy network 211, a parallel density evaluation network 221, a space utilization evaluation network 222, and a location compliance evaluation network 223, a layout engine 212, a layout state data update module 213, an evaluation network parameter update module 214, a strategy network parameter update module 215, a first instant reward calculation module 231, a second instant reward calculation module 232, and a third instant reward calculation module 233.

[0028] The compactness evaluation network 221, space utilization evaluation network 222, and location compliance evaluation network 223 correspond to different layout optimization objectives. In some embodiments, the compactness evaluation network 221 is used to evaluate the relative positional relationships between chips in the current layout scheme, especially the spacing distribution and clustering trend between adjacent chips, thereby quantifying the compactness of the layout. The space utilization evaluation network 222 focuses on measuring the actual utilization efficiency of the mask area within the target layout area. By analyzing the distribution of unoccupied blank areas and the overall fill rate, it generates a value prediction of the space utilization level. The location compliance evaluation network 223 is used to determine whether the current layout meets preset physical and process constraints, such as minimum spacing requirements between chips, prohibited layout areas, alignment rules, and electrical isolation conditions.

[0029] During training, the placement engine 212 selects chips to be placed sequentially and places them in appropriate positions within the target placement area based on the action probability distribution output by the policy network 211. After completing a placement action, the placement state data update module 213 updates the placement state information of the target placement area in real time and inputs this placement state data into the first immediate reward calculation module 231, the second immediate reward calculation module 232, and the third immediate reward calculation module 233, respectively. The first immediate reward calculation module 231 calculates the first immediate reward related to placement density based on indicators such as the average spacing between adjacent chips and the maximum gap area. The second immediate reward calculation module 232 calculates the second immediate reward related to space utilization based on the change in the ratio of the currently utilized area to the total available area, combined with the connectivity and shape complexity of the void area. The third immediate reward calculation module 233 detects whether the current placement violates preset design rules, imposes penalties on each violation, and generates a third immediate reward related to position compliance. These immediate reward signals not only guide the learning direction of the policy network but also provide independent temporal difference learning objectives for each parallel evaluation network, thereby achieving refined modeling and stable training of multi-objective value estimation.

[0030] In some embodiments, the chip layout model further includes a compliance check module 240. After the chip layout model is trained, the policy network 211 outputs the placement order sequence of each chip based on the input set of chips to be laid out and the current layout state, driving the layout engine 212 to generate the corresponding placement position sequence. The compliance check module 240 performs a final verification of the output layout result to ensure that all chips meet the process rules such as minimum spacing, rotation constraints, and area restrictions.

[0031] Since the specific process of model training will be described in detail below, it will not be repeated here.

[0032] Figure 3 The diagram shown is a schematic flowchart of an exemplary chip layout model training method according to an embodiment of this application. The following is in conjunction with... Figure 2 This application provides a detailed description of the training method for the chip layout model according to embodiments of the present application. For example... Figure 3 As shown, the training method in this application embodiment includes: In step S310, based on the layout state data of the target layout region in the current training cycle, a first instant reward related to layout density, a second instant reward related to space utilization, and a third instant reward related to chip location compliance are calculated in parallel.

[0033] In some embodiments, before step S310, based on the layout state data S(i-1) of the target layout region in the previous training cycle (i.e., the (i-1)th training cycle, where i is an integer greater than 0), the policy network 211 selects the target chip BlockID(i) to be laid out in the target layout region in the current cycle. Specifically, in some embodiments, the layout state data S(i-1) of the previous training cycle is input into the policy network 211, and the policy network 211 outputs the probability of each chip in the remaining chips to be laid out being preferentially selected for layout, and randomly selects a chip from the remaining chips to be laid out to determine the target chip BlockID(i) to be laid out in the target layout region in the current cycle. After obtaining the decision action of the policy network 211, the layout engine 212 calculates the optimal placement position and orientation of the target chip BlockID(i) according to the geometric features of the target chip BlockID(i) and the current layout space, and generates the layout position of the target chip BlockID(i) in the target layout region. The layout engine feeds back the layout position of the target chip BlockID(i) to the layout state data update module 213. The layout state data update module 213 updates the layout state data S(i) of the current training cycle (i.e., the i-th training cycle) and uses the layout state data S(i) of the current training cycle to calculate the three immediate reward values ​​in step S310. It should be noted that the layout state data S(i) of the current training cycle includes at least: the current list of chips to be laid out and the physical constraints of the target layout region. The physical constraints of the target layout region include the size of the target layout region, the direct constraints of the chips to be laid out (e.g., the minimum distance between chips, the minimum horizontal distance between chips, the minimum vertical distance between chips, the layout condition information of the chips (e.g., priority information and rotation constraint information)), and the positional region constraints of the chip center point coordinates. The current list of chips to be laid out records the chip identifier (ID) and size, as well as the layout position information of the already laid-out chips. It should be noted that during the model testing phase, the policy network 211 can output the probability of each chip in the remaining chips to be laid out being preferentially selected for placement. The chip with the highest probability is selected from the remaining chips to be laid out and determined as the target chip to be laid out in the target placement area in the current cycle.

[0034] In some embodiments, the first instant reward calculation module 231, the second instant reward calculation module 232, and the third instant reward calculation module 233 calculate, respectively, a first instant reward O1(i) related to layout density, a second instant reward O2(i) related to space utilization, and a third instant reward O3(i) related to chip position compliance based on the updated layout state data S(i) of the current training cycle. In some embodiments, the first instant reward O1(i) of the current training cycle is calculated based on the quotient of the area of ​​the currently laid-out chip and the area of ​​the minimum outer border of the currently laid-out chip. The second instant reward O2(i) of the current training cycle is calculated based on the quotient of the area of ​​the currently laid-out chip and the area of ​​the target layout region. The third instant reward O3(i) of the current training cycle is a score set according to whether the chip layout position meets the preset compliance constraints.

[0035] In step S320, the first instant reward, the second instant reward, and the third instant reward are used to calculate the corresponding time difference target and time difference error, respectively, and the parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network are updated in parallel.

[0036] In some embodiments, for any evaluation network: the evaluation network parameter update module 214 inputs the layout state data S(i+1) of the next training cycle into the corresponding evaluation network to obtain the expected reward of the next training cycle; the corresponding immediate reward of the current training cycle is added to the product of the discount factor γ and the expected reward of the next training cycle to form the temporal difference objective (also known as the TD objective); the layout state data S(i) of the current training cycle is input into the corresponding evaluation network to obtain the expected reward of the current training cycle; the expected reward of the current training cycle is subtracted from the corresponding temporal difference objective to obtain the temporal difference error (also known as the TD error); the gradient of the corresponding evaluation network parameters is calculated using the square of the temporal difference error as the loss, and gradient descent is performed using an adaptive learning rate optimizer.

[0037] In other words, for the density evaluation network 221, the evaluation network parameter update module 214 inputs the layout state data S(i+1) of the next training cycle into the density evaluation network 221 to obtain the expected density reward value of the next training cycle. This reward value is then combined with the first immediate reward O1(i) of the current training cycle, plus the product of the discount factor and the expected density reward of the next training cycle, to form the temporal difference objective in the density dimension. Simultaneously, the layout state data S(i) of the current training cycle is input into the density evaluation network 221 to obtain the expected density reward value of the current training cycle. The difference between this expected reward value and the temporal difference objective in the density dimension is calculated to obtain the temporal difference error in the density dimension. The square of this error is used as the loss function to calculate the gradient of the density evaluation network parameters, and an adaptive learning rate optimizer is used for gradient descent updates.

[0038] Similarly, for the space utilization evaluation network 222, the evaluation network parameter update module 214 inputs the layout state data S(i+1) of the next training cycle into the space utilization evaluation network 222 to obtain the expected reward value of space utilization in the next training cycle. It then combines the second immediate reward O2(i) of the current training cycle with the product of the discount factor and the expected reward of the next training cycle to form the time difference target in the space utilization dimension. At the same time, it inputs the layout state data S(i) of the current training cycle into the network to obtain the current expected reward value, calculates the difference between it and the time difference target, obtains the time difference error in the space utilization dimension, uses its square as the loss function, calculates the gradient of the network parameters, and uses an adaptive learning rate optimizer to update the parameters.

[0039] Similarly, for the location compliance evaluation network 223, the evaluation network parameter update module 214 inputs the layout state data S(i+1) of the next training cycle into the location compliance evaluation network 223 to obtain the expected reward value of location compliance in the next training cycle. It then combines the product of the third immediate reward O3(i) of the current training cycle, the discount factor, and the expected reward of the next training cycle to form the temporal difference target of the location compliance dimension. At the same time, it inputs the layout state data S(i) of the current training cycle into the network to obtain the current expected reward value, calculates the difference between it and the temporal difference target, obtains the temporal difference error of the location compliance dimension, uses its square as the loss function, calculates the gradient of the network parameters, and uses an adaptive learning rate optimizer to update the parameters.

[0040] It should be noted that the three evaluation networks complete the above steps independently and in parallel, achieving synchronous parameter updates, and then proceed to the next training cycle after the updates are completed. The three evaluation networks independently calculate the temporal difference error and update their parameters on the dimensions of compactness, spatial utilization, and positional compliance, ensuring effective decomposition and accurate feedback of multi-objective reward signals. In each training cycle, the three evaluation networks independently calculate gradients and update their parameters, avoiding the bias problem of a single value network modeling multi-objective coupling. Simultaneously, by sharing layout state data input, they achieve co-evolution of feature responses, continuously enhancing the sensitivity of each evaluation network to specific optimization dimensions.

[0041] In step S330, the density evaluation network, space utilization evaluation network, and location compliance evaluation network with updated parameters are used to output the first expected reward, the second expected reward, and the third expected reward, respectively.

[0042] In some embodiments, the layout state data S(i) of the current training cycle is input into the layout state data update module 213, the evaluation network parameter update module 214, and the policy network parameter update module 215 after parameter updates, respectively. The layout state data update module 213, the evaluation network parameter update module 214, and the policy network parameter update module 215 output the first expected reward U1(i), the second expected reward U2(i), and the third expected reward U3(i) of the current training cycle, respectively.

[0043] In step S340, an advantage function is calculated based on the first expected reward, the second expected reward, the third expected reward, and the corresponding first instant reward, the second instant reward, and the third instant reward, and the parameters of the policy network are updated based on the advantage function.

[0044] In some embodiments, the policy network parameter update module 215 weights and sums the first expected reward U1(i), the second expected reward U2(i), and the third expected reward U3(i) to obtain the comprehensive expected reward value U(i). For example, U(i) = α1×U1(i) + β1×U2(i) + γ1×U3(i), where α1, β1, and γ1 are corresponding weight coefficients used to balance the contributions of different objectives. Simultaneously, the first immediate reward O1(i), the second immediate reward O2(i), and the third immediate reward O3(i) of the current period are weighted and summed to obtain the comprehensive immediate reward O(i). For example, O(i) = α2×O1(i) + β2×O2(i) + γ2×O3(i), where α2, β2, and γ2 are corresponding weight coefficients. Then, the comprehensive immediate reward O(i) and the comprehensive expected reward U(i) are used to calculate the corresponding time difference error, which is used as the advantage function A(i). The advantage function A(i) is used to measure the performance of the current policy under a specific state. In some embodiments, the calculation process of the advantage function A(i) is as follows: The comprehensive immediate reward O(i) obtained in the current training cycle is added to the comprehensive expected reward U(i+1) obtained in the next training cycle after being weighted by the discount factor γ to obtain the time difference target value. The time difference target value is then subtracted from the comprehensive expected reward U(i) obtained in the current training cycle, and the difference is the time difference error. The calculated time difference error is the value of the advantage function A(i). Based on the advantage function, the gradient of the policy network parameters is calculated, and an adaptive learning rate optimizer is used to perform a gradient ascent update on the policy network parameters. After the update is completed, the next training cycle begins. It should be noted that this advantage function guides the policy network to update parameters by maximizing positive advantages, thereby enhancing its ability to generate better layout schemes, while suppressing the action selection corresponding to negative advantages. This process relies on the accurate expected reward signals provided by three independent evaluation networks to ensure that the contributions of each objective dimension are clearly separated and updated stably, achieving efficient convergence in multi-objective collaborative optimization. Through continuous iteration, the policy network gradually learns to prioritize assembly actions with high spatial utilization and strong positional fit under complex constraints. Three evaluation networks focus on the dimensions of compactness, spatial utilization, and positional constraints, respectively, providing stable and decorrelational value estimates, effectively mitigating the training bias problem caused by a single evaluation function.

[0045] Figure 4 The diagram shown is a schematic flowchart of an exemplary chip layout method according to an embodiment of this application. The chip layout method of this embodiment is applied to the chip layout model trained by the above-described training method. Figure 4 As shown, the chip layout method of this application embodiment includes: In step S410, the initial layout state data of the target layout area is input into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out.

[0046] In some embodiments, initial layout state data S(0) of the target layout area is obtained. The initial layout state data S(0) includes the current list of chips to be laid out and the physical constraints of the target layout area. The physical constraints of the target layout area include the size of the target layout area, direct constraints on the chips to be laid out (e.g., minimum distance between chips, minimum horizontal distance between chips, minimum vertical distance between chips, chip layout condition information (e.g., priority information and rotation constraint information)), and positional constraints on the chip center point coordinates. The current list of chips to be laid out records the chip identifiers (IDs) and sizes, as well as the layout position information of the already laid-out chips.

[0047] In some embodiments, the initial layout state data S(0) is input into the policy network 211, and the policy network 211 outputs the probability of each chip in the chip to be laid out being preferentially selected for layout. The order of the probability of each chip in the chip to be laid out being preferentially selected for layout is used as the layout order of each chip in the chip to be laid out.

[0048] In step S420, the layout engine is invoked to lay out the chip to be laid out, thereby obtaining the layout position of the chip to be laid out in the target layout area.

[0049] In some embodiments, the layout engine 212 is invoked to lay out the chips to be laid out one by one. Combining the physical constraints of the target layout area and the position information of the already laid-out chips, the optimal layout coordinates of each chip to be laid out in the candidate position set are calculated to obtain the layout position of the chip to be laid out in the target layout area.

[0050] In step S430, a compliance check is performed on the current layout result, and fine-tuning is performed using a mixed-integer linear programming method if it fails to meet the requirements.

[0051] In some embodiments, the compliance check items of the compliance check module 240 include fixed constraints and user-defined constraints. Fixed constraints include overlap checks and whether boundary limits are exceeded. User-defined constraints include special chip block position limits, minimum distance limits, main chip placement positions, and dicing track size limits. In some embodiments, after obtaining the initial solution of the placement position of the chip to be placed in the target placement area, the compliance check module 240 can be invoked to perform a compliance check on the current placement result. If it passes the check, the current placement result can be output to draw a GDSII diagram. It should be noted that a GDSII diagram is a binary file format used for integrated circuit design, carrying the geometric and hierarchical information of the chip layout, and is widely used in the tape-out manufacturing process. After the current placement result passes the compliance check, the system automatically generates a GDSII output file that meets the process requirements.

[0052] In some embodiments, when the compliance check of the current layout result fails, the compliance check module 240 uses a mixed-integer linear programming method to fine-tune the current layout result. If the current layout result passes the fine-tuning, the globally optimal solution for the layout position of the chip to be laid out in the target layout region is obtained, and the current layout result can be output to draw a GDSII diagram. In some embodiments, the mixed-integer linear programming method may include introducing a penalty term into the fine-tuning objective function, adding a large penalty to the part that violates the constraints to guide the compliance check module 240 to adjust it to the feasible region, ensuring that the final layout meets all physical and process constraints. It should be noted that the internal parameters of the compliance check module 240 can be automatically modified according to specific rule conditions input by the user, thereby dynamically adjusting the constraint weights and penalty coefficients. By dynamically adjusting the constraint weights and penalty coefficients, it is possible to quickly converge to a layout scheme that meets all rule requirements under complex constraint conditions, improving the flexibility and adaptability of the layout scheme.

[0053] In some embodiments, minimizing container height and maximizing space utilization are among the core optimization objectives in the fine-tuning process. By taking into account the height and width of the target layout area, the height and width of the chip block to be laid out, and the dicing constraints, the penalty coefficient in the objective function is optimized. Simultaneously, using the horizontal and vertical coordinates of the chip to be laid out within the target layout area as decision variables, the positions of the main chip and special function blocks are fine-tuned to ensure that their coordinates fall within the allowable range and strictly adhere to boundary conditions, ultimately achieving a high-density, conflict-free chip layout scheme that conforms to manufacturing specifications.

[0054] Figure 5 The diagram shown is a schematic representation of an exemplary chip layout model training device according to an embodiment of this application. In this embodiment, the chip layout model includes a policy network and parallel denseness evaluation networks, space utilization evaluation networks, and location compliance evaluation networks. Figure 5 As shown, the training device 500 includes an instant reward calculation unit 510, an evaluation network parameter update unit 520, an expected reward calculation unit 530, and a policy network parameter update unit 540.

[0055] The instant reward calculation unit 510 is used to calculate in parallel a first instant reward related to layout density, a second instant reward related to space utilization, and a third instant reward related to chip position compliance, based on the layout state data of the target layout region in the current training cycle.

[0056] The evaluation network parameter update unit 520 is used to calculate the corresponding time difference target and time difference error using the first instant reward, the second instant reward and the third instant reward respectively, and update the parameters of the density evaluation network, the space utilization evaluation network and the location compliance evaluation network in parallel.

[0057] The expected reward calculation unit 530 is used to output the first expected reward, the second expected reward, and the third expected reward respectively using the updated parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network.

[0058] The policy network parameter update unit 540 is used to calculate the advantage function based on the first expected reward, the second expected reward, the third expected reward and the corresponding first instant reward, the second instant reward and the third instant reward, and update the parameters of the policy network based on the advantage function.

[0059] Since the specific process of model training has been described in detail above, it will not be repeated here.

[0060] Figure 6 The diagram shown is a schematic representation of an exemplary chip layout apparatus according to an embodiment of this application. The chip layout apparatus 600 of this embodiment is used for the chip layout model trained by the training method. Figure 6 As shown, the chip layout apparatus 600 includes a layout order determination unit 610, a layout position determination unit 620, and a compliance check unit 630.

[0061] The layout order determination unit 610 is used to input the initial layout state data of the target layout area into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out.

[0062] The layout position determination unit 620 is used to call the layout engine to lay out the chip to be laid out, and obtain the layout position of the chip to be laid out in the target layout area.

[0063] The compliance check unit 630 is used to perform compliance checks on the current layout result and to fine-tune it using a mixed-integer linear programming method when it fails to meet the requirements.

[0064] Since the specific process of chip layout has been described in detail above, it will not be repeated here.

[0065] Figure 7 The diagram shown is a schematic representation of an exemplary chip layout structure in a target layout area according to an embodiment of this application. Figure 7 As shown, Figure 7 As shown, multiple chip cells are arranged in an orderly manner within the target layout area. This layout structure is automatically generated by the chip layout model proposed in this application in a multi-project wafer (MPW) scenario, fully demonstrating the model's comprehensive optimization capabilities for layout density, space utilization efficiency, and geometric compliance while meeting physical design constraints. Compared to Figure 1 The irregular gaps and low utilization rate of the manual layout scheme in China are problems. Figure 7 The layout results shown significantly improve the utilization efficiency of the mask area, achieving a balance between higher integration and better electrical performance, and verifying the effectiveness and advancement of the method in practical application scenarios.

[0066] This disclosure also provides an electronic device 800, such as... Figure 8 As shown, it includes a memory 820, a processor 810, a power supply component 830, a network interface 840, an input / output interface 850, and a program stored in the memory 820 and executable on the processor 810. When the program is executed by the processor 810, it can implement the various processes of the embodiments of the above methods and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0067] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, this disclosure also provides a storage medium storing a computer program or instructions that, when executed by a processor, can implement the various processes of the embodiments of the above methods.

[0068] Since the instructions stored in the storage medium can execute the steps of the method provided in the embodiments of this disclosure, the beneficial effects achievable by the method provided in the embodiments of this disclosure can be realized, as detailed in the preceding embodiments, and will not be repeated here. Specific implementations of the above operations can be found in the preceding embodiments, and will not be repeated here.

[0069] Finally, it should be noted that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The embodiments described above, as per the implementation of this application, do not exhaustively describe all details, nor do they limit the application to only the specific embodiments described. Clearly, many modifications and variations can be made based on the above description. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to make good use of this application and modifications based on it. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training a chip layout model, wherein the chip layout model includes a policy network and parallel compactness evaluation networks, space utilization evaluation networks, and location compliance evaluation networks, comprising: Based on the layout state data of the target layout region in the current training cycle, the first instant reward related to layout density, the second instant reward related to space utilization, and the third instant reward related to chip position compliance are calculated in parallel. Using the first instant reward, the second instant reward, and the third instant reward, the corresponding time difference target and time difference error are calculated respectively, and the parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network are updated in parallel. Using the updated density evaluation network, space utilization evaluation network, and location compliance evaluation network, the first expected reward, the second expected reward, and the third expected reward are output respectively. Based on the first expected reward, the second expected reward, the third expected reward, and the corresponding first instant reward, the second instant reward, and the third instant reward, a dominance function is calculated, and the policy network parameters are updated based on the dominance function.

2. The training method according to claim 1, wherein, Before the parallel calculation of the first instant reward related to layout density, the second instant reward related to space utilization, and the third instant reward related to chip location compliance based on the layout state data of the target layout region in the current training cycle, the training method further includes: Based on the layout state data of the target layout region in the previous training cycle, the policy network selects the target chip to be laid out in the target layout region in the current training cycle. The target chip is laid out using a layout engine to obtain the layout position of the target chip in the target layout area, and the layout state data is updated accordingly as the layout state data for the current training cycle.

3. The training method according to claim 1, wherein, The step of calculating the advantage function based on the first expected reward, the second expected reward, the third expected reward, and the corresponding first instant reward, the second instant reward, and the third instant reward, and updating the parameters of the policy network based on the advantage function includes: The first expected reward, the second expected reward, and the third expected reward are weighted and summed to obtain the comprehensive expected reward. The first instant reward, the second instant reward, and the third instant reward are weighted and summed to obtain the comprehensive instant reward. The corresponding time difference error is calculated using the comprehensive instant reward and the comprehensive expected reward, and is used as the advantage function; Based on the aforementioned advantage function, the gradient of the policy network parameters is calculated, and an adaptive learning rate optimizer is used to perform a gradient ascent update on the policy network parameters. After the update is completed, the next training cycle begins.

4. The training method according to claim 1, wherein, The step of using the first, second, and third instant rewards to calculate the corresponding time difference targets and time difference errors, and updating the parameters of the density evaluation network, space utilization evaluation network, and location compliance evaluation network in parallel includes: For any evaluation network: input the layout state data of the next training cycle into the corresponding evaluation network to obtain the expected reward for the next training cycle; The time difference objective is formed by adding the discount factor to the product of the immediate reward for the current training cycle and the expected reward for the next training cycle. Input the layout state data of the current training cycle into the corresponding evaluation network to obtain the expected reward of the current training cycle. Subtract the expected reward of the current training cycle from the corresponding time difference objective to obtain the time difference error. The gradient of the corresponding evaluation network parameters is calculated using the square of the time difference error as the loss, and gradient descent is performed to update them using an adaptive learning rate optimizer. The density evaluation network, space utilization evaluation network, and location compliance evaluation network each complete the above steps independently and in parallel to achieve synchronous parameter updates. After the update is completed, the next training cycle begins.

5. The training method according to claim 1, wherein, The first immediate reward for the current training cycle is calculated based on the quotient of the area of ​​the currently deployed chip and the area of ​​the minimum outer border of the currently deployed chip. The second immediate reward for the current training cycle is calculated based on the quotient of the area of ​​the currently laid-out chip and the area of ​​the target layout region; The third instant reward for the current training cycle is a score set based on whether the chip layout position meets the preset compliance constraints.

6. The training method according to claim 2, wherein, The target chip selected by the policy network for the current training cycle in the target layout region based on the layout state data of the previous training cycle of the target layout region includes: The layout state data from the previous training cycle is input into the policy network, and the policy network outputs the probability that each chip in the remaining chips to be laid out is preferentially selected for layout. Randomly select one chip from the remaining chips to be laid out and determine it as the target chip for the current cycle of the target layout area.

7. A chip layout method, the chip layout method being used for a chip layout model trained according to the training method of any one of claims 1 to 6, comprising: The initial layout state data of the target layout region is input into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out. The layout engine is invoked to lay out the chip to be laid out, and the layout position of the chip to be laid out in the target layout area is obtained. Perform a compliance check on the current layout result, and fine-tune it using a mixed-integer linear programming method if it fails to meet the requirements.

8. The chip layout method according to claim 7, wherein, The step of inputting the initial layout state data of the target layout region into a trained policy network, and having the policy network output the probability of each chip in the chips to be laid out being preferentially selected for layout, to determine the layout order of the chips to be laid out, includes: The initial layout state data is input into the policy network, and the policy network outputs the probability that each chip in the chips to be laid out is preferentially selected for layout. The order in which each chip in the chips to be laid out is determined by the probability of it being preferentially selected for placement.

9. A training apparatus for a chip layout model, the chip layout model comprising a policy network and parallel compactness evaluation networks, space utilization evaluation networks, and location compliance evaluation networks, comprising: The instant reward calculation unit is used to calculate in parallel the first instant reward related to the layout density, the second instant reward related to the space utilization, and the third instant reward related to the chip position compliance, based on the layout state data of the target layout region in the current training cycle. The evaluation network parameter update unit is used to calculate the corresponding time difference target and time difference error using the first instant reward, the second instant reward and the third instant reward respectively, and update the parameters of the density evaluation network, the space utilization evaluation network and the location compliance evaluation network in parallel. The expected reward calculation unit is used to output the first expected reward, the second expected reward, and the third expected reward respectively using the updated parameters of the density evaluation network, the space utilization evaluation network, and the location compliance evaluation network. The policy network parameter update unit is used to calculate the advantage function based on the first expected reward, the second expected reward, the third expected reward and the corresponding first instant reward, the second instant reward and the third instant reward, and update the parameters of the policy network based on the advantage function.

10. A chip placement apparatus, the chip placement apparatus being used for a chip placement model trained according to the training method of any one of claims 1 to 6, comprising: The layout order determination unit is used to input the initial layout state data of the target layout area into the trained policy network, and the policy network outputs the probability of each chip in the chips to be laid out being preferentially selected for layout, so as to determine the layout order of the chips to be laid out. The layout position determination unit is used to call the layout engine to lay out the chip to be laid out, and to obtain the layout position of the chip to be laid out in the target layout area. The compliance check unit is used to perform compliance checks on the current layout result and to fine-tune it using a mixed-integer linear programming method when it fails to meet the requirements.

11. An electronic device, comprising: One or more processors; A memory for storing executable instructions, which, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1-6.

Images