Co-evolution method and apparatus for integrated circuit macro module placement

Abstract

The application discloses a co-evolution method and device for integrated circuit macro module layout, comprising: initializing a population composed of multiple strategy agents and setting initial strategy parameters; interacting each strategy agent with the macro module layout environment of the integrated circuit, generating a placement trajectory and obtaining a layout quality evaluation value; storing all placement trajectories in a shared experience pool; after each preset number of interaction and storage processes, determining whether the current iteration meets a global search triggering condition; if yes, performing a global search operation to generate an updated population; if not, performing a local strategy optimization update; repeating the above process until a preset termination condition is reached, and outputting the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macro module layout result. The application can efficiently complete layout iteration optimization and improve the integrated circuit macro module layout optimization efficiency and quality.

Description

Technical Field

[0001] This application relates to the field of integrated circuit design technology, specifically to a collaborative evolution method and apparatus for the layout of integrated circuit macro modules. Background Technology

[0002] Integrated circuit macro-module placement is a core aspect of chip physical design. It is a very large-scale combinatorial optimization problem with an extremely large placement design space. It directly determines key performance characteristics such as chip line length, power consumption, and area, and is crucial to chip design efficiency and finished product quality.

[0003] Currently, existing integrated circuit macro-module placement mainly employs two types of methods: optimization-based and reinforcement learning-based. Optimization-based methods typically formalize the placement problem as a constrained optimization problem, iteratively exploring and relaxing the design space of hard constraints to find feasible solutions. Reinforcement learning-based methods model the macro-module placement process as a Markov decision process, determining the position of each macro-module through the sequential decisions of an agent, and have the potential to achieve fully automated placement. However, optimization-based methods rely on complex post-processing steps to satisfy the non-overlapping constraints of macro-modules and struggle to effectively utilize historical placement experience, resulting in significant limitations in placement quality and computational efficiency. Reinforcement learning-based methods require extensive environmental interaction for policy exploration, leading to low sample utilization efficiency, premature convergence to local optima, and an inability to fully traverse the vast placement search space. Therefore, existing technologies suffer from insufficient global search capabilities and low sample utilization efficiency, making it difficult to efficiently obtain high-quality integrated circuit macro-module placement schemes.

[0004] The preceding description is intended to provide general background information and does not necessarily constitute prior art. Summary of the Invention

[0005] This application provides a collaborative evolution method and apparatus for integrated circuit macro module layout, which can efficiently complete layout iteration optimization and improve the efficiency and quality of integrated circuit macro module layout optimization.

[0006] In a first aspect, embodiments of this application provide a co-evolution method for integrated circuit macro-module layout, comprising: Initialize a population consisting of multiple policy agents and set initial policy parameters for each policy agent; Each policy agent interacts with the macro-module layout environment of the integrated circuit, and each macro-module is placed on the chip layout in sequence to generate a complete placement trajectory and obtain the layout quality evaluation value corresponding to each placement trajectory. Store the placement trajectories generated by all policy agents in a shared experience pool; After each preset number of interactions and storage processes, determine whether the current iteration meets the global search trigger condition; If the global search triggering condition is met, at least one elite policy agent is selected from the current population, and the policy parameters of the elite policy agent are used to perform a crossover operation on the policy parameters of other policy agents to generate an updated population. If the global search triggering condition is not met, each policy agent samples the placement trajectory from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectory. Repeat the interaction, storage, and optimization update process until the preset termination condition is met, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module.

[0007] Secondly, embodiments of this application provide a collaborative evolution device for integrated circuit macro module layout, comprising: The initialization module is used to initialize a population consisting of multiple policy agents and set initial policy parameters for each policy agent. The interactive evaluation module is used to interact with each policy agent and the macro module layout environment of the integrated circuit, sequentially place each macro module on the chip layout, generate a complete placement trajectory, and obtain the layout quality evaluation value corresponding to each placement trajectory. The storage module is used to store the placement trajectories generated by all policy agents into a shared experience pool; The judgment module is used to determine whether the current iteration meets the global search trigger condition after each preset number of interactions and storage processes. The global search module is used to select at least one elite policy agent from the current population if the global search triggering condition is met, and to perform a cross operation on the policy parameters of other policy agents using the policy parameters of the elite policy agent to generate an updated population. The local optimization module is used to sample placement trajectories from the shared experience pool by each policy agent if the global search triggering condition is not met, and to perform local policy optimization updates on its own policy parameters based on the sampled placement trajectories. The output module is used to repeatedly execute the interaction, storage, and optimization update process until the preset termination condition is reached, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module.

[0008] This application provides a co-evolutionary method and apparatus for integrated circuit macro-module placement. First, a population of multiple policy agents is initialized, allowing each agent to independently interact with the integrated circuit macro-module placement environment and generate complete placement trajectories. This enables parallel placement exploration along multiple paths, broadening the search coverage of placement schemes. Then, the placement trajectories generated by all agents are uniformly stored in a shared experience pool, achieving centralized collection and cross-agent sharing of placement experience, providing sufficient data support for subsequent optimization. Next, during the iteration process, two update modes—global cross-update and local policy optimization—are flexibly switched according to preset conditions. When the conditions are met, the population is cross-updated using elite agents to expand the search boundary; when the conditions are not met, local optimization is performed through experience sampling to improve placement accuracy, effectively balancing the breadth of placement exploration and the depth of optimization. Through multiple rounds of interaction, storage, and update iterations, placement schemes are continuously screened and iteratively optimized, ultimately outputting the integrated circuit macro-module placement result with the optimal placement quality evaluation value. This improves placement optimization efficiency while ensuring the quality and reliability of the final placement scheme. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is an application environment diagram of the co-evolution method for integrated circuit macro module layout provided in the embodiments of this application; Figure 2 This is a flowchart illustrating the collaborative evolution method for integrated circuit macro module layout provided in an embodiment of this application; Figure 3 This is a schematic diagram of the issue generation process provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the co-evolution device for integrated circuit macro module layout provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0011] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with those detailed in the appended claims or with some aspects of this application.

[0012] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover descriptions such as non-exclusive inclusion, so that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0013] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0014] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0015] To address the aforementioned technical problems and overcome the shortcomings of existing technologies, this application provides a collaborative evolution method and apparatus for integrated circuit macro module layout, which can efficiently complete layout iteration optimization and improve the efficiency and quality of integrated circuit macro module layout optimization.

[0016] Figure 1 This is an application environment diagram of a co-evolutionary method for integrated circuit macromodule layout in one embodiment. (Refer to...) Figure 1This co-evolutionary method for integrated circuit macro-module layout is applied to a co-evolutionary system for integrated circuit macro-module layout. The co-evolutionary system for integrated circuit macro-module layout includes a terminal 110 and a server 120. The terminal 110 and server 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal, and the mobile terminal can be at least one of a mobile phone, tablet computer, or laptop computer. The server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers. Server 120 is configured to execute the aforementioned co-evolutionary method for integrated circuit macro-module layout, including: initializing a population composed of multiple policy agents and setting initial policy parameters for each policy agent; interacting with the macro-module layout environment of the integrated circuit, placing each macro-module sequentially on the chip layout, generating complete placement trajectories, and obtaining the layout quality evaluation value corresponding to each placement trajectory; storing the placement trajectories generated by all policy agents in a shared experience pool; after each preset number of interactions and storage processes, determining whether the current iteration meets the global search trigger condition; if the global search trigger condition is met, selecting at least one elite policy agent from the current population, using the policy parameters of the elite policy agent to perform cross operations on the policy parameters of other policy agents, generating an updated population; if the global search trigger condition is not met, sampling placement trajectories from the shared experience pool through each policy agent, and performing local policy optimization updates on its own policy parameters based on the sampled placement trajectories; repeating the interaction, storage, and optimization update process until a preset termination condition is reached, and outputting the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macro-module layout result.

[0017] Please see Figure 2 , Figure 2 This is a flowchart illustrating a co-evolution method for integrated circuit macro-module layout according to an embodiment of this application. This embodiment primarily uses the application of this co-evolution method for integrated circuit macro-module layout to a computer device as an example for illustration. Specifically, the co-evolution method for integrated circuit macro-module layout provided in this embodiment may include the following steps: To facilitate understanding of the co-evolution method for integrated circuit macro-module layout provided in this embodiment, this embodiment also provides another implementation of the co-evolution method for integrated circuit macro-module layout, including the following steps: S1. Initialize the population consisting of multiple policy agents and set initial policy parameters for each policy agent; Specifically, for step S1, firstly, based on the layout optimization requirements, determine the number of policy agents and form a population consisting of multiple independent policy agents. Each policy agent in the population is a unit that can independently execute layout decisions. Then, configure initial policy parameters for each policy agent in the population. The initial policy parameters are the core basis for the agent to make subsequent macro-module placement decisions. After completing the setting of the population and initial parameters, the basic conditions for starting multi-agent collaborative layout exploration are met.

[0018] S2. Interact with the macro-module layout environment of each policy agent and the integrated circuit, place each macro-module on the chip layout in sequence, generate a complete placement trajectory, and obtain the layout quality evaluation value corresponding to each placement trajectory. Specifically, in step S2, after population initialization, each policy agent independently interacts with the integrated circuit macro-module placement environment. This environment includes the basic space of the chip layout, macro-module placement rules, and other constraints. Each agent, according to its own policy parameters, sequentially places all macro-modules to be placed onto the chip layout, from the first macro-module to the last, forming a continuous and complete placement trajectory. For each independently generated placement trajectory, a corresponding placement quality evaluation value is calculated and obtained according to a preset standard. This value is used to intuitively quantify the quality of the placement scheme corresponding to that trajectory. For example, if an agent sequentially places six macro-modules on the chip, this continuous placement process forms a complete placement trajectory, and the placement quality score of that trajectory is obtained through the evaluation standard.

[0019] S3. Store the placement trajectories generated by all policy agents into the shared experience pool; Specifically, for step S3, after all policy agents complete the layout interaction, generate placement trajectories, and obtain evaluation values, the complete placement trajectories generated by all agents in the population are uniformly stored in a pre-built shared experience pool. The shared experience pool serves as a unified storage carrier for layout trajectories, centrally collecting the layout exploration results of all agents, and providing jointly accessible experience data for subsequent policy optimization by each agent.

[0020] S4. After each preset number of interactions and storage processes, determine whether the current iteration meets the global search trigger condition; Specifically, for step S4, a global search trigger cycle is pre-set. This means that after a fixed number of agent-environment interactions and trajectory storage processes are completed, a condition judgment phase begins. Based on preset trigger rules, it is determined whether the current iteration meets the requirements for executing a global search. This judgment determines whether to use a global search update or a local strategy optimization update, enabling orderly switching between the two update methods. For example, a judgment is preset to be triggered after every four interaction storage processes. Once four processes are completed, it is immediately determined whether the global search conditions are met.

[0021] S5. If the global search triggering condition is met, select at least one elite policy agent from the current population, and use the policy parameters of the elite policy agent to perform a crossover operation on the policy parameters of other policy agents to generate an updated population. Specifically, for step S5, if the current iteration meets the global search triggering condition, firstly, based on the layout quality evaluation value corresponding to each agent, at least one elite policy agent with the best layout performance is selected from the current population; then, based on the policy parameters of this elite agent, a cross operation is performed on the policy parameters of other non-elite agents in the population, and the iterative upgrade of the agent policies in the population is achieved through parameter cross-operation, and finally a new population that has completed the global update is generated, thereby broadening the search range of layout schemes.

[0022] S6. If the global search triggering condition is not met, each policy agent samples the placement trajectory from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectory. Specifically, for step S6, if the current iteration does not meet the global search triggering condition, each policy agent in the population independently extracts a portion of the placement trajectory from the shared experience pool as an optimization reference; each agent adjusts its own policy parameters based on the sampled placement trajectory and corresponding evaluation value, thereby completing the local policy optimization update of a single agent and improving the accuracy of its own layout decision.

[0023] S7. Repeat the interaction, storage and optimization update process until the preset termination condition is reached, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module; Specifically, for step S7, the entire process of agent-environment interaction to generate trajectory, trajectory storage, condition judgment and policy update is repeated to continuously iterate and optimize the layout scheme; when the iteration process reaches the preset termination condition, the loop process is terminated; finally, from all generated placement trajectories, the placement trajectory with the best layout quality evaluation value is selected, and the macro module placement scheme corresponding to the trajectory is output as the final integrated circuit macro module layout result.

[0024] This embodiment explores integrated circuit macro-module layout schemes in parallel using multiple strategy agents, then shares layout experience in a unified manner, and flexibly switches between global cross-updates and local strategy optimizations according to iteration conditions to continuously iterate and optimize the layout scheme, ultimately outputting the optimal integrated circuit macro-module layout result efficiently.

[0025] Furthermore, in some embodiments, step S1, "initializing a population composed of multiple policy agents and setting initial policy parameters for each policy agent," may specifically include: S11. Determine the population size and generate the corresponding number of policy agents; Specifically, for step S11, considering the actual needs of integrated circuit macro-module layout, the total number of policy agents participating in the layout co-evolution is first determined, i.e., the population size is determined. After the population size is determined, a corresponding number of policy agents are generated one by one according to this standard, completing the main construction of the initial population and ensuring that the population size is adapted to the exploration and optimization needs of the current macro-module layout. For example, for a medium-sized chip layout task containing 10 macro-modules, the population size is determined to be 6, and then 6 independent policy agents are generated to form the basic layout exploration population.

[0026] S12. Construct a policy network with the same network structure for each policy agent. The policy network is used to output the probability distribution of macro module placement actions based on the input chip layout state information. Specifically, for step S12, a policy network with a completely identical structure is built for each policy agent in the population, ensuring that all agents have a unified decision-making framework. The core function of this policy network is to receive real-time chip layout status information as input, process it internally, and output the probability distribution of placement actions for macro modules at various positions on the chip layout, providing a basis for the agents to make placement decisions. For example, by building a policy network with the same structure for all six policy agents, when the current chip layout status information is input, each agent's policy network can output the probability distribution result of the corresponding placement position.

[0027] S13. Randomly initialize the policy parameters of each policy agent to give each policy agent a differentiated initial policy; Specifically, in step S13, the policy parameters of the policy network corresponding to each policy agent are independently and randomly assigned to complete the initial parameter settings. This random initialization ensures that the policy parameters of each agent are different, thus giving each agent a differentiated placement decision logic and exploration direction in the initial deployment phase. For example, the policy network structures of the six agents may be identical, but their policy parameters are all initialized randomly, resulting in significant differences in the initial macro-module placement preferences and decision logic of each agent.

[0028] This embodiment constructs a unified policy network after determining the population size and randomly initializes the policy parameters, thereby creating a population of agents with a unified structure but differentiated initial exploration strategies. This provides a foundation for subsequent multi-path parallel exploration of macro-module layout.

[0029] Furthermore, in some embodiments, the policy network in this embodiment adopts an encoder-decoder structure, and the construction method of the policy network includes: S121. Construct an encoder. The encoder extracts the spatial features of the chip layout state information step by step through multiple convolutional layers, and sets a max pooling layer between every two convolutional layers for downsampling. Specifically, in step S121, the encoder module of the policy network is constructed. The encoder uses multiple convolutional layers as core feature extraction units to process the input chip layout state information layer by layer. It mines and extracts the spatial features contained in the chip layout state through multiple convolutional layers. Between each pair of adjacent convolutional layers, a max-pooling layer is set to perform downsampling on the feature map output by the convolutional layer. While preserving the core spatial features, this compresses the feature map dimension, reduces data processing volume, and improves the encoder's feature extraction efficiency. For example, the encoder uses five convolutional layers to extract the spatial features of the chip layout state. Between adjacent convolutional layers such as the first and second layers, and the second and third layers, a max-pooling layer is set to perform downsampling, gradually refining the key spatial features of the chip layout.

[0030] S122. Construct the decoder. The decoder gradually recovers the spatial resolution through multiple upsampling layers and uses skip connections to concatenate the features of the corresponding layer of the encoder with the features of the current layer of the decoder. Specifically, in step S122, the decoder module of the policy network is constructed. The decoder uses multiple upsampling layers as its core processing unit. By performing upsampling operations layer by layer, it gradually restores the low-resolution feature map output by the encoder to a spatial resolution suitable for the chip layout. At the same time, a skip connection processing method is adopted to directly concatenate the features output by the corresponding layer in the encoder with the features of the current processing layer of the decoder, thus completely preserving the detailed features of the chip layout and avoiding the loss of feature information during the encoding and decoding process. For example, the decoder uses upsampling layers with the same number of layers as the encoder to restore the spatial resolution. During each upsampling process, skip connections are used to concatenate the features of the corresponding layer of the encoder, allowing the decoder to fuse deep abstract features with shallow detailed features.

[0031] S123. Set a convolutional layer in the last layer of the decoder to output a placement action probability matrix corresponding to the chip layout size; Specifically, in step S123, after the last processing unit of the decoder, a dedicated convolutional layer is set as the output layer of the policy network. This output convolutional layer performs the final calculation based on the feature information processed by the decoder, and outputs a placement action probability matrix that corresponds exactly to the chip layout size. The value at each position in the matrix represents the placement probability of the macromodule at the corresponding position on the chip layout, directly providing data support for placement decisions. For example, if the chip layout size is 256×256, the last convolutional layer of the decoder will output a placement action probability matrix of 256×256, with each element in the matrix corresponding to the placement probability of the layout position.

[0032] This embodiment constructs a policy network with an encoder-decoder structure to accurately extract the spatial features of the chip layout and output a placement action probability matrix that matches the layout size, providing a stable and accurate computational basis for the macro-module placement decision of the policy agent.

[0033] Furthermore, in some embodiments, step S2, "interacting with the macro-module layout environment of each policy agent and the integrated circuit, sequentially placing each macro-module on the chip layout, generating a complete placement trajectory, and obtaining the layout quality evaluation value corresponding to each placement trajectory," may specifically include: S21. Obtain the current chip layout status information, including a location mask for indicating the placeable area and a line length mask for indicating the placement cost; Specifically, for step S21, in the initial stage of the interaction between the strategy agent and the integrated circuit macro-module layout environment, the current chip layout status information is collected in real time. This status information includes two types of key mask data: a position mask is used to clearly identify the placeable and non-placeable areas on the chip layout, defining the legal placement range of the macro-module; and a line length mask is used to quantify the line length-related costs incurred when placing the macro-module at different positions on the layout, providing a cost reference for layout decisions. For example, when collecting the status of a chip layout, the position mask will mark the occupied areas and boundary restricted areas as non-placeable positions, and the blank core areas as placeable positions; the line length mask will correspondingly mark the line length cost values ​​at different positions on the layout, intuitively reflecting the placement cost at each position.

[0034] S22. Each policy agent selects the target position of the macro module to be placed in sequence according to the current state information and the placement action probability distribution output by the corresponding policy network, and updates the chip layout state after each placement action is completed. Specifically, in step S22, each policy agent, based on the currently acquired chip layout state information, outputs a probability distribution of macro-module placement actions through its corresponding policy network. Following this probability distribution, the agent sequentially selects target placement locations for the macro-modules to be placed and completes the placement operation. After each macro-module placement action, the chip layout state is immediately updated, and the available placement area and placement cost are adjusted synchronously to provide the latest environmental basis for the next placement action. For example, the agent first places the first macro-module based on the initial layout state, updates the layout after placement, and marks the location as occupied; then, based on the updated layout state, it places the second macro-module according to the probability distribution, and so on, sequentially placing all macro-modules in this manner.

[0035] S23. After each placement action is completed, calculate the change in the semi-circle length caused by the placement action, and use the negative value of the change as the immediate reward. Accumulate the total reward value of the entire placement trajectory, and use the total reward value as the layout quality evaluation value of the placement trajectory. Specifically, for step S23, after each macro module placement action, the change in the half-perimeter length of the chip layout caused by the placement action is calculated in real time. The negative value of this change in half-perimeter length is used as the immediate reward for this placement action, thereby guiding the agent to optimize the length index. The immediate rewards of all placement actions in the entire placement trajectory are accumulated to obtain the total reward value, which is directly used as the layout quality evaluation value of this placement trajectory to quantify the merits of the layout scheme. For example, if a placement action increases the half-perimeter length by 4, the immediate reward is -4; if another placement action decreases the half-perimeter length by 6, the immediate reward is 6. The total reward of 18 is obtained by accumulating all the immediate rewards of this trajectory, which is the layout quality evaluation value of this placement trajectory.

[0036] This embodiment collects the layout status, completes the sequential placement of macro modules according to probability distribution, and calculates rewards based on line length changes. It can generate a complete placement trajectory in a standardized manner and accurately quantify the layout quality, providing an objective evaluation basis for subsequent strategy optimization.

[0037] Furthermore, in some embodiments, the location mask and line length mask are constructed in the following ways: S211. Construct the location mask into a binary matrix. In the binary matrix, the positions with a value of 0 are marked as areas where macro modules are prohibited from being placed, and the positions with a value of 1 are marked as areas where macro modules are allowed to be placed. Specifically, for step S211, a binary matrix perfectly matching the chip layout size is constructed as a location mask. Each element in this matrix is ​​identified by only two values: 0 and 1. A value of 0 corresponds to an area on the chip layout where macro modules are prohibited, while a value of 1 corresponds to an area on the chip layout where macro modules are allowed. By distinguishing these binary values, the legal placement range of macro modules is clearly and intuitively defined. For example, if the chip layout is 256×256, the location mask is a 256×256 binary matrix, where elements corresponding to chip boundaries and occupied areas are assigned a value of 0, and elements corresponding to empty, available areas are assigned a value of 1.

[0038] S212. Construct the line length mask into a continuous value matrix, where each element of the continuous value matrix is ​​used to quantize the half-circumference line length increment caused when the current macro module is placed in the corresponding position; Specifically, for step S212, a continuous value matrix consistent with the chip layout size is constructed as a line length mask based on the actual size of the chip layout. Each element in this matrix is ​​a continuous value, and each value is specifically used for precise quantization. It represents the increase in half-perimeter line length caused when the macro module to be placed is placed at the corresponding position on the layout. The value directly reflects the line length cost at that position. For example, to match the 256×256 chip layout mentioned above, the line length mask is a 256×256 continuous value matrix. If an element at a certain position in the matrix is ​​3, it means that placing the macro module at this position will increase the half-perimeter line length by 3; if an element at a certain position is 7, it means that placing it at this position will increase the half-perimeter line length by 7.

[0039] This embodiment uses a binary matrix to divide the legal placement area and a continuous value matrix to quantify the line length increment, providing a clear and quantitative basis for identifying the chip layout status and ensuring the accuracy and rationality of macro module placement decisions.

[0040] Furthermore, in some embodiments, before outputting the placement probability distribution of the macro module, the method further includes: S201. Multiply the original probability matrix output by the policy network element by element with the position mask, and set the probability value corresponding to the unplaceable position to zero; Specifically, in step S201, before the policy network outputs the final probability distribution of macromodule placement actions, the original probability matrix generated by the policy network is first obtained. This matrix contains the placement probability values ​​for all positions on the chip layout. Then, the original probability matrix is ​​multiplied element-wise with the position mask. Since the values ​​corresponding to non-placeable areas in the position mask are 0 and the values ​​corresponding to placeable areas are 1, the probability value of non-placeable positions is directly set to 0 after multiplication, retaining only the original probability values ​​of legally placeable positions, thus masking the probability of illegal placement positions. For example, in a 10×10 original probability matrix corresponding to the chip layout, the 3rd row and 4th column represent a non-placeable area. The position mask for this position is 0, and the original probability for this position is 0.08. After multiplication, the probability of this position becomes 0; the probabilities of the remaining placeable positions remain unchanged.

[0041] S202. Normalize all non-zero probability values ​​in the probability matrix after masking to obtain a probability distribution that only indicates the legal placement position; Specifically, in step S202, the probability matrix after the masking process is processed by performing normalization only on the remaining non-zero probability values. Through normalization adjustment, the sum of the probability values ​​of all legally placeable positions is made equal to 1, ultimately obtaining a probability distribution for placement actions only for legally placeable positions, completely eliminating the interference of illegal positions on placement decisions. For example, after masking, the probability values ​​of legal positions are 0.12, 0.15, and 0.18, respectively. After normalization, these values ​​are adjusted proportionally, and their sum becomes 1, forming a final probability distribution that only applies to legal positions.

[0042] This embodiment uses probability masking and normalization to eliminate probability interference from illegal placement locations and generate a placement probability distribution only for legal areas, ensuring that macro module placement decisions always comply with the requirements of the layout.

[0043] Furthermore, in some embodiments, step S3, "Storing the placement trajectories generated by all policy agents into the shared experience pool," may specifically include: S31. Treat each complete placement trajectory as an independent storage unit; Specifically, in step S31, when storing layout data in the shared experience pool, a single macro-module placement action is not used as the storage unit. Instead, the complete placement trajectory formed by the policy agent after placing all macro-modules is used as the smallest independent storage unit. This ensures that each piece of data stored in the experience pool corresponds to a complete macro-module layout scheme, guaranteeing the integrity and decision-making reference value of the experience data. For example, if a policy agent sequentially completes the placement operations of all 7 macro-modules on the chip, forming a continuous and complete layout process from the first module to the last module, this entire layout process is treated as an independent storage unit and prepared for storage in the shared experience pool.

[0044] S32. Determine whether the layout quality evaluation value of the current placement trajectory is better than the best trajectory already stored in the shared experience pool; Specifically, in step S32, before storing the new placement trajectory, the optimal trajectory with the highest layout quality evaluation value is extracted from all the trajectories already stored in the shared experience pool. The layout quality evaluation value of the current placement trajectory to be stored is compared with the evaluation value of the optimal trajectory to determine whether the current trajectory is a better layout scheme. For example, if the layout quality evaluation value of the optimal trajectory already stored in the shared experience pool is 92, and the evaluation value of the newly generated trajectory is 96, the current trajectory can be determined to be better by comparison; if the evaluation value of the new trajectory is 88, it is determined that it does not exceed the optimal trajectory.

[0045] S33. If so, the placement trajectory will be stored in the optimal experience zone of the shared experience pool; Specifically, in step S33, if, after comparison, the layout quality evaluation value of the current placement trajectory to be stored is better than the optimal trajectory already stored in the shared experience pool, then the placement trajectory is directly stored in the optimal experience area of ​​the shared experience pool, thus completing the retention of high-quality layout experience and preserving the optimal complete layout scheme data for the current stage. For example, if the evaluation value of the new trajectory is 96, which is higher than the original optimal value of 92, the trajectory is directly stored in the optimal experience area as the new optimal layout experience.

[0046] S34. If not, the placement trajectory will be stored in the latest experience area of ​​the shared experience pool, and the placement trajectory with the longest storage time in the latest area will be removed according to the rolling update method. Specifically, in step S34, if the evaluation value of the current trajectory to be stored is not better than the optimal trajectory, it is stored in the latest experience area of ​​the shared experience pool. Simultaneously, storage management is performed according to the rolling update rule. When the storage capacity of the latest experience area reaches its limit, the placement trajectory with the longest storage time and lowest timeliness is automatically removed to free up storage space for new trajectories, ensuring that the latest experience area always retains recently generated layout trajectories. For example, if the maximum storage capacity of the latest experience area is 6 trajectories, when storing the 7th new trajectory, the oldest stored trajectory is directly removed, maintaining the capacity and data timeliness of the latest experience area.

[0047] This embodiment uses the complete layout trajectory as an independent storage unit, stores it in partitions and layers according to layout quality, and updates it on a rolling basis. This not only stably retains the best layout experience, but also preserves the latest exploration data, making the experience data in the shared experience pool more referential and practical.

[0048] Furthermore, in some embodiments, the internal structure of the shared experience pool is divided in the following ways: S301. Set the optimal experience zone to store several placement trajectories with the highest historical layout quality evaluation values. When the evaluation value of a new placement trajectory is better than the worst trajectory in the optimal experience zone, perform a replacement operation. Specifically, in step S301, an independent optimal experience area is defined within the shared experience pool. This area is specifically used to store the highest-quality placement trajectories throughout the entire layout exploration process, continuously retaining historically optimal layout schemes. Simultaneously, a dynamic replacement rule is configured for the optimal experience area. When a newly generated placement trajectory enters the storage stage, it is compared with the trajectory in the optimal experience area currently having the lowest evaluation value. If the new trajectory has better layout quality, the worst trajectory in the optimal experience area is removed, and the new trajectory is stored in this area, ensuring that the optimal experience area always stores the layout trajectory with the best quality at the current stage. For example, the optimal experience area is set to store four optimal placement trajectories. If the trajectory with the lowest evaluation value in the current area is 86, and the newly generated trajectory has an evaluation value of 90, which is better than the worst trajectory, then the trajectory with an evaluation value of 86 is removed, and the new trajectory is stored in the optimal experience area.

[0049] S302. Set up a latest experience zone and use a first-in-first-out rolling update method to store the most recently generated placement trajectory. When the latest experience zone is full, remove the placement trajectory with the longest storage time to store the new trajectory. Specifically, in step S301, a separate "Latest Experience Area" is defined within the shared experience pool. This area uses a first-in, first-out (FIFO) rolling update method to store recently generated placement trajectories, retaining the latest layout exploration data. When the storage capacity of the Latest Experience Area reaches a preset limit and new trajectories cannot be directly stored, the placement trajectory with the longest storage time is removed according to the storage time order, and then the new trajectory is stored, ensuring that the Latest Experience Area always carries the most timely layout exploration results. For example, the Latest Experience Area is preset to store a maximum of 6 trajectories. When a 7th new trajectory needs to be stored, the oldest stored trajectory is removed, and then the new trajectory is added, maintaining the capacity and data timeliness of the Latest Experience Area.

[0050] This embodiment divides the shared experience pool into the optimal experience area and the latest experience area, and sets appropriate storage and update rules for each. This not only allows for the stable retention of high-quality layout experience, but also enables the real-time retention of the latest exploration data, making the structure of the shared experience pool more reasonable and the data value more prominent.

[0051] Furthermore, in some embodiments, step S5, "using the policy parameters of elite policy agents to perform a crossover operation on the policy parameters of other policy agents to generate an updated population," may specifically include: S51. Set a global search cycle value. After completing the interaction and storage process for each cycle value number of times, determine if the global search trigger condition is met. Specifically, for step S51, a fixed global search cycle value is preset. This value represents the total number of interactions between the agent and the layout environment, and the total number of trajectory storage operations that need to be completed. Each time the interaction and storage process is completed a cumulative number of times within this cycle, the current iteration is determined to meet the trigger condition for global search, and the process enters the global search and population update phase. For example, if the global search cycle value is set to 6, after a cumulative total of 6 agent interaction layout and trajectory storage operations, the global search trigger condition is immediately determined to be met, and the subsequent population cross-update process is initiated.

[0052] S52. Sort each policy agent according to its layout quality assessment value in the current period, and select at least one policy agent with the highest ranking as an elite policy agent. Specifically, for step S52, all policy agents participating in the layout exploration within the current period are sorted from highest to lowest according to their corresponding layout quality evaluation values. Based on the sorting results, at least one policy agent with a high ranking is selected and identified as an elite policy agent, serving as a source of high-quality parameters for population cross-updates. For example, if the current population contains 10 policy agents, after arranging the layout quality evaluation values ​​of each agent from highest to lowest, the top two agents are selected as the elite policy agents for this global search.

[0053] S53. Select target agents to be cross-referenced from non-elite policy agents, replace the parameters of at least one complete network layer in the policy network of the elite policy agents with the corresponding network layer in the policy network of the target agent, generate updated policy agents, and form an updated population. Specifically, for step S53, from the ordinary agents in the population other than the elite policy agents, select the target agents to be cross-referenced that need parameter updates; directly replace all parameters of at least one complete network layer in the policy network of the elite policy agent with the same level position in the policy network of the target agent, completing the cross-reference operation and generating a policy agent with updated parameters; after completing the above operation for all target agents to be updated, the updated agents together form a new generation of population. For example, select an ordinary agent as the target object, replace all parameters of the second complete network layer in the policy network of the elite agent with the second network layer of the target agent, completing the parameter cross-reference; after performing this operation on all non-elite agents in the population, a new population with global updates is obtained.

[0054] This embodiment achieves global iterative upgrades of the population by periodically triggering global search and selecting elite agents based on layout quality, and performing cross-updates with complete network layer parameters. This effectively broadens the search range of layout schemes and improves the overall layout exploration capability of the population.

[0055] Furthermore, in some embodiments, after generating the updated population, the method further includes: S54. Re-evaluate the updated layout quality assessment value of the policy agent; Specifically, in step S54, after completing the cross operation and generating the updated policy agent, for each policy agent that has undergone cross-update, the layout interaction and quality evaluation operation is performed again to obtain the updated layout quality evaluation value of the agent. This allows for a precise determination of the actual impact of the cross operation on the layout performance of the policy agent. For example, if a policy agent's layout quality evaluation value before the cross operation is 88, after completing the parameter cross-update, the macro module layout process is re-executed and calculated, resulting in an updated layout quality evaluation value of 75.

[0056] S55. If the layout quality evaluation value of the updated policy agent decreases by more than a preset threshold compared to before the cross operation, the policy parameters of the updated policy agent will be restored to the state before the cross operation. Specifically, for step S55, a preset allowable threshold for the decrease in layout quality evaluation value is set. The updated layout quality evaluation value of the policy agent is compared with the original evaluation value before the crossover operation, and the decrease in evaluation value is calculated. If the decrease exceeds the preset threshold, it indicates that the layout performance of the agent has been significantly degraded by this crossover operation. At this time, the policy parameters of the policy agent are restored to the original state before the crossover operation, and its original effective layout strategy is retained. For example, if the preset threshold for the decrease in evaluation value is 10, and the above agent's evaluation value before the crossover is 88 and after the update is 75, the decrease is 13, which exceeds the preset threshold. Therefore, the policy parameters of the agent are restored to the state before the crossover operation.

[0057] This embodiment avoids layout performance degradation caused by crossover operations by re-evaluating the performance of agents after crossover updates and rolling back parameters for agents with significantly degraded performance, thus ensuring the stability of the overall layout optimization of the population.

[0058] Furthermore, in some embodiments, step S6, "samples placement trajectories from the shared experience pool for each policy agent and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectories," may specifically include: S61. Multiple placement trajectories are randomly sampled from the shared experience pool by each policy agent to form a training batch; Specifically, in step S61, during the local policy optimization update, each policy agent independently performs a random sampling operation from all placement trajectories stored in the shared experience pool, selecting multiple complete placement trajectories at once. These selected trajectories are then integrated into a group to form a training batch specifically for this local optimization, providing stable and batch-based reference data for policy parameter optimization. For example, in this local optimization, a policy agent randomly selects 8 complete placement trajectories from the shared experience pool, combining these 8 trajectories into a training batch as the basis for subsequent parameter optimization calculations.

[0059] S62. The objective function for optimizing the placement trajectory calculation strategy based on sampling is used to maximize the expected cumulative reward under the constraint of the policy update magnitude; Specifically, for step S62, using all placement trajectories within the training batch as data, a strategy optimization objective function is calculated. This objective function plays a dual role: first, it constrains the update range of strategy parameters to prevent excessive parameter updates from causing layout strategy disorder; second, it guides the parameters to adjust towards improving layout quality by maximizing the expected cumulative reward of the layout process. For example, based on the layout data of the above 8 placement trajectories, a corresponding strategy optimization objective function is calculated and generated. This function both limits the maximum range of a single parameter update and aims to increase the overall cumulative reward.

[0060] S63. Use gradient descent to update the policy network parameters and value network parameters of the policy agent to complete the local policy optimization in this round; Specifically, in step S63, gradient descent is used as a parameter optimization method. Combined with the calculation results of the policy optimization objective function, the policy network parameters and value network parameters of the policy agent are iteratively adjusted. By gradually approaching a better parameter state through gradient descent, this round of local policy optimization update for a single policy agent is completed, allowing for a more refined improvement in the agent's deployment decision-making ability. For example, following the gradient direction of the objective function, the parameters of the policy network and value network are gradually corrected using gradient descent. Once the parameter adjustment is complete, the local policy optimization process ends.

[0061] This embodiment constructs a training batch by sampling trajectories, calculates a constrained optimization objective function, and uses gradient descent to update the parameters of the two networks, thereby achieving local precise optimization of the policy agent and steadily improving the layout decision performance of a single agent.

[0062] Furthermore, in some embodiments, step S7, "repeatedly executing the interaction, storage, and optimization update process until a preset termination condition is met, and outputting the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macromodule layout result," may specifically include: S71. After each round of interaction, storage and optimization update is completed, determine whether the current iteration number has reached the preset maximum iteration number, or determine whether the layout quality evaluation value obtained in several consecutive iterations no longer improves; Specifically, for step S71, after each round of interaction between the agent and the layout environment, storage of placement trajectories, and policy optimization and update, the iteration termination condition is immediately double-checked: on the one hand, the total number of iterations already executed is checked to determine whether the preset maximum number of iterations has been reached; on the other hand, the layout quality evaluation value of multiple consecutive iterations is statistically analyzed to determine whether the value continues to show no improvement or no further optimization changes. As long as either condition is met, the iteration termination condition is determined to be met. For example, if the preset maximum number of iterations is 100 rounds, the termination condition is determined to be met when 100 iterations have been completed; or if the preset termination condition is that the evaluation value does not improve after 5 consecutive iterations, the termination condition is also determined to be met if the layout quality evaluation value obtained after 5 consecutive iterations does not increase.

[0063] S72. If the termination condition is met, the iteration process is terminated, and the placement trajectory with the best layout quality evaluation value is selected from all the generated placement trajectories as the optimal placement trajectory. Specifically, for step S72, when the iteration termination condition is met, all iterative operations of layout exploration and strategy optimization are immediately stopped; all placement trajectories generated by all agents throughout the entire iteration process are summarized, and filtered according to the layout quality evaluation value from high to low, and the placement trajectory with the highest evaluation value is selected as the optimal placement trajectory for this layout optimization. For example, after the iteration terminates, all 200 generated placement trajectories are summarized, and the trajectory with the highest layout quality evaluation value of 98 is selected as the optimal placement trajectory.

[0064] S73. Output the final coordinate positions of each macro module corresponding to the optimal placement trajectory on the chip layout. The final coordinate positions satisfy the physical constraint that the macro modules do not overlap. Specifically, for step S73, based on the selected optimal placement trajectory, the final coordinate positions of each macromodule on the chip layout are extracted, and this coordinate information is output as the final layout result. All output macromodule coordinate positions strictly adhere to the physical layout constraint that macromodules do not overlap, ensuring the layout scheme is practically feasible. For example, the optimal placement trajectory records the x, y coordinates and size information of eight macromodules on the chip. The output coordinates ensure that the placement areas of each macromodule are independent and their boundaries do not overlap, meeting the chip layout physical layout requirements.

[0065] This embodiment outputs compliant coordinates after iteratively determining the termination condition and selecting the optimal layout trajectory, thus stably ending the layout optimization process and finally outputting the integrated circuit macro module layout result that satisfies physical constraints and has the best quality.

[0066] To facilitate understanding of the co-evolution method for integrated circuit macro-module layout provided in this embodiment, such as Figure 3 As shown, this embodiment also provides a specific implementation of a co-evolutionary method for integrated circuit macro-module placement, comprising three main parts: population initialization and evaluation, a global search mechanism (evolutionary algorithm), and a local search mechanism (reinforcement learning). The entire framework achieves efficient solution to the integrated circuit macro-module placement problem by alternately executing global exploration and local optimization. The specific steps are as follows: Population initialization: First, multiple policy agents are randomly generated to form the initial population. All agents share the same network structure, but their parameters are randomly initialized to achieve diverse starting exploration points.

[0067] Environmental Interaction and Evaluation: In each iteration, each agent interacts with the layout environment, placing macromodules sequentially, generating a complete placement trajectory, and obtaining a corresponding reward value (based on HPWL negative increments). This process simultaneously records the state, action, and reward at each step, serving as empirical data for subsequent training.

[0068] Experience replay buffer updates: Trajectories generated by all agents are uniformly stored in a shared experience replay buffer. This buffer is divided into two parts: the optimal experience pool (stores the trajectory with the highest historical reward); and the latest experience pool (stores the latest interaction data using a rolling update method). This structure ensures that agents can both learn from historical best experiences and acquire the latest exploration information, thereby effectively escaping local optima.

[0069] Global Search (Evolutionary Algorithm): Every few iterations (e.g., every 5), the system performs a global search. First, the best agent is selected from the current population as the elite. Then, a hierarchical crossover operation is performed on the other agents, replacing a certain layer of network parameters from the elite agent with those from ordinary agents. This operation enhances the diversity of the population and expands the search space.

[0070] Local Search (Reinforcement Learning): In the iterations of the non-global search, the system performs a Policy Optimization (PPO) update for each agent in the population. Each agent samples experience from the shared buffer to optimize its policy, leveraging the collective experience of the group's exploration to accelerate its own convergence.

[0071] Rollback mechanism: If an agent's performance significantly degrades after a global search, the system will trigger a rollback mechanism to restore its parameters to their state before the crossover operation. This mechanism effectively suppresses the negative impact of random perturbations and enhances the robustness of optimization.

[0072] Optimal solution output: After reaching the maximum number of iterations, the system outputs the best layout result obtained throughout the process and the corresponding elite agent.

[0073] This embodiment also proposes a global search mechanism based on cross-operations. This mechanism generates diverse candidate policies for reinforcement learning by performing cross-operations between the evolutionary population and the elite policy library. This process effectively enhances the framework's global exploration capability, expands its search range for high-performance placement policies, and effectively alleviates the problem of policy training getting trapped in local optima by providing diverse training experiences to the reinforcement learning agent. To further ensure the stability of the optimization process, the framework introduces a rollback mechanism (current performance evaluation only assesses the HPWL value and the final TotalReward; since there are clear calculation rules between TotalReward and HPWL, the final evaluation metric can be considered HPWL): for offspring agents whose performance improvement is not significant after evaluation, their policy parameters are discarded and they revert to their parent policy. This mechanism significantly enhances the robustness of the overall macro-placement optimization process while suppressing fluctuations caused by randomness.

[0074] Furthermore, this embodiment achieves efficient local search by introducing a shared experience replay buffer and a corresponding collaborative update mechanism. This mechanism allows each agent in the population to optimize its strategy using the collective experience accumulated by the entire group's exploration, thereby accelerating the improvement of individual performance. This collaborative learning based on group experience can effectively guide agents to avoid local optimum traps and converge towards a globally better direction. The global exploration advantage of evolutionary algorithms is deeply coupled with the local optimization capability of reinforcement learning to automatically generate high-quality integrated circuit macro-module layouts that meet physical constraints. Through the aforementioned bidirectional collaborative mechanism, this framework demonstrates significant comprehensive advantages over existing methods in key indicators such as layout quality, convergence speed, and computational resource efficiency.

[0075] Specifically, regarding the shared experience replay buffer, the shared buffer in ERLPlace is divided into two distinct regions: one storing the optimal result and the other storing the best result. When a new experience is passed in, it is first compared with the best experience. If it is better than the worst experience in the best experience pool, it is replaced; otherwise, the oldest experience in the latest experience pool is replaced. The buffer's capacity unit is not the original Transaction, but the Trajectory. Using the complete layout trajectory during layout as the unit of measurement, rather than a single action, makes it easier to evaluate whether a Transaction should be retained when partitioning the buffer.

[0076] Specifically, regarding the global search mechanism, the extraction dimensions and quantification methods for macro-module layout features are as follows: Macro-module layout features are primarily integrated into two masks (i.e., a location mask and a line length mask). The location mask indicates which locations can be used to place modules, while the line length mask indicates the line length increment caused by placing modules at each location. The sizes of these two masks are set according to different circuits (usually related to the smallest macro-module size within the layout). For example, in ISPD2005, it is set to 128 for Adaptec1 and 512 for the larger Adaptec4.

[0077] If feature extraction refers to the convolutional kernels of the CNN in UNet, the CNN layer on the left side of UNet is configured as follows: CNN(in=3,out=8,kernel=3,stride1,padding=1); CNN(in=8,out=16,kernel=3,stride1,padding=1); CNN(in=16,out=32,kernel=3,stride1,padding=1); CNN(in=32,out=32,kernel=3,stride1,padding=1); Each CNN layer has a corresponding MaxPooling layer for transition, followed by a fully connected layer, and then the corresponding upsampling and CNN layer output probability matrix.

[0078] Regarding the construction rules of the elite strategy pool, the elite strategy pool consists of individuals that achieved the best layout result (best average HPWL) between the current time and the last crossover. The top 20% of individuals in the population are selected (i.e., the top 1 out of 5 individuals, and the top 2 out of 10 individuals).

[0079] Regarding the specific process of layer weight crossover, it differs from the conventional EA algorithm. When using fine-grained crossover operations, the individual performance degrades significantly. Although the exploration speed is fast, the exploration quality is poor. Therefore, we adopt the method of directly exchanging the weights of the entire layer during crossover to ensure both performance and exploration speed.

[0080] This embodiment also provides specific rules for population initialization and strategies for maintaining the diversity of the elite policy library. Population initialization involves initializing several individuals with the same network structure but randomized weights. The diversity of the elite policy library originates from the different starting points (initial network weights), which gradually converge as training progresses (with continuous crossovers). The elite policies are derived from recording the final layout scores of individuals at each interaction, averaging and ranking them before the crossover occurs, with the top one or more individuals being the elite policies. During the crossover, a softmin is performed to extract the elite individuals to be used as targets.

[0081] In the local search mechanism, this framework employs the Proximal Policy Optimization (PPO) algorithm to update the policy of each agent in the population, thereby improving local optimization capabilities. All agents share a unified experience replay buffer, which consists of two parts: the optimal part and the latest part. The optimal part stores historical optimal trajectories, providing escape directions for agents trapped in local optima; the latest part is updated using a rolling update method to ensure that agents can acquire the latest exploration experience and avoid reusing outdated data.

[0082] The agent's network architecture is based on U-Net, comprising an encoder and a decoder. The encoder extracts features from the visual input through four convolutional blocks. The visual input includes a position mask (a binary matrix identifying feasible placement locations) and a mesh mask (a continuous matrix quantizing the half-perimeter line length increments caused by the placement macromodule). The decoder recovers the spatial resolution using skip connections, ultimately outputting the probability distribution of the placement action. The value network and policy network share the same convolutional structure, outputting scalar state values ​​only in the fully connected layers.

[0083] During the update process, each agent samples trajectories from the shared buffer and calculates the pruning objective function of the PPO using importance sampling and pruning mechanisms. The policy network and value network are then optimized using stochastic gradient descent. Each update cycle executes 10 PPO rounds. Through this shared experience and collaborative update mechanism, each agent can leverage the collective experience of the group to achieve self-improvement, effectively accelerating convergence, escaping local optima, and achieving better layout quality within a limited number of environmental interactions.

[0084] The reward function is designed so that the immediate reward at each step is the negative increment of HPWL caused by placing the current macromodule, and the total reward is the cumulative sum, guiding the agent to minimize the final line length. Finally, for each connected network (consisting of pins that need to be connected), its HPWL value is calculated (by obtaining the maximum and minimum X and Y coordinates of the pins, and then calculating the HPWL of the current network). All HPWL values ​​are then summed to obtain the HPWL value of the entire layout. The ultimate goal is to optimize the HPWL value of the entire layout to be as small as possible when there is no macromodule overlap.

[0085] Because of the introduction of position masks, after the agent obtains the action probability matrix using the position mask and line length mask, the final step involves masking the position mask to eliminate all illegal and unknown probabilities. Then, the probabilities of all feasible positions are normalized to obtain a new probability matrix. Finally, the action is obtained by applying Softmax.

[0086] In summary, compared with existing technologies, the co-evolutionary method for integrated circuit macro-module layout provided in this embodiment generates new candidate solutions for reinforcement learning through cross-operations between swarm and elite agents. The mechanism leverages diverse individuals to provide the RL agent with richer experience, thereby enhancing global exploration capabilities, expanding the state space search range, and mitigating local optima. If an agent with poor performance is generated, the system will eliminate it and revert to a previous agent version, which reduces the adverse effects of randomness and enhances the robustness of macro-layout optimization. The local search mechanism provided in this embodiment, implemented through reinforcement learning, modifies the original experience replay into a shared experience replay and update scheme, enabling each agent to improve itself using collective experience. Therefore, this mechanism allows agents to escape local optima through collaborative learning during training.

[0087] To facilitate better implementation of the co-evolution method for integrated circuit macro-module layout according to the embodiments of this application, this application also provides a co-evolution apparatus for integrated circuit macro-module layout based on the above-described co-evolution method for integrated circuit macro-module layout. The meanings of the terms used are the same as in the above-described co-evolution method for integrated circuit macro-module layout, and specific implementation details can be found in the description of the method embodiments.

[0088] Please see Figure 4 , Figure 4 This is a schematic diagram of a co-evolution device for integrated circuit macro-module layout provided in an embodiment of this application. Specifically, the co-evolution device for integrated circuit macro-module layout may include: The initialization module 201 is used to initialize a population consisting of multiple policy agents and set initial policy parameters for each policy agent. The interactive evaluation module 202 is used to interact with the macro module layout environment of the integrated circuit and place each macro module on the chip layout in sequence to generate a complete placement trajectory and obtain the layout quality evaluation value corresponding to each placement trajectory. Storage module 203 is used to store the placement trajectories generated by all policy agents into a shared experience pool; The judgment module 204 is used to determine whether the current iteration meets the global search triggering condition after each preset number of interactions and storage processes. The global search module 205 is used to select at least one elite policy agent from the current population if the global search triggering condition is met, and to perform a crossover operation on the policy parameters of other policy agents using the policy parameters of the elite policy agents to generate an updated population. The local optimization module 206 is used to sample the placement trajectory from the shared experience pool through each policy agent if the global search triggering condition is not met, and to perform local policy optimization update on its own policy parameters based on the sampled placement trajectory. The output module 207 is used to repeatedly execute the interaction, storage and optimization update process until the preset termination condition is reached, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module.

[0089] For specific limitations regarding the co-evolution device used for integrated circuit macro-module layout, please refer to the limitations on the co-evolution method used for integrated circuit macro-module layout above, which will not be repeated here. Each module in the aforementioned co-evolution device for integrated circuit macro-module layout can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the operations corresponding to each module.

[0090] The co-evolutionary device for integrated circuit macro-module layout provided in this embodiment completes macro-module layout interaction, unified storage and sharing of layout experience by constructing a multi-strategy intelligent agent population, and then updates the population by alternating global search and local optimization. This enables efficient exploration of the layout space, improves layout optimization efficiency and quality, and finally outputs the optimal integrated circuit macro-module layout result.

[0091] Furthermore, embodiments of this application also provide an electronic device, such as... Figure 5 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more computer-readable storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 301 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 302, and by calling data stored in the memory 302, thereby providing overall monitoring of the electronic device. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0092] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and co-evolution methods for integrated circuit macro-module layout by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0093] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0094] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0095] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 302 according to the following instructions, and the processor 301 runs the applications stored in the memory 302 to realize various functions, as follows: Initialize a population of multiple policy agents and set initial policy parameters for each agent. Each agent interacts with the macro-module layout environment of the integrated circuit, sequentially placing each macro-module on the chip layout to generate complete placement trajectories and obtaining the layout quality evaluation value for each trajectory. Store the placement trajectories generated by all policy agents in a shared experience pool. After a preset number of interactions and storage processes, determine if the current iteration meets the global search trigger condition. If the global search trigger condition is met, select at least one elite policy agent from the current population and use its policy parameters to perform a cross operation on the policy parameters of other policy agents, generating an updated population. If the global search trigger condition is not met, each policy agent samples placement trajectories from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectories. Repeat the interaction, storage, and optimization update process until a preset termination condition is reached, and output the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macro-module layout result.

[0096] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0097] This application embodiment constructs a multi-strategy intelligent agent population to collaboratively complete macro-module layout interaction, uniformly store and share layout experience, and then updates the population in an alternating manner of global search and local optimization. This can efficiently explore the layout space, improve the efficiency and quality of layout optimization, and finally output the optimal integrated circuit macro-module layout result.

[0098] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0099] To this end, embodiments of this application provide a storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the co-evolution methods for integrated circuit macro-module layout provided in embodiments of this application. For example, the instructions can execute the following steps: Initialize a population of multiple policy agents and set initial policy parameters for each agent. Each agent interacts with the macro-module layout environment of the integrated circuit, sequentially placing each macro-module on the chip layout to generate complete placement trajectories and obtaining the layout quality evaluation value for each trajectory. Store the placement trajectories generated by all policy agents in a shared experience pool. After a preset number of interactions and storage processes, determine if the current iteration meets the global search trigger condition. If the global search trigger condition is met, select at least one elite policy agent from the current population and use its policy parameters to perform a cross operation on the policy parameters of other policy agents, generating an updated population. If the global search trigger condition is not met, each policy agent samples placement trajectories from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectories. Repeat the interaction, storage, and optimization update process until a preset termination condition is reached, and output the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macro-module layout result.

[0100] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0101] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0102] Since the instructions stored in the storage medium can execute the steps in any of the co-evolution methods for integrated circuit macro-module layout provided in the embodiments of this application, the beneficial effects that any of the co-evolution methods for integrated circuit macro-module layout provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0103] The above provides a detailed description of a collaborative evolution method and apparatus for integrated circuit macro module layout provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A co-evolution method for integrated circuit macro module placement, characterized by, include: Initialize a population consisting of multiple policy agents and set initial policy parameters for each policy agent; Each policy agent interacts with the macro-module layout environment of the integrated circuit, and each macro-module is placed on the chip layout in sequence to generate a complete placement trajectory and obtain the layout quality evaluation value corresponding to each placement trajectory. Store the placement trajectories generated by all policy agents in a shared experience pool; After each preset number of interactions and storage processes, determine whether the current iteration meets the global search trigger condition; If the global search triggering condition is met, at least one elite policy agent is selected from the current population, and the policy parameters of the elite policy agent are used to perform a crossover operation on the policy parameters of other policy agents to generate an updated population. If the global search triggering condition is not met, each policy agent samples the placement trajectory from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectory. Repeat the interaction, storage, and optimization update process until the preset termination condition is met, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module.

2. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The initialization of the population, consisting of multiple policy agents, and the setting of initial policy parameters for each policy agent, includes: Determine the size of the population and generate a corresponding number of policy agents; A policy network with the same network structure is constructed for each policy agent. The policy network is used to output the probability distribution of macro module placement actions based on the input chip layout state information. The policy parameters of each policy agent are randomly initialized to give each policy agent a differentiated initial policy.

3. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The process involves interacting with the macro-module layout environment of each policy agent and the integrated circuit, sequentially placing each macro-module on the chip layout, generating a complete placement trajectory, and obtaining the layout quality evaluation value corresponding to each placement trajectory, including: Obtain the current chip layout status information, which includes a location mask for indicating the placeable area and a line length mask for indicating the placement cost; Each policy agent selects the target position of the macro module to be placed in turn according to the current state information and the placement action probability distribution output by the corresponding policy network, and updates the chip layout state after each placement action is completed. After each placement action is completed, the change in the semi-circle length caused by the placement action is calculated, and the negative value of the change is used as an immediate reward. The total reward value of the entire placement trajectory is accumulated and used as the layout quality evaluation value of the placement trajectory.

4. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The step of storing the placement trajectories generated by all policy agents into a shared experience pool includes: Treat each complete placement trajectory as an independent storage unit; Determine whether the layout quality evaluation value of the current placement trajectory is better than the optimal trajectory already stored in the shared experience pool; If so, the placement trajectory is stored in the optimal experience area of ​​the shared experience pool; If not, the placement trajectory is stored in the latest experience area of ​​the shared experience pool, and the placement trajectory with the longest storage time in the latest area is removed according to the rolling update method.

5. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The step of using the policy parameters of the elite policy agent to perform a crossover operation on the policy parameters of other policy agents to generate an updated population includes: A global search cycle value is set, and after each cycle value number of interactions and storage processes are completed, it is determined that the global search trigger condition is met. Based on the deployment quality assessment value of each strategic agent in the current period, at least one strategic agent with the highest ranking is selected as an elite strategic agent. Select a target agent from the non-elite policy agents to be cross-referenced, and replace the parameters of at least one complete network layer in the policy network of the elite policy agents with the corresponding network layer in the policy network of the target agent to generate an updated policy agent and form an updated population.

6. The co-evolutionary method for integrated circuit macro-module layout according to claim 5, characterized in that, After generating the updated population, the method further includes: Re-evaluate the updated layout quality assessment value of the policy agent; If the layout quality assessment value of the updated policy agent decreases by more than a preset threshold compared to before the cross-operation, the policy parameters of the updated policy agent will be restored to the state before the cross-operation.

7. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, Each policy agent samples placement trajectories from the shared experience pool and performs local policy optimization updates on its own policy parameters based on the sampled placement trajectories, including: Each policy agent randomly samples multiple placement trajectories from the shared experience pool to form a training batch; The objective function for optimizing the placement trajectory calculation strategy based on sampling is used to maximize the expected cumulative reward under the condition of constraining the policy update magnitude. The gradient descent method is used to update the policy network parameters and value network parameters of the policy agent, thus completing the local policy optimization in this round.

8. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The repeated execution of the interaction, storage, and optimization update process continues until a preset termination condition is met, outputting the placement trajectory with the optimal layout quality evaluation value as the integrated circuit macromodule layout result, including: After each round of interaction, storage, and optimization update is completed, it is determined whether the current iteration number has reached the preset maximum iteration number, or whether the layout quality evaluation value obtained in several consecutive iterations no longer improves. If the termination condition is met, the iteration process is terminated, and the placement trajectory with the best layout quality evaluation value is selected from all the generated placement trajectories as the optimal placement trajectory. Output the final coordinate positions of each macromodule corresponding to the optimal placement trajectory on the chip layout, wherein the final coordinate positions satisfy the physical constraint that the macromodules do not overlap.

9. The co-evolutionary method for integrated circuit macro-module layout according to claim 1, characterized in that, The method also includes dividing the internal structure of the shared experience pool, including: An optimal experience zone is set up to store several placement trajectories with the highest historical layout quality evaluation values. When the evaluation value of a new placement trajectory is better than the worst trajectory in the optimal experience zone, a replacement operation is performed. A latest experience zone is set up, and the most recently generated placement trajectory is stored using a first-in-first-out rolling update method. When the latest experience zone is full, the placement trajectory with the longest storage time is removed and a new trajectory is stored.

10. A co-evolution device for integrated circuit macro-module layout, characterized in that, include: The initialization module is used to initialize a population consisting of multiple policy agents and set initial policy parameters for each policy agent. The interactive evaluation module is used to interact with each policy agent and the macro module layout environment of the integrated circuit, sequentially place each macro module on the chip layout, generate a complete placement trajectory, and obtain the layout quality evaluation value corresponding to each placement trajectory. The storage module is used to store the placement trajectories generated by all policy agents into a shared experience pool; The judgment module is used to determine whether the current iteration meets the global search trigger condition after each preset number of interactions and storage processes. The global search module is used to select at least one elite policy agent from the current population if the global search triggering condition is met, and to perform a cross operation on the policy parameters of other policy agents using the policy parameters of the elite policy agent to generate an updated population. The local optimization module is used to sample placement trajectories from the shared experience pool by each policy agent if the global search triggering condition is not met, and to perform local policy optimization updates on its own policy parameters based on the sampled placement trajectories. The output module is used to repeatedly execute the interaction, storage, and optimization update process until the preset termination condition is reached, and output the placement trajectory with the best layout quality evaluation value as the layout result of the integrated circuit macro module.

Images