Direction sensing macro cell layout optimization method and device for chip design

Abstract

The embodiment of the invention provides a direction sensing macro cell layout optimization method and device for chip design. The method comprises the steps that population initialization of an evolutionary algorithm is set; calculating fitness scores and candidate solution probabilities of candidate solutions of the population; calculating a quality score of each macro; visual features are extracted from the current layout state through a reinforcement learning agent, an encoder in the reinforcement learning agent extracts multi-scale spatial features of visual feature layout layer by layer, and the encoder and a decoder generate placement probability distribution in a direction position joint action space; outputting the orientation and placement coordinates of the macro-cells; multiple bottlenecks of an existing learning driving frame are broken through, in a layout frame of the reinforcement learning agent, the orientation (rotation and mirror image) and the position of a macro unit are jointly contained in the same action space for the first time, so that the agent can sample at a time and determine the position and direction of the macro unit, an explorable solution space is expanded, and the synergistic effect between the explorable solution space and the macro unit is captured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and in particular, to a chip design direction-aware macro cell layout optimization device, a chip design direction-aware macro cell layout optimization method, a computer device, and a computer readable storage medium. BACKGROUND

[0002] In modern very large scale integrated circuit (VLSI) design, the performance, power consumption and area (PPA) of the chip are always the core target of optimization. In the physical design stage, macro placement is one of the key steps that affect the final design quality. Macro cells are usually on-chip memories, caches or interface modules, which occupy a larger chip area and determine the subsequent standard cell routing space. If the macro placement is unreasonable, it will lead to routing congestion, timing violation and excessive power consumption, so an efficient and high-quality macro cell placement method is of great significance in the industrial design process.

[0003] In the traditional optimization-driven method, evolutionary algorithms (EA) are widely used for macro cell placement problems, and with the development of reinforcement learning (RL) and deep learning, researchers have begun to explore learning-centered macro cell placement methods.

[0004] On this basis, DeepPR proposes a more complete physical design pipeline, which unifies macro cell placement, standard cell placement and routing process under the same framework for joint optimization, so that the macro cell placement can better adapt to the subsequent standard cell placement and routing steps.

[0005] Further improvement comes from MaskPlace. This method uses a pure convolutional neural network (CNN) architecture and introduces multiple mask mechanisms to explicitly constrain the layout process. MaskPlace solves the sparse reward problem in RL by converting the HPWL increment into a dense reward, thereby significantly improving the training efficiency and convergence speed.

[0006] Although the existing learning-driven methods (such as GraphPlace, DeepPR, MaskPlace, ChiPFormer and EfficientPlace) have made significant progress in the macro cell layout task, they still have the following shortcomings: the end-to-end RL method (GraphPlace, DeepPR, MaskPlace) needs to constantly make a large number of macro layout attempts to converge, which is unacceptable in an industrial scenario; and the offline RL method (ChiPFormer) improves efficiency, but is heavily dependent on large-scale high-quality expert data, which is difficult to obtain in actual design. The MCTS+RL method (EfficientPlace) still has the problem of high computational complexity in macro cell layout: MCTS needs to expand and evaluate a large number of candidate states at each decision step, and the computational overhead increases sharply with the size of the circuit, making it difficult to expand to large-scale chips.

[0007] The method represented by the end-to-end RL framework proposed by Google (GraphPlace, DeepPR, MaskPlace, ChiPFormer) mostly adopts a "zero generation" search mode, that is, starting from an empty chip layout, all macro cells are placed step by step. This paradigm requires the agent to learn complex long-term dependencies, making training extremely difficult. SUMMARY

[0008] In view of the above problems, the embodiments of the present application are proposed in order to provide a chip design direction-aware macro cell layout optimization method, a chip design direction-aware macro cell layout optimization device, a computer device and a computer readable storage medium, which overcome the above problems or at least partially solve the above problems.

[0009] In order to solve the above problems, the embodiments of the present application disclose a chip design direction-aware macro cell layout optimization method, which comprises: Setting the population initialization of the evolutionary algorithm; Calculating the fitness score and candidate solution probability of the population; Calculating the quality score of each macro; The reinforcement learning agent extracts visual features from the current layout state, and the encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer. The encoder and the decoder jointly generate a placement probability distribution in the direction position joint action space; In the output stage, the reinforcement learning agent shields illegal positions through a mask mechanism, and performs softmax normalization on the remaining logic values to output the orientation and placement coordinates of the macro cell.

[0010] Preferably, the method further comprises: The elite strategy is adopted, and the new candidate solution is used to replace the worst solution; The fitness score of each candidate solution is calculated by the evolutionary algorithm to screen out potential candidate solutions, and the screened candidate solutions are subjected to macro quality scoring to identify key macro units forming a bottleneck for layout quality; An optimization macro unit identifier list is generated by the evolutionary algorithm, and the optimization macro unit identifier list and the layout state of the candidate solution jointly constitute the core parameters transmitted to the reinforcement learning agent by the evolutionary algorithm; After receiving the core parameters of the optimization macro unit identifier list, the reinforcement learning agent makes a re-placement decision for the specified macro unit and outputs a joint optimization action containing a direction and coordinates, which is fed back to the evolutionary algorithm as a mutation result to generate a new candidate solution and complete population iteration.

[0011] Preferably, the method further comprises: The encoder is composed of multiple layers of convolution and down-sampling modules, each layer extracts local spatial patterns through convolution operation, and gradually compresses the spatial resolution through pooling or step convolution; The decoder adopts a symmetric up-sampling structure, each layer first gradually recovers the spatial resolution through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer through a jump connection.

[0012] Preferably, the method further comprises: The input tensor of the reinforcement learning agent is obtained by concatenating the mask feature and the global state feature along the channel to obtain a 16-channel input tensor.

[0013] Preferably, the mask feature includes a line length increment mask, a position mask, an overlap mask, and a boundary mask; and the global state feature includes a layout area and a wiring density.

[0014] Preferably, the direction and position joint action space includes a four-dimensional logits tensor with a shape of (B, O, H, W), wherein B represents the batch size, O represents the number of directions corresponding to eight directions {N, S, W, E, FN, FS, FW, FE}, and (H, W) is the grid size of the macro layout.

[0015] Preferably, the method further comprises: A defined reward function of the reinforcement learning agent is generated according to the position and direction, the line length change of the macro unit, the wiring congestion intensity, the macro unit overlap area, and the weight parameter.

[0016] Embodiments of the present application disclose a direction-aware macro unit layout optimization device for chip design, the device comprising: The setting module is configured to set the population initialization of the evolutionary algorithm. The first calculation module is configured to calculate the fitness score and the candidate solution probability of the candidate solution of the population; The second calculation module is configured to calculate the quality score of each macro; The extraction module is configured to extract visual features from the current layout state by the reinforcement learning agent, and an encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer, and the encoder and a decoder jointly generate a placement probability distribution on a direction position joint action space. The output module is configured to, in the output stage, shield illegal positions by a mask mechanism, and perform softmax normalization on the remaining logic values to output the orientation and placement coordinates of the macro unit.

[0017] The embodiment of the application discloses a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the direction-aware macro unit layout optimization method of the chip design when executing the computer program.

[0018] The embodiment of the application discloses a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the direction-aware macro unit layout optimization method of the chip design when executed by a processor.

[0019] The embodiment of the application has the following advantages: The embodiment of the application proposes a macro unit layout method combining direction awareness, reinforcement learning and evolutionary search, which breaks through the multiple bottlenecks of the existing learning-driven framework. In the layout framework of the reinforcement learning agent, the orientation (rotation and mirror image) and position of the macro unit are first jointly included in the same action space, so that the agent can sample and determine the position and direction of the macro at one time, thereby expanding the exploratory solution space and capturing the synergistic effect between the two. The application proposes a global optimization method based on the evolutionary iteration paradigm. In each iteration, the macro units that have the most significant impact on the overall layout quality are first identified by a quality evaluation function, and these macro units are preferentially selected as the objects of re-placement, ensuring that the optimization process focuses on the layout bottleneck, thereby avoiding the computational waste caused by indiscriminate reconstruction of all macro units in traditional methods. On this basis, the reinforcement learning agent is introduced to perform fine-grained joint optimization of the direction and position of the selected macro units in a local range. Through the combination of "evolution-guided iterative optimization" and "reinforcement learning-driven local refinement", the application realizes the organic unification of global exploration and local reinforcement, which not only maintains the diversity of the solution, but also significantly improves the search efficiency and the final layout quality. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0021] Figure 1 is a whole framework of direction perception driven macro layout of the prior art which combines evolutionary algorithm and reinforcement learning; Figure 2 is a schematic diagram of a direction perception macro cell layout optimization method of a chip design according to an embodiment of the present application; Figure 3 is a schematic diagram of an RL agent according to an embodiment of the present application; Figure 4 is a schematic diagram of input feature visualization according to an embodiment of the present application; Figure 5 Subgraph a of is a schematic diagram of a separate modeling of the prior art; Figure 5 Subgraph b of is a schematic diagram of joint modeling according to an embodiment of the present application; Figure 6 is a schematic diagram of a direction perception macro cell layout optimization device of a chip design according to an embodiment of the present application; Figure 7 is an internal structure diagram of a computer device according to an embodiment. DETAILED DESCRIPTION

[0022] In order to make the technical problems, technical solutions and beneficial effects solved by the embodiments of the present application more clearly understood, the embodiments of the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0023] The present application proposes a whole framework of direction perception driven macro layout which combines evolutionary algorithm (EA) and reinforcement learning (RL) as shown in Figure 1 The core synergistic mechanism of this framework is as follows: the evolutionary algorithm is responsible for maintaining a solution population at the global level, and identifying potential candidate solutions through selection operation; then, the reinforcement learning agent as the core optimization engine, performs accurate local repositioning and orientation optimization on the key macros in the candidate solution. This design makes the algorithm have global exploration ability and local fine optimization efficiency. The new solution after optimization is fed back to the population, and the population is continuously evolved through the updating mechanism of the evolutionary algorithm, so as to realize the significant improvement of the layout quality, and the whole framework is as shown in Figure 1 .

[0024] Referring to Figure 2 , a schematic diagram of an embodiment of a chip design direction-aware macro cell layout optimization method is shown, which can specifically include the following steps: Step S101, initialize the population of the evolutionary algorithm; Step S102, calculate the fitness score and candidate solution probability of the population; Step S103, calculate the quality score of each macro; Step S104, the reinforcement learning agent extracts visual features from the current layout state, and the encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer, and the encoder and the decoder jointly generate a placement probability distribution in the direction position joint action space; Step S105, in the output stage, the reinforcement learning agent shields illegal positions through the mask mechanism, and performs softmax normalization on the remaining logic values, and outputs the orientation and placement coordinates of the macro cell.

[0025] Further applied to the embodiment of the application, the method further comprises: An elite preservation strategy is used to replace the worst solution with a new candidate solution; The fitness score of each candidate solution is calculated by the evolutionary algorithm to screen out potential candidate solutions, and the macro quality score of the screened candidate solutions is calculated to identify key macro cells that form a bottleneck for layout quality; An optimization macro cell identifier list is generated by the evolutionary algorithm, and the optimization macro cell identifier list and the layout state of the candidate solution together constitute the core parameters transmitted by the evolutionary algorithm to the reinforcement learning agent; After receiving the core parameters of the optimization macro cell identifier list, the reinforcement learning agent makes a re-placement decision for the specified macro cell and outputs a joint optimization action containing orientation and coordinates, which is fed back to the evolutionary algorithm as a mutation result to generate new candidate solutions and complete population iteration.

[0026] In a specific embodiment, the method further comprises: The encoder is composed of multiple layers of convolution and down-sampling modules, each layer extracts local spatial patterns through convolution operation, and gradually compresses the spatial resolution through pooling or step convolution; The decoder adopts a symmetric up-sampling structure, each layer first restores the spatial resolution step by step through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer through a jump connection.

[0027] Further applied to the embodiment of the application, the method further comprises: The input tensor of the reinforcement learning agent is obtained by splicing the mask feature and the global state feature along the channel.

[0028] In a specific example, the mask feature includes a line length increment mask, a position mask, an overlap mask, and a boundary mask; and the global state feature includes a layout area and a routing density.

[0029] Specifically, the direction-position joint action space includes a four-dimensional logits tensor with a shape of (B, O, H, W), wherein B represents a batch size, O represents a number of directions corresponding to eight directions {N, S, W, E, FN, FS, FW, FE}, and (H, W) is a grid size of a macro layout.

[0030] In practical applications, the method further includes: A definition reward function of the reinforcement learning agent is generated according to the position and direction, the line length change of the macro unit, the routing congestion intensity, the overlap area of the macro unit, and the weight parameter.

[0031] The evolutionary algorithm part of the application is based on EGPlace, inherits the efficient population management mechanism thereof, and makes two key improvements based on the EGPlace: one is to expand the optimization dimension from a single position optimization to a position-direction joint optimization; and the other is to introduce a reinforcement learning agent in the mutation operation to replace the original greedy relocation strategy, so as to realize intelligent fine adjustment of key macro units. These improvements effectively expand the coverage range of the search space and enhance the quality potential of the solution. The overall process of the algorithm is as follows, mainly including population initialization, selection, mutation, and replacement stages: 1. Population initialization: An initial solution is generated by using a greedy strategy to ensure that the population has a high initial quality.

[0032] 2. Selection operation: First, fitness evaluation is performed by comprehensively considering factors such as half-period line length, congestion, and overlap.

[0033] ; Wherein, represents the fitness score of the candidate solution ; represents the global line length; is the peak congestion index; represents the overlap area between all macro modules; , is a weight parameter.

[0034] Subsequently, a softmax normalized probability sampling strategy is used: ; in, Indicates candidate solutions The probability of being selected. For candidate solutions fitness value, This represents the index of the current candidate solution in the population. It is the index used to traverse each candidate solution in the population during the summation operation of the denominator, and is used to calculate the normalization. This mechanism ensures that high-fitness solutions are more likely to be selected, while preserving population diversity.

[0035] 3. Mutation operation: First, a quality score is calculated for each macro to assess the module's negative impact on layout quality.

[0036] ; in, Macro Quality rating; Contribute to the distance of the connection; Contribute to congestion; The area of ​​overlap with other macros; , These are the width and height of the macro, used for normalization to eliminate the influence of macro modules of different sizes.

[0037] 4. Contribution of connection distance: ; in, For Macro The pin set; Pin indication To the center of the smallest enclosing rectangle of its associated net The average Manhattan distance between them. This parameter characterizes the impact of module location on global HPWL. This value increases significantly if the module is far from its primary connection point.

[0038] 5. Congestion contribution: ; in, For pins The network to which it belongs; This represents the RUDY value of the network; The global maximum RUDY value; The set threshold ratio; This is an indicator function. This item reflects whether the macro is in a severely congested area. If a large number of pins of the macro module fall in a network where the RUDY value exceeds the threshold, its congestion score increases, indicating that the macro module exacerbates local routing bottlenecks.

[0039] Subsequently, a certain proportion (e.g. 50%) of the macros are selected according to the normalized probability, and are repositioned and directionally optimized by an RL agent (reinforcement learning agent). The probability of high-score macros being selected for repositioning is higher, so as to focus on optimizing the layout bottleneck; the probability of low-score macros being selected is still small, so as to avoid falling into a local optimum.

[0040] For the selected macro unit, the RL agent first extracts visual features from the current layout state, including macro unit distribution, RUDY, and multi-channel image information such as boundary and overlap constraint masks. The feature input is input into a neural network structure, which is composed of an encoder and a joint policy decoder: the encoder extracts multi-scale spatial features of the layout layer by layer, and the joint policy decoder generates a placement probability distribution in the direction-position joint action space. In the output stage, the network masks illegal positions (such as out-of-bound or overlapping areas) through a mask mechanism, and performs softmax normalization on the remaining legal logits (logistic values), so as to establish a complete probability distribution in the three-dimensional joint space. The agent then samples an action from the distribution. where represents the orientation of the macro unit, and represents the placement coordinates.

[0041] 6. Population replacement: An elite preservation strategy is adopted, and the worst solution is replaced by a new candidate solution, so as to maintain the population size and diversity.

[0042] 7. Agent update: When the experience buffer is filled, the PPO2 algorithm is used to update the neural network parameters, so as to continuously improve the quality of the agent policy.

[0043] The core advantage of the framework lies in the effective cooperation between the evolutionary algorithm and the reinforcement learning: the EA is responsible for global exploration, and avoids the search from falling into a local optimum; the RL agent realizes efficient local fine optimization through experience learning. This complementary mechanism of exploration and utilization finally achieves a good balance between search efficiency and final layout quality.

[0044] Referring to Figure 3 , an architecture schematic diagram of an RL agent embodiment of an embodiment of the application is shown, In this architecture, the RL agent needs to make decisions according to the cost of the macro in different positions and directions, and is the core of the search quality. The architecture design includes the construction of input features, the modeling of action space, network model, reward design and training method.

[0045] 1) input feature; In the macro placement problem, the RL agent needs to perceive the routing cost, feasibility constraints, and global layout state under different placement directions and positions. To this end, based on the geometry information and pin distribution of the macro, a series of feature tensors obtained through coordinate transformation are constructed. These features can provide the agent with information such as the length increment, overlap cost, boundary legality, and position feasibility in the joint direction-position space, thereby providing a direct basis for the agent's decision-making.

[0046] Specifically, let the upper left corner of the macro be at the grid coordinates , the size be , and the physical scaling ratio be . The pin in the macro center coordinate system is . Through the following steps, the pin coordinates are mapped to the global grid coordinates one by one, and the input features are constructed on this basis: (a) mirror transformation; ; where is a 2x2 mirror matrix corresponding to X-axis flip, Y-axis flip, double-axis flip, or no flip, is the pin coordinates after mirror transformation.

[0047] No flip: ;X flip: ;Y flip: ; Double-axis flip: ; (b) After mirror transformation, the pin is rotated counterclockwise around the macro center by : ; ; ; where is a counterclockwise rotation matrix, is the pin coordinates after rotation.

[0048] (c) To convert to the coordinates with the upper left corner of the macro as the reference point, add half the width and half the height: ; where is the pin coordinates of a macro with the upper left corner as the reference coordinate system.

[0049] (d) Combined with the grid coordinates of the macro , it can be converted to global physical coordinates: ; where Global physical coordinates of the pin.

[0050] (e) Convert to global grid coordinates: ; where, Global grid coordinates of the pin.

[0051] (f) Combine transformation simplification; Mirror matrix and rotation matrix There are 16 forms in total, but after simplification, there are only 8 unique equivalent matrices, which correspond to the 8 orientations (N, S, W, E, FN, FS, FW, FE) in the LEF / DEF specification.

[0052] N: No transformation (R0); S: 180° rotation (R180); W: 90° rotation (R90); E: 270° rotation (R270); FN: Y-axis mirror (MY); FS: X-axis mirror (MX); FW: X-flip + 90° rotation (MX90); FE: Y-flip + 90° rotation (MY90).

[0053] On this basis, a series of mask-based input features are constructed to characterize the placement cost and constraint conditions of macro modules in different positions under different orientations: WireMask is a continuous matrix used to quantify the incremental impact of different positions on HPWL. Since the calculation of HPWL is closely related to the pin position of the macro, it is necessary to combine the pin coordinates after rotation and mirroring. Each orientation corresponds to an independent WireMask, so there are 8 direction-dependent cost mappings.

[0054] PositionMask is used to mark whether the current macro is feasible in a given position, and the main constraints come from the boundary and the position of the placed macro. The mask is a binary matrix, where a value of 1 indicates a legal placement point and a value of 0 indicates an illegal position. Since whether the length and width of the macro are exchanged depends on the rotation and mirroring state, PositionMask only has two types: normal direction (N / FN / S / FS), keeping the original width and height; rotated direction (W / E / FW / FE), length and width are exchanged.

[0055] The overlap mask is used to characterize the overlap area between a newly added macro and an already placed module at different positions, and it is a cost-based supplement to PositionMask. A larger mask value indicates a higher potential overlap cost. Similar to PositionMask, OverlapMask only varies with the orientation combination of the macro's length and width, and therefore there are only two types.

[0056] Boundary mask: Used to prevent macros from being placed outside the boundary; it is a binary matrix.

[0057] When a macro becomes illegal at a certain position due to out-of-bounds access, that position is marked as 0; otherwise, it is marked as 1. The category of BoundMask only varies with the orientation combination of the macro's width and height, therefore there are two categories.

[0058] In addition to the mask mentioned above, two global features are introduced to represent the macro layout state: Layout area (Canvas): Records the distribution positions of macrocells placed on the current chip layout, used to provide space occupancy information; Routing density (Rectangular Uniform Wire Density, RUDY): Characterizes the density distribution of wiring demand at candidate locations and is a commonly used congestion estimation metric.

[0059] By concatenating the mask features (Wire, Position, Overlap, Bound) with the global state features (Canvas, RUDY) along the channels, a 16-channel input tensor is obtained, such as... Figure 4 As shown: 1 Canvas, 1 RUDY, 8 orientation-dependent WireMasks, and 2 channels each for Position, Overlap, and Boundmask. This feature space design can comprehensively characterize the costs and feasibility under different orientations and positions, enabling the agent to effectively capture the coupling effects of rotation, mirroring, and position.

[0060] 2) Actionspace design oriented towards direction awareness; A direct but limited approach is to model the direction and position of the macrocell as independent outputs: such as Figure 5(a) As shown, one branch in the decoder is used to predict the discrete orientation distribution, and the other branch outputs the logits probability map of the position on the placement grid. This modeling approach is called decoupled policy modeling. Its advantage is that it is simple to implement, but it also implies that the direction and position can be selected independently of each other. However, in the macro cell layout task, this assumption often does not hold, because the change of direction will directly affect the pin geometry distribution of the macro and its placement feasibility on the chip grid, thereby causing the direction decision to be unable to reflect the influence on the position legality and the connection topology in real time.

[0061] To overcome this defect, the present application designs a direction-position joint action space, so that the policy network can determine the direction and position coordinates of the macro cell at the same time in one decision, thereby realizing the coordinated optimization of the two in structure. As shown in Figure 5 (b) As shown, the network outputs a logits with a shape of (O, H, W), where O represents the number of orientations, corresponding to 8 orientations {N, S, W, E, FN, FS, FW, FE}, and (H, W) represent the spatial dimensions of the layout grid, respectively. For illegal direction-position combinations (such as situations that cause the macro cell to overlap or exceed the boundary), the network sets their logits to a minimum value through a mask mechanism before softmax normalization, thereby automatically excluding these illegal actions in the sampling stage. Finally, an action (o, x, y) is sampled from the normalized joint distribution, where o represents the direction and (x, y) represents the position coordinates. This action not only satisfies the placement constraints (filtered by the mask), but also realizes the joint optimization of the direction and position in the geometric coupling sense.

[0062] 3) Network architecture; In terms of network architecture, the policy network adopts a U-Net-like symmetric structure similar to EfficientPlace, to fully utilize the multi-scale spatial information and contextual constraint relationship in the layout task. Its overall structure consists of an encoder and a joint policy decoder, as shown in Figure 3 .

[0063] Encoder part: The encoder is composed of multiple layers of convolution and down-sampling modules. Each layer extracts local spatial patterns (such as local congestion areas and inter-macro spacing features) through convolution operations, and gradually compresses the spatial resolution through pooling or stride convolution, thereby forming feature representations at different scales. Shallow features focus on local legality and neighborhood constraints (such as overlap and boundary information), while deep features capture the global topological structure of the layout (such as the main connection direction, congestion distribution, and module cluster relationship). In addition, the encoder fuses time step embedding information in the feature channel, which is used to represent the stage-by-stage state of the layout optimization, enabling the agent to have the ability to perceive the historical adjustment trend in multiple rounds of iteration.

[0064] Joint policy decoder section: The decoder employs a symmetrical upsampling structure. Each layer first recovers the spatial resolution stepwise through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer via skip connections. Subsequently, the fused feature map undergoes further processing through multiple convolutional operations to fully integrate cross-scale semantics and local geometric information. Through this "upsampling + skip connections + convolutional fusion" structure, the decoder can utilize both global layout semantics (from deep encoded features) and retain fine geometric boundary information (from shallow features) when generating action distributions.

[0065] Output and masking mechanism: During the output phase, the network prunes the logits tensor using a masking mechanism, assigning minimal weights to the logits values ​​corresponding to illegal placement points such as out-of-bounds and overlapping points, thereby automatically shielding illegal actions after softmax normalization. Ultimately, a legal placement probability distribution is formed in the (O, H, W) 3D joint space, from which the agent can sample the orientation and position (o, x, y) of macrocells.

[0066] This structural design enables the policy network to make coordinated orientation-position decisions under the constraints of the global layout context. The encoder extracts global semantic information, providing a macroscopic understanding of the overall layout structure and constraint relationships; the decoder focuses on the reconstruction of local regions and fine-grained placement adjustments, ensuring the accuracy of the output in spatial resolution. Simultaneously, the skip connection mechanism establishes direct pathways for multi-layered features during encoding and decoding, allowing global and local information to be dynamically fused and consistently propagated across different scales. Based on this joint policy generation mechanism of multi-scale feature fusion and mask constraints, the agent can achieve efficient and stable joint orientation-position decisions in complex and constraint-rich layout states, thereby generating high-quality macrocell placement schemes.

[0067] 4) Reward design and training methods To ensure consistency between the goals of EA and RL, the evaluation metrics of the evolutionary algorithm are used in the training of the RL agent, and the reward function is defined as follows: ; in, Indicates the location and direction Place the next The change in line length caused by each macrocell is used to characterize the impact of placement on global routing cost. This indicates the wiring congestion intensity at the corresponding location and direction; Measure the area of ​​macrocell overlap caused by corresponding position and orientation. , is a weight parameter for balancing the importance of each indicator in the overall reward.

[0068] Through this unified reward function, the optimization direction of the candidate solution generated by the EA and the RL agent is consistent, thereby realizing the collaborative evolution of both under the same goal.

[0069] In the training method, the PPO2 algorithm is used to update the agent, and the decision-making ability of the agent is continuously improved.

[0070] In the present application, the collaborative optimization between the evolutionary algorithm and the reinforcement learning agent is not realized by a single fitness score in series, but by a structured and bidirectional data flow to realize closed-loop interaction. This interaction mechanism runs through the three key links of initial population generation, candidate solution evaluation and selection, and macro quality evaluation, ensuring the effective coupling of global exploration and local optimization.

[0071] The generation of the initial population provides a high-quality starting point for the entire optimization process. The present application uses a greedy strategy instead of a random strategy to generate the initial population, and the high-quality initial layout scheme generated thereby constitutes the initial value of the "layout state" parameter transmitted by the subsequent EA to the RL. This design ensures that the interaction process starts from an optimized starting point, improving the overall convergence efficiency.

[0072] In the evaluation and selection of candidate solutions, the EA calculates the fitness score of each candidate solution L. The score mainly serves the selection operation within the EA to filter out potential candidate solutions for the subsequent optimization stage.

[0073] Macro quality evaluation is the core bridge connecting the global evaluation of EA and the local optimization of RL. EA performs macro quality scoring on the filtered candidate solutions, which is independent of the fitness score and is specifically used to identify key macro units that form bottlenecks in layout quality. Based on the scoring results, EA generates a list of macro units to be optimized. This list, together with the layout state of the candidate solution, constitutes the core parameter transmitted by EA to RL, accurately defining the specific object to be optimized by RL and its environmental context. After receiving the above-mentioned parameters, the RL agent makes a re-placement decision for the specified macro unit and outputs a joint optimization action containing the heading and coordinates. The action is fed back to the EA as a mutation result to generate new candidate solutions, completing the population iteration.

[0074] By embedding the RL agent into the mutation link of the EA, the present application realizes the deep integration of the global exploration ability of the EA and the local optimization ability of the RL, effectively improving the optimization efficiency and layout quality.

[0075] The key of the present application is to propose a macro cell layout method fusing direction perception, reinforcement learning and evolutionary search, which breaks through the multiple bottlenecks of the existing learning-driven framework. In the layout framework of the reinforcement learning agent, the orientation (rotation and mirror) and position of the macro cell are first jointly included in the same action space, so that the agent can sample and determine the position and direction of the macro at one time, thereby expanding the solvable solution space and capturing the synergistic effect between the two. The present application proposes a global optimization method based on the evolutionary iteration paradigm. In each iteration, the macro cell that has the most significant impact on the overall layout quality is first identified by the quality evaluation function, and these macro cells are preferentially selected as the objects of repositioning, ensuring that the optimization process focuses on the layout bottleneck, thereby avoiding the computational waste caused by indiscriminate reconstruction of all macro cells in traditional methods. On this basis, the reinforcement learning agent is introduced to perform fine-grained joint optimization of the direction and position of the selected macro cell in a local range. Through the combination of "evolution-guided iterative optimization" and "reinforcement learning-driven local refinement", the present application realizes the organic unification of global exploration and local reinforcement, which not only maintains the diversity of solutions, but also significantly improves the search efficiency and the final layout quality.

[0076] It should be noted that, for the method embodiment, in order to simply describe, it is expressed as a series of action combinations, but those skilled in the art should know that the embodiment is not limited by the described action sequence, because according to the embodiment, certain steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiment.

[0077] Referring to Figure 6 , a structural block diagram of a chip design direction perception macro cell layout optimization device embodiment of the present application is shown, which can specifically include the following modules: The setting module 301 is used for setting the population initialization of the evolutionary algorithm; The first calculation module 302 is used for calculating the fitness score and candidate solution probability of the population candidate solution; The second calculation module 303 is used for calculating the quality score of each macro; The extraction module 304 is used for extracting visual features from the current layout state by the reinforcement learning agent. The encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer, and the encoder jointly with the decoder generates a placement probability distribution on the direction and position joint action space; The output module 305 is used for, in the output stage, shielding the illegal position by the mask mechanism, and performing softmax normalization on the remaining logic value, to output the orientation and placement coordinates of the macro cell.

[0078] Preferably, the device further comprises: a replacement module for replacing the worst solution with a new candidate solution using an elitist strategy; a scoring module for calculating the fitness score of each candidate solution by an evolutionary algorithm to filter out potential candidate solutions, and performing macro quality scoring on the filtered candidate solutions to identify key macro cells that form a bottleneck for layout quality; a generation module for generating a list of macro cell identifiers to be optimized by an evolutionary algorithm, which together with the layout state of the candidate solution constitutes the core parameters delivered to the reinforcement learning agent by the evolutionary algorithm; a feedback module for the reinforcement learning agent to make a re-placement decision on the specified macro cell after receiving the core parameters of the list of macro cell identifiers to be optimized, and output a joint optimization action containing orientation and coordinates, which is fed back to the evolutionary algorithm as a mutation result to generate a new candidate solution and complete population iteration.

[0079] Preferably, the device further comprises an encoder and a decoder: The encoder is composed of multiple layers of convolution and down-sampling modules, each layer extracts local spatial patterns through convolution operation, and gradually compresses the spatial resolution through pooling or step convolution; The decoder adopts a symmetric up-sampling structure, each layer first restores the spatial resolution step by step through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer through a skip connection.

[0080] Preferably, the device further comprises: a splicing module for splicing the mask features and global state features along the channel to obtain a 16-channel input tensor of the reinforcement learning agent.

[0081] Preferably, the mask features include line length increment mask, position mask, overlap mask and boundary mask; the global state features include layout area and wiring density.

[0082] Preferably, the direction and position joint action space includes a four-dimensional logits tensor with a shape of (B, O, H, W), where B represents the batch size, O represents the number of orientations corresponding to 8 orientations {N, S, W, E, FN, FS, FW, FE}, and (H, W) is the grid size of the macro layout.

[0083] Preferably, the device further comprises: a generation module for generating a defined reward function of the reinforcement learning agent according to the position and direction, the line length change of the macro cell, the wiring congestion intensity, the macro cell overlap area and the weight parameter.

[0084] The modules in the direction-aware macro cell layout optimization apparatus of the chip design can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so as to be invoked and executed by the processor to perform the operations corresponding to the modules.

[0085] The direction-aware macro cell layout optimization apparatus of the chip design provided above can be used to perform the direction-aware macro cell layout optimization method of the chip design provided in any of the embodiments above, and has the corresponding functions and advantages.

[0086] In one embodiment, a computer device is provided, which includes a medical infusion device, and the internal structure diagram of the computer device can be as shown in Figure 7 The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement a direction-aware macro cell layout optimization apparatus of a chip design. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball, or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0087] Those skilled in the art can understand that Figure 7 The structure shown in the above

[0088] In one embodiment, a computer device is provided, which includes a memory and a processor, and the memory stores a computer program. The processor executes the computer program to implement the steps of the above embodiments: Initialize the population of the evolutionary algorithm; Calculate the fitness score and candidate solution probability of the candidate solution of the population; Calculate the quality score of each macro; The reinforcement learning agent extracts visual features from the current layout state, and an encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer, and the encoder and a decoder generate a placement probability distribution on a direction-position joint action space; In the output stage, the reinforcement learning agent shields illegal positions through a mask mechanism, and performs softmax normalization on the remaining logic values to output the orientation and placement coordinates of the macro cell.

[0089] Preferably, the method further comprises: The quality of the worst solution is replaced by a new candidate solution using an elitist strategy; The fitness score of each candidate solution is calculated by an evolutionary algorithm to screen potential candidate solutions, and the screened candidate solutions are macro quality scored to identify key macro cells that form a bottleneck for layout quality; An optimization macro cell identifier list is generated by an evolutionary algorithm, and the optimization macro cell identifier list and the layout state of the candidate solution together constitute the core parameters transmitted by the evolutionary algorithm to the reinforcement learning agent; After receiving the core parameters of the optimization macro cell identifier list, the reinforcement learning agent makes a re-placement decision for the specified macro cell and outputs a joint optimization action containing orientation and coordinates, which is fed back to the evolutionary algorithm as a mutation result to generate new candidate solutions and complete population iteration.

[0090] Preferably, the method further comprises: The encoder is composed of multiple layers of convolution and down-sampling modules, each layer extracts local spatial patterns through convolution operation, and gradually compresses the spatial resolution through pooling or step convolution; The decoder adopts a symmetric up-sampling structure, each layer first restores the spatial resolution step by step through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer through a jump connection.

[0091] Preferably, the method further comprises: The input tensor of the reinforcement learning agent is obtained by concatenating the mask feature and the global state feature along the channel to obtain a 16-channel input tensor.

[0092] Preferably, the mask feature includes a line length increment mask, a position mask, an overlap mask, and a boundary mask; and the global state feature includes a layout area and a wiring density.

[0093] Preferably, the direction-position joint action space includes a four-dimensional logits tensor with a shape of (B, O, H, W), where B represents the batch size, O represents the number of orientations corresponding to eight orientations {N, S, W, E, FN, FS, FW, FE}, and (H, W) is the grid size of the macro layout.

[0094] Preferably, the method further comprises: According to the position and direction, the line length change of the macro unit, the wiring congestion intensity, the macro unit overlapping area, and the weight parameter, a definition reward function of the reinforcement learning agent is generated.

[0095] In one embodiment, a computer readable storage medium is provided, and a computer program is stored thereon, and the computer program is executed by a processor to implement the steps of the above-mentioned embodiments: The population initialization of the evolutionary algorithm is set; The fitness score and candidate solution probability of the candidate solution of the population are calculated; The quality score of each macro is calculated; The reinforcement learning agent extracts visual features from the current layout state, and the encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer. The encoder and the decoder generate a placement probability distribution in the direction position joint action space. In the output stage, the reinforcement learning agent shields illegal positions through a mask mechanism, and performs softmax normalization on the remaining logical values to output the orientation and placement coordinates of the macro unit.

[0096] Preferably, the method further comprises: An elite preservation strategy is adopted to replace the worst solution with a new candidate solution; The fitness score of each candidate solution is calculated by the evolutionary algorithm to screen out potential candidate solutions. The macro quality score of the screened candidate solutions is used to identify key macro units that form a bottleneck for layout quality; An optimization macro unit identification list is generated by the evolutionary algorithm, and the optimization macro unit identification list and the layout state of the candidate solution jointly constitute the core parameters delivered by the evolutionary algorithm to the reinforcement learning agent; After receiving the core parameters of the optimization macro unit identification list, the reinforcement learning agent makes a re-placement decision for the specified macro unit and outputs a joint optimization action containing orientation and coordinates. The action is fed back to the evolutionary algorithm as a mutation result to generate new candidate solutions and complete population iteration.

[0097] Preferably, the method further comprises: The encoder is composed of multiple layers of convolution and down-sampling modules. Each layer extracts local spatial patterns through convolution operation and gradually compresses spatial resolution through pooling or step convolution; The decoder adopts a symmetric up-sampling structure. Each layer first restores the spatial resolution step by step through bilinear interpolation, and then fuses the result with the encoder features of the corresponding layer through a jump connection.

[0098] Preferably, the method further comprises: The input tensor of the reinforcement learning agent is obtained by splicing the mask feature and the global state feature along the channel.

[0099] Preferably, the mask feature comprises a line length increment mask, a position mask, an overlap mask and a boundary mask; and the global state feature comprises a layout area and a routing density.

[0100] Preferably, the direction-position joint action space comprises a four-dimensional logits tensor with a shape of (B, O, H, W), wherein B represents a batch size, O represents a number of directions corresponding to eight directions {N, S, W, E, FN, FS, FW, FE}, and (H, W) is a grid size of a macro layout.

[0101] Preferably, the method further comprises: According to the position and direction, the line length change of the macro unit, the routing congestion intensity, the overlap area of the macro unit and the weight parameter, a defined reward function of the reinforcement learning agent is generated.

[0102] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other.

[0103] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0104] The embodiments of the present application are described with reference to flowcharts and / or block diagrams according to the method, terminal device (system) and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of the flows and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device realize the functions specified in the flowchart and / or block diagram. Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks Figure 1 The device for realizing the functions specified in one flow or multiple flows and / or blocks

[0105] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flow Figure 1 one or more flows and / or blocks Figure 1 one or more blocks or multiple blocks.

[0107] While preferred embodiments of the application have been described, modifications and variations can be apparent to those skilled in the art once aware of the general underlying concepts. Accordingly, the appended claims are intended to encompass within their scope all adaptations or variations of preferred embodiments of the application.

[0108] Finally, it should be noted that the terms "first", "second", and the like, herein do not necessarily have any actual meaning such as meaning or implying any relative or sequential significance, but are merely used to distinguish one entity or operation from another. Also, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article, or apparatus that includes the stated elements.

[0109] The chip design direction-aware macro cell layout optimization method, the chip design direction-aware macro cell layout optimization apparatus, the computer device, and the computer readable storage medium are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method and the core idea of the present application. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges can be changed, and the above description of the present application should not be understood as a limitation.

Claims

1. A method for optimizing the placement of orientation-aware macrocells in chip design, characterized in that, The method includes: Set the population initialization for the evolutionary algorithm; Calculate the fitness scores and probabilities of candidate solutions in the population; Calculate the quality score for each macro; The reinforcement learning agent extracts visual features from the current layout state. The encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer. The encoder and decoder jointly generate a placement probability distribution in the orientation, position and action space. During the output phase, the reinforcement learning agent uses a masking mechanism to block illegal positions and performs softmax normalization on the remaining logical values ​​to output the orientation and placement coordinates of the macrocells.

2. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The method further includes: An elite-preserving strategy is adopted, replacing the worst-quality solution with a new candidate solution; The fitness score of each candidate solution is calculated by an evolutionary algorithm to screen out potential candidate solutions. The selected candidate solutions are then subjected to macro-quality scoring to identify key macro-units that pose a bottleneck to the layout quality. An evolutionary algorithm generates a list of macrocell identifiers to be optimized. The list of macrocell identifiers to be optimized and the layout state of candidate solutions together constitute the core parameters that the evolutionary algorithm transmits to the reinforcement learning agent. After receiving the core parameters of the list of macrocell identifiers to be optimized, the reinforcement learning agent makes a relocation decision for the specified macrocell and outputs a joint optimization action containing orientation and coordinates. This action is fed back to the evolutionary algorithm as a mutation result to generate new candidate solutions and complete the population iteration.

3. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The method further includes: The encoder consists of multiple convolutional and downsampling modules. Each layer extracts local spatial patterns through convolutional operations and gradually compresses spatial resolution through pooling or stride convolution. The decoder employs a symmetrical upsampling structure. Each layer first recovers the spatial resolution step by step through bilinear interpolation, and then fuses the results with the encoder features of the corresponding layer through skip connections.

4. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The method further includes: By concatenating the mask features and global state features along the channels, a 16-channel input tensor for the reinforcement learning agent is obtained.

5. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The mask features include line length increment mask, position mask, overlap mask, and boundary mask; the global state features include layout area and wiring density.

6. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The orientation and position joint action space includes a four-dimensional logits tensor with the shape (B, O, H, W), where B represents the batch size, O represents the number of orientations, corresponding to 8 orientations {N, S, W, E, FN, FS, FW, FE}, and (H, W) is the grid size of the macro layout.

7. The orientation-aware macrocell layout optimization method for chip design according to claim 1, characterized in that, The method further includes: The definition reward function of the reinforcement learning agent is generated based on the position and orientation, the change in macrocell line length, the wiring congestion intensity, the macrocell overlap area, and the weight parameters.

8. A direction-aware macrocell layout optimization device for chip design, characterized in that, The device includes: The configuration module is used to configure the population initialization for the evolutionary algorithm. The first calculation module is used to calculate the fitness scores and probabilities of candidate solutions in the population. The second calculation module is used to calculate the quality score for each macro. The extraction module is used to extract visual features from the current layout state through a reinforcement learning agent. The encoder in the reinforcement learning agent extracts multi-scale spatial features of the visual feature layout layer by layer. The encoder and decoder jointly generate a placement probability distribution in the orientation, position and action space. The output module is used in the output stage by having the reinforcement learning agent mask illegal positions through a masking mechanism and perform softmax normalization on the remaining logical values ​​to output the orientation and placement coordinates of macrocells.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the orientation-aware macrocell layout optimization method for chip design as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the orientation-aware macrocell layout optimization method for chip design as described in any one of claims 1 to 7.

Images