A chip layout optimization method, system and device based on improved reinforcement learning

Abstract

The present invention belongs to the field of integrated circuit design, and specifically relates to a chip layout optimization method, system, and device based on improved reinforcement learning. The method comprises the following steps: S1: constructing a layout optimization model including a feature encoding module, a feature fusion module, an experience pool, a strategy network, and a value network. The feature encoding module is used to extract the position mask, line mask, and view mask of the netlist file; the feature fusion module extracts local features and global features, and generates comprehensive features; the experience pool, the strategy network, and the value network constitute a reinforcement learning framework. S2: A large number of random netlists are used as sample data for training the layout optimization model. S3: The chart file to be optimized is input into the layout optimization model after training, and the layout optimization model outputs the optimized netlist file, and then generates the corresponding optimized circuit layout diagram according to the netlist file. The present invention solves the problems of low efficiency and insufficient generalization of the existing chip layout optimization method.

Description

Technical Field

[0001] The present invention belongs to the field of integrated circuit design, and specifically relates to a chip layout optimization method, system, and device based on improved reinforcement learning. Background Art

[0002] Chip layout is one of the key tasks in modern chip design, which involves placing millions of circuit modules on a two-dimensional chip canvas. This process is critical to minimizing latency and power consumption. Traditionally, chip layout requires months of intensive work by hardware engineers to produce the layout, which is time-consuming and labor-dependent. Over time, deep reinforcement learning has become an emerging automated tool for solving chip layout problems. In multiple experiments and studies, it has been found that although learning-based methods are still in their early stages, they have been able to produce promising results and significantly advance the chip design process in an automated manner.

[0003] However, existing learning-based methods have some defects. For example, learning-based reinforcement learning methods such as Graph Placement and DeepPR use hypergraphs to represent networks, but hypergraphs have scalability issues in fully encoding network list information. They discard the relative position information of pins, which may lead to inaccurate line length estimation. The MaskPlace method has made great improvements in extracting node information from datasets, but the processing is relatively rough in the subsequent reinforcement learning value network and feature fusion. Only traditional CNN encoders are used in the value network, which has limited ability to capture global dependencies, low training efficiency, and poor generalization ability. In the fusion of global and local features, only the feature information of the last layer is fused, and some important information may be lost during network transmission. Summary of the invention

[0004] In order to solve the problems of low efficiency of existing chip layout optimization methods and insufficient generalization of solutions, the present invention provides a chip layout optimization method, system, and device based on improved reinforcement learning.

[0005] The technical solution provided by the present invention is:

[0006] A chip layout optimization method based on improved reinforcement learning comprises the following steps:

[0007] S1: Construct a layout optimization model including feature encoding module, feature fusion module, experience pool, strategy network and value network.

[0008] The feature encoding module is used to perform pixel-level feature encoding on the nodes, positions and wire mesh features contained in the macro modules of the netlist, and then generate corresponding position masks, wire masks and view masks.

[0009] The feature fusion module includes a first fusion unit, a second fusion unit, and a third fusion unit. The first fusion unit uses a 1×1 convolutional layer and is used to perform feature fusion of position masks and line masks to generate local features. The second fusion unit uses an enhanced layer structure that combines ResNet and Transformer and is used to perform feature fusion of line masks and view masks to generate global features. The third fusion unit is used to first unify the dimensions of local features and global features, and then perform feature fusion in the channel dimension to generate comprehensive features.

[0010] The experience pool, policy network, and value network constitute the reinforcement learning framework. The policy network is used to generate the probabilities of various layout optimization actions that should be taken based on the embedding vector of the input comprehensive features. The value network is used to calculate the incentives and their values ​​corresponding to the updated state of each round of actions based on the embedding vector of the input global features. The experience pool is used to store new states, actions, rewards, and values ​​as experience, and to update the policy network and value network based on the stored experience.

[0011] S2: Obtain a large number of random netlists containing multiple components and use them as sample data to form the required original data set; divide the original data set into a training set and a test volume, and use them to train and test the layout optimization model, and save the model parameters of the layout optimization model that meets the target after training.

[0012] S3: Input the diagram file of the original circuit diagram to be optimized into the trained layout optimization model, and the layout optimization model outputs the optimized netlist file, and then generates the corresponding optimized circuit layout diagram according to the netlist file.

[0013] As a further improvement of the present invention, the second fusion unit includes four feature extraction layers Layer composed of convolutional layers and residual connections in ResNet, and four encoding layers Transformer Encoder; the Layer layers and the Transformer Encoder layers are alternately distributed in the feature extraction module, and each Layer layer is located before each Transformer Encoder layer.

[0014] As a further improvement of the present invention, the position mask is an N×N binary matrix, wherein cells where components can be placed are recorded as 1, and cells where components cannot be placed are recorded as 0. The method for generating the position mask is as follows:

[0015] Create an N×N temporary matrix with all values ​​1, and then create an N×N weight matrix with all values ​​1;

[0016] By traversing the horizontal and vertical coordinates and the length and width of each macromodule, the values ​​of the overlapping places in the temporary matrix are changed to 0;

[0017] The temporary matrix and the weight matrix are multiplied to obtain a position matrix representing the position mask.

[0018] As a further improvement of the present invention, the line mask is an N×N continuous matrix, and the elements in the matrix represent the increase in the wire length of the macro module placed in each unit relative to the previous round position.

[0019] The line mask generation method includes: creating a third matrix with all values ​​​​as 0, and updating the line length value of each position in the third matrix by traversing the horizontal and vertical coordinates and pin information of each macro module; the difference between the current line length value and the original line length value of the previous round is the increase value; wherein, the line length value calculation adopts the half-cycle line length HPWL.

[0020] As a further improvement of the present invention, the view mask is an N×N binary matrix, in which cells where macro modules are already placed are 1, and cells where no macro modules are placed are 0.

[0021] As a further improvement of the present invention, the value network includes a front fully connected layer, a multi-head attention encoding layer, and three rear fully connected layers in sequence. Each functional layer in the value network is connected by a ReLU function.

[0022] Among them, the number of input channels of the front fully connected layer is 1600, and the number of output channels is 64. The number of heads of the multi-head attention encoding layer is 2, and the dimension is 128. The number of input channels of the rear fully connected layer is 64, and the number of output channels is 64.

[0023] As a further improvement of the present invention, the policy network includes two convolutional layers, a transformation layer, and a Softmax normalization layer. The process of the policy network outputting the corresponding action probability x according to the embedded vector input of the comprehensive feature input is as follows:

[0024] The input feature vector input is divided into three paths, two of which are sliced ​​and reconstructed at different scales to obtain coarse-grained features F1 and fine-grained features F2, and the other is used to extract mask features to obtain mask vectors mask. The coarse-grained feature F1 is processed by a convolution layer using coarse convolution to obtain the first feature H1. The fine-grained feature F2 is processed by a convolution layer with a multi-head attention module to obtain the second feature H2. After the first feature H1 and the second feature H2 are concatenated, they are merged by the transformation layer and transformed into a fused feature matrix FS1 of a specified dimension. The fused feature matrix FS1 is masked using the mask vector mask to obtain the action feature matrix A1. The action feature matrix A1 is processed by the Softmax normalization layer to obtain the action probability x.

[0025] As a further improvement of the present invention, in the reinforcement learning framework, the action space is each cell in the N×N circuit layout diagram.

[0026] The half-period line length HPWL is used as a reward; the state transition function includes the following three processes:

[0027] Actions are selected by the policy network and executed in the virtual environment, which returns new states and rewards.

[0028] The new states, actions, and rewards are stored in the experience pool as experience, and the policy network and value network are updated based on the stored experience.

[0029] The reset condition is to place all components, and the accumulated rewards will be cleared after the reset condition is met.

[0030] The present invention also includes a chip layout optimization system based on improved reinforcement learning, in which a layout optimization model trained in the aforementioned chip layout optimization method based on improved reinforcement learning is run; the layout optimization model is used to generate an optimized circuit layout diagram based on an input diagram file.

[0031] The present invention also includes a chip layout optimization device, which includes a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, the steps of the chip layout optimization method based on improved reinforcement learning as described above are implemented; and then the trained layout optimization model is used to optimize the circuit layout of the input large integrated circuit design netlist file.

[0032] The technical solution provided by the present invention has the following beneficial effects:

[0033] The present invention transforms the circuit space layout into a spatial optimization problem for placing components in the canvas space, and performs iterative optimization through reinforcement learning methods. The scheme captures the feature information in the circuit netlist file through position masks, line masks and view masks, and introduces an optimized ResNet+Transformer network structure in the process of extracting global features. In addition, the present invention also achieves the effective fusion of global and local features by coordinating feature dimensions at different levels. This method can not only capture the information that may be lost during feature propagation, but also enhance the feature extraction capability of the model, significantly enrich and improve the feature representation, and thus obtain a more superior layout result.

[0034] The solution provided by the present invention allows the GPU to enhance the model's ability to capture various information through parallel computing, improve the model's stability, and speed up the training and reasoning process. Compared with the prior art, the present invention has significantly improved key performance indicators, such as being able to minimize the half-circuit length HPWL of the circuit layout, and significantly speeding up the model training and layout reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flowchart of the steps of the chip layout optimization method based on improved reinforcement learning provided in Example 1 of the present invention.

[0036] Figure 2 This is a model architecture diagram of the layout optimization model constructed in Example 1 of the present invention.

[0037] Figure 3 This is a module schematic diagram of the second fusion unit designed based on Swin Transformer and ResNet18 in Example 1 of the present invention.

[0038] Figure 4 This is an architectural diagram of the value network in the layout optimization model of Example 1 of the present invention.

[0039] Figure 5 This is an architecture diagram of the policy network in the layout optimization model of Example 1 of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] Example 1

[0042] This embodiment provides a chip layout optimization method based on improved reinforcement learning, such as Figure 1 As shown, it includes the following steps:

[0043] S1: Construct a Figure 2 The layout optimization model shown includes a feature encoding module, a feature fusion module, an experience pool, a strategy network and a value network.

[0044] In the layout optimization model, the feature encoding module is used to perform pixel-level feature encoding on the nodes, positions and wire mesh features contained in the macro modules of the netlist, and then generate the corresponding position masks, wire masks and view masks.

[0045] Specifically, the position mask is an N×N binary matrix, in which cells where components can be placed are marked as 1, and cells where components cannot be placed are marked as 0. The purpose of the position mask is to ensure that there is no overlap between modules, that is, to satisfy the overlap constraint, and to learn the relationship between position and length. In this embodiment, the method for generating the position mask is as follows:

[0046] First, create an N×N temporary matrix with all values ​​1, and then create an N×N weight matrix with all values ​​1. Then, by traversing the horizontal and vertical coordinates and the length and width of each macro module, the values ​​of the overlapping places in the temporary matrix are changed to 0. Finally, the temporary matrix and the weight matrix are multiplied to obtain the position matrix representing the position mask. This embodiment generates a position matrix through a temporary matrix and a weight matrix, and then the position matrix can be synchronously updated while the temporary matrix is ​​searched in real time.

[0047] The line mask is an N×N continuous matrix, and the elements in the matrix represent the increase in the wire length of the macromodule placed in each unit relative to the previous round position. The purpose of designing the line mask in this embodiment is to find the best position with the smallest increase in line length. Specifically, the method for generating the line mask includes: creating a third matrix with all values ​​​​as 0, and updating the line length values ​​​​of each position in the third matrix by traversing the horizontal and vertical coordinates and pin information of each macromodule; the difference between the current line length value and the original line length value of the previous round is the increase; wherein, the line length value is calculated using the half-circle line length HPWL.

[0048] The view mask is an N×N binary matrix, in which cells where macro modules are already placed are 1, and cells where no macro modules are placed are 0. The purpose of the view mask in this embodiment is to determine whether each grid cell is occupied by a module.

[0049] Combination Figure 2It can be seen that the feature fusion module includes a first fusion unit, a second fusion unit and a third fusion unit. In this embodiment, the first fusion unit adopts a 1×1 convolution layer, and is used to perform feature fusion of the position mask and the line mask to generate local features. The second fusion unit adopts an enhancement layer structure that combines ResNet and Transformer, and is used to perform feature fusion of the line mask and the view mask to generate global features. The third fusion unit is used to first unify the dimensions of the local features and the global features, and then perform feature fusion in the channel dimension to generate comprehensive features.

[0050] This example is based on the Swin Transformer framework, combining Swin Transformer with ResNet18 to create a Figure 3 The novel ResNet+Transformer architecture shown in the figure is used in this embodiment to perform feature fusion of line masks and view masks. Specifically, the second fusion unit includes four feature extraction layers Layer1 to 4 composed of convolutional layers and residual connections in ResNet, and four encoding layers Transformer Encoder; Layer layers and Transformer Encoder layers are alternately distributed in the feature extraction module, and each Layer layer is located before each Transformer Encoder layer.

[0051] The second fusion module designed in this embodiment incorporates the Transformer module layer into the convolution layer of ResNet, thereby enhancing the feature extraction capability of the model. ResNet is good at extracting spatial features through convolutional layers and residual connections, while capturing local and global information. In contrast, Transformer is good at processing sequence data, and effectively simulates long-distance dependencies through its self-attention mechanism. This embodiment can explore intricate relationships by flattening image features into sequences and inputting them into the transformer. The combination of Swin Transformer and ResNet18 in this embodiment is particularly effective for processing multimodal data. ResNet can process image data, while Transformer can manage text data. The feature fusion of these two models contributes to efficient multimodal data modeling, and the Transformer's multi-head self-attention mechanism also provides detailed feature representation and enhances overall expressiveness.

[0052] In the layout optimization model, the experience pool, policy network, and value network constitute a reinforcement learning framework. The policy network is used to generate the probabilities of various layout optimization actions that should be taken based on the embedding vector of the input comprehensive features. The value network is used to calculate the incentives and their values ​​corresponding to the updated states of each round of actions based on the embedding vector of the input global features. The experience pool is used to store new states, actions, rewards, and values ​​as experience, and to update the policy network and value network based on the stored experience.

[0053] In this embodiment, Figure 4 As shown in the figure, the value network includes a front fully connected layer Linear1, a multi-head attention encoding layer Multi Head Attention, and three rear fully connected layers Linear2 to Linear4. Each functional layer in the value network is connected through the ReLU function. Among them, the number of input channels of the front fully connected layer Linear1 is 1600, and the number of output channels is 64. The number of heads of the multi-head attention encoding layer is 2, and the dimension is 128. The number of input channels of the rear fully connected layers Linear2 to Linear4 is 64, and the number of output channels is 64.

[0054] like Figure 5 The strategy network provided in this embodiment includes two convolutional layers, a transformation layer, and a Softmax normalization layer. The strategy network outputs the corresponding action probability x according to the embedded vector input of the input comprehensive feature as follows:

[0055] The input feature vector input is divided into three paths, two of which are sliced ​​and reconstructed at different scales to obtain coarse-grained features F1 and fine-grained features F2, and the other is used to extract mask features to obtain mask vectors mask. The coarse-grained feature F1 is processed by a convolution layer using coarse convolution to obtain the first feature H1. The fine-grained feature F2 is processed by a convolution layer with a multi-head attention module to obtain the second feature H2. After the first feature H1 and the second feature H2 are concatenated, they are merged by the transformation layer and transformed into a fused feature matrix FS1 of a specified dimension. The fused feature matrix FS1 is masked using the mask vector mask to obtain the action feature matrix A1. The action feature matrix A1 is processed by the Softmax normalization layer to obtain the action probability x.

[0056] S2: Obtain a large number of random netlists containing multiple components and use them as sample data to form the required original data set; divide the original data set into a training set and a test volume, and use them to train and test the layout optimization model, and save the model parameters of the layout optimization model that meets the target after training.

[0057] In this embodiment, before training, it is necessary to preset the number of test rounds Epoch, the score obtained by the action score, the half-circle line length hpwl, the custom loss cost, the time spent on each test round, etc. During the training phase, the policy network and the value network are updated at each time point. When updating the value network, stop the gradient back-stepping of the global feature embedding to prevent interference with the policy network. In the testing phase, the probability matrix will be obtained from the policy network in each round, and the actions will be sampled according to the probability matrix. If the threshold is exceeded, the actions are uniformly sampled, and the corresponding values ​​are obtained from the line mask, and the half-circle field line length is estimated before executing these actions.

[0058] In addition, the reinforcement learning framework in the constructed layout optimization model needs to be initialized before training, including setting the state space, setting the action space, setting the reward function, setting the state transfer function, and setting the reset condition. In reinforcement learning, the state space refers to the range of states that a component can reach. Generally, it is set to a discrete state. In this embodiment, the state space contains two states: Unplaced modules that are not laid out and Placed modules that have been laid out. Among them, whether the module has completed the layout is evaluated by state features such as Width, Height, Pin Offset, and Pin to Net Connection. In reinforcement learning, the action space is the range of all actions that a component can make. In this embodiment, the chip canvas is regarded as a grid and divided into N×N units, thereby generating N×N possible actions, that is, randomly placing the unlaid module in any unit where no module is placed.

[0059] In reinforcement learning, the reward function refers to the feedback signal received by the agent when it achieves a goal in the environment, which directly affects the learning process of the agent. In this embodiment, the half-circle line length HPWL is regarded as a reward because the line length is the main optimization target among different performance indicators. This is different from the prior art, which weights the half-circle line length and congestion as rewards and adjusts the weight coefficient as an additional hyperparameter. Specifically, a dense reward is achieved by defining a local HPWL, which only calculates the currently placed pins. For example, the partial HPWL after the t-1 step is defined as HPWL(t-1), and HPWL(t) is calculated after taking action. The reward for the t step is r(t)=HPWL(t-1)-HPWL(t), which is the opposite of the increase in HPWL.

[0060] In the reinforcement learning of this embodiment, the state transfer function includes the following three processes:

[0061] Actions are selected by the policy network and executed in the virtual environment, which returns new states and rewards.

[0062] The new states, actions, and rewards are stored in the experience pool as experience, and the policy network and value network are updated based on the stored experience.

[0063] The reset condition is when all components are placed, and the accumulated rewards are reset to zero after the reset condition is met. That is, state[i,0]>=placed_num_macro-1.

[0064] S3: Input the diagram file of the original circuit diagram to be optimized into the trained layout optimization model, and the layout optimization model outputs the optimized netlist file, and then generates the corresponding optimized circuit layout diagram according to the netlist file.

[0065] The chip layout optimization method based on improved reinforcement learning provided in this embodiment converts the circuit space layout into a spatial optimization problem for placing components in the canvas space, and iterates the optimization through the reinforcement learning method. The scheme of this embodiment captures the feature information in the circuit netlist file through position masks, line masks and view masks, and introduces the optimized ResNet+Transformer network structure in the process of extracting global features. In addition, the scheme of this embodiment also realizes the effective fusion of global and local features by coordinating feature dimensions at different levels. This method can not only capture the information that may be lost during feature propagation, but also enhance the feature extraction capability of the model, significantly enrich and improve the feature representation, and thus obtain a more superior layout result.

[0066] The solution provided in this embodiment can allow the GPU to enhance the model's ability to capture various information through parallel computing, improve the model's stability, and speed up the training and reasoning process. Compared with the prior art, the solution of this embodiment has significant improvements in key performance indicators, such as being able to minimize the half-circuit length HPWL of the circuit layout, and significantly speeding up the model training and layout reasoning.

[0067] Example 2

[0068] On the basis of the scheme in Example 1, this embodiment further provides a chip layout optimization system based on improved reinforcement learning, in which a layout optimization model trained in the chip layout optimization method based on improved reinforcement learning in Example 1 is run; the layout optimization model is used to generate an optimized circuit layout diagram based on an input diagram file.

[0069] In order to implement the solution of Example 1, this embodiment also provides a chip layout optimization device, which includes a memory, a processor, and a computer program stored in the memory. When the processor executes the computer program, the steps of the chip layout optimization method based on improved reinforcement learning in Example 1 are implemented, and then the required chip layout optimization system is created, and the circuit layout optimization is performed on the input netlist file of the large integrated circuit design.

[0070] The chip layout optimization device provided in this embodiment is essentially a computer device, which can take various forms, including an embedded chip or module, or a general-purpose data processing device, such as an intelligent terminal capable of executing a program, a tablet computer, a laptop computer, a desktop computer, a rack server, a blade server, a tower server or a cabinet server (including an independent server, or a server cluster consisting of multiple servers), etc.

[0071] The computer device of this embodiment includes at least but is not limited to: a memory and a processor that can be connected to each other through a system bus. The memory (i.e., a readable storage medium) includes a flash memory, a hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device.

[0072] In other embodiments, the memory may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the computer device. Of course, the memory may also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory is generally used to store the operating system and various application software installed on the computer device. In addition, the memory may also be used to temporarily store various data that have been output or are to be output.

[0073] The processor may be a central processing unit (CPU), a graphics processing unit (GPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor is generally used to control the overall operation of a computer device. In this embodiment, the processor is used to run program codes stored in a memory or process data.

[0074] Simulation test

[0075] In order to verify the performance of the chip layout optimization method based on improved reinforcement learning provided by the present invention, the technicians conducted a simulation experiment and compared the scheme of the present invention with various existing schemes. The experimental content is as follows

[0076] 1. Experimental environment and parameter settings

[0077] In this experiment, the software and hardware environment used to train the network is: Ubuntu 20.04LTS operating system, Python3.10.3, PyTorch 1.11, CUDA 11.3, Intel(R)Xeon(R)Gold 6226CPU, 125GB RAM, NVIDIARTX 3090GPU, 24GB RAM.

[0078] The Adam optimizer is used for model training in the experiment. The Adam optimizer combines adaptive learning rate and momentum to effectively adapt to the gradient changes between different parameters. Initially, Adam adaptively adjusts the learning rate, using a larger learning rate in the early stages of training to accelerate convergence, and then gradually reduces the learning rate in the later stages to refine the search for the optimal solution. In addition, Adam integrates momentum by accumulating the squared gradients of previous updates, thereby adjusting the direction and size of parameter updates. This mechanism helps the optimizer avoid local minima and increases the possibility of finding the global optimum.

[0079] 2. Control group and data set

[0080] This experiment used the solution RTplace of the present invention as the experimental group. In order to accurately evaluate the performance of the solution of the present invention, the technicians also compared the solution of the present invention with several most advanced circuit layout optimization methods. The control group included DREAMPlace, DeepPR and MaskPlace.

[0081] Each method is evaluated using various public circuit designs as samples and evaluated according to their respective experimental settings. The optimized circuits are selected from public datasets, including the widely used ISPD2005 benchmark suite and the Ariane RISC-V CPU design.

[0082] 3. Experimental data and analysis:

[0083] The optimization effects of the scheme of the present invention and the control group method on various sample circuits are shown in the following table:

[0084] Table 1: HPWL results of each macro circuit after layout optimization using different schemes

[0085]

[0086]

[0087] Analyzing the data in the above table, we can find that:

[0088] The lower the HPWL, the better the performance. The layout optimization performance of RTplace provided by the present invention in each macro circuit is better than other methods, and only in one case is it worse than Maskplace, which reflects the superiority of the solution of the present invention.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A chip layout optimization method based on improved reinforcement learning, characterized in that: It includes the following steps: S1: Build a layout optimization model including feature encoding module, feature fusion module, experience pool, strategy network and value network; The feature encoding module is used to perform pixel-level feature encoding on the nodes, positions and wire mesh features contained in the macromodule of the netlist, and then generate corresponding position masks, wire masks and view masks; The feature fusion module includes a first fusion unit, a second fusion unit and a third fusion unit; the first fusion unit adopts a 1×1 convolution layer and is used to perform feature fusion on the position mask and the line mask to generate a local feature; the second fusion unit adopts an enhancement layer structure combining ResNet and Transformer and is used to perform feature fusion on the line mask and the view mask to generate a global feature; the third fusion unit is used to first unify the dimensions of the local features and the global features, and then perform feature fusion on the channel dimension to generate a comprehensive feature; The experience pool, policy network and value network constitute a reinforcement learning framework; the policy network is used to generate the probabilities of various layout optimization actions that should be taken according to the embedded vector of the input comprehensive features; the value network is used to calculate the incentives and their values ​​corresponding to the states updated according to each round of actions according to the embedded vector of the input global features; the experience pool is used to store new states, actions, rewards and values ​​as experience, and update the policy network and value network according to the stored experience; S2: obtaining a large number of random netlists containing multiple components and using them as sample data to form the required original data set; dividing the original data set into a training set and a test volume for training and testing the layout optimization model, and saving the model parameters of the layout optimization model that meets the target after training; S3: Input the diagram file of the original circuit diagram to be optimized into the trained layout optimization model, and the layout optimization model outputs the optimized netlist file, and then generates the corresponding optimized circuit layout diagram according to the netlist file.

2. The chip layout optimization method based on improved reinforcement learning according to claim 1, characterized in that: The second fusion unit includes four feature extraction layers Layer composed of convolutional layers and residual connections in ResNet, and four encoding layers Transformer Encoder; the Layer layers and the Transformer Encoder layers are alternately distributed in the feature extraction module, and each Layer layer is located before each Transformer Encoder layer.

3. The chip layout optimization method based on improved reinforcement learning according to claim 2, characterized in that: The position mask is an N×N binary matrix, wherein cells where components can be placed are marked as 1, and cells where components cannot be placed are marked as 0. The method for generating the position mask is as follows: Create an N×N temporary matrix with all values ​​1, and then create an N×N weight matrix with all values ​​1; By traversing the horizontal and vertical coordinates and the length and width of each macromodule, the values ​​of the overlapping places in the temporary matrix are changed to 0; The temporary matrix and the weight matrix are multiplied to obtain a position matrix representing the position mask.

4. The chip layout optimization method based on improved reinforcement learning according to claim 2, characterized in that: The line mask is an N×N continuous matrix, and the elements in the matrix represent the increase in the wire length of the macromodule placed in each unit relative to the position of the previous round; The line mask generation method includes: creating a third matrix whose values ​​are all 0, and updating the line length value of each position of the third matrix by traversing the horizontal and vertical coordinates and pin information of each macro module; the difference between the current line length value and the original line length value of the previous round is the added value; Among them, the line length value is calculated using the half-circle line length HPWL.

5. The chip layout optimization method based on improved reinforcement learning according to claim 2, characterized in that: The view mask is an N×N binary matrix, in which cells where macroblocks are already placed are 1, and cells where no macroblocks are placed are 0.

6. The chip layout optimization method based on improved reinforcement learning according to claim 1, characterized in that: The value network includes a front fully connected layer, a multi-head attention encoding layer, and three rear fully connected layers in sequence; each functional layer in the value network is connected by a ReLU function; The number of input channels of the front fully connected layer is 1600, and the number of output channels is 64; the number of heads of the multi-head attention encoding layer is 2, and the dimension is 128; the number of input channels of the rear fully connected layer is 64, and the number of output channels is 64.

7. The chip layout optimization method based on improved reinforcement learning according to claim 1, characterized in that: The strategy network includes two convolutional layers, a transformation layer, and a Softmax normalization layer. The strategy network outputs the corresponding action probability x according to the embedded vector input of the input comprehensive feature as follows: The input feature vector input is divided into three paths, two of which are sliced ​​and reconstructed at different scales to obtain coarse-grained features F1 and fine-grained features F2, and the other is used to extract mask features to obtain mask vectors mask; the coarse-grained feature F1 is processed by a convolution layer using a rough convolution to obtain a first feature H1; the fine-grained feature F2 is processed by a convolution layer with a multi-head attention module to obtain a second feature H2; the first feature H1 and the second feature H2 are concatenated and merged by a transformation layer to be transformed into a fused feature matrix FS1 of a specified dimension; the fused feature matrix FS1 is masked using a mask vector mask to obtain an action feature matrix A1; The action feature matrix A1 is processed by the Softmax normalization layer to obtain the action probability x.

8. The chip layout optimization method based on improved reinforcement learning according to claim 1, characterized in that: In the reinforcement learning framework, the action space is each cell in the N×N circuit layout diagram; The half-period line length HPWL is used as a reward; the state transition function includes the following three processes: The policy network selects actions and executes them in the virtual environment, which returns new states and rewards. Store the new states, actions, and rewards in the experience pool as experience, and update the policy network and value network based on the stored experience; The reset condition is to place all components, and the accumulated rewards will be cleared after the reset condition is met.

9. A chip layout optimization system based on improved reinforcement learning, characterized in that: A layout optimization model trained in the chip layout optimization method based on improved reinforcement learning as described in any one of claims 1 to 8 is run therein; the layout optimization model is used to generate an optimized circuit layout diagram based on an input chart file.

10. A chip layout optimization device, characterized in that: It includes a memory, a processor and a computer program stored in the memory, and is characterized in that: when the processor executes the computer program, the steps of the chip layout optimization method based on improved reinforcement learning as described in any one of claims 1 to 8 are implemented, and then the circuit layout optimization of the input large-scale integrated circuit design netlist file is optimized using the trained layout optimization model.

Images