3D-IC thermal simulation method based on self-attention and operator learning

Abstract

The invention belongs to the field of 3D integrated circuits, and provides a 3D-IC thermal simulation method based on self-attention and operator learning. According to the method, dimension raising transformation of an input observation value is realized through a full-connection neural network P, and then a feature map Vt (x) is obtained through iterative U-Fourier layer processing. Thirdly, calculating a space attention map As and a channel attention map Ac through a self-attention layer, and generating an attention enhancement feature map Vt'x; and then, projecting the attention-enhanced feature map back to an original space by using a full-connection neural network Q to obtain a thermal simulation result z (x). And finally, through transfer learning training, pre-training the model M1 by using a large amount of low-resolution data, and then performing fine tuning by using a small amount of high-resolution data so as to improve the prediction performance of the model. The method is high in calculation efficiency and good in prediction precision, reduces dependence on high-precision data, and is suitable for thermal simulation of a 3D integrated circuit.

Description

Technical Field

[0001] The present invention relates to the field of 3D integrated circuits and provides a 3D-IC thermal simulation method based on self-attention and operator learning. Background Art

[0002] With the rapid development of 3D integrated circuit (IC) technology, thermal management issues have become increasingly important. Compared with traditional 2D and 2.5D designs, 3D integration technology has a higher power density, which means that heat is more likely to accumulate, thus forming hot spots, affecting the performance and life of the chip. The high power density of 3D ICs makes the heat dissipation path complicated, and it is difficult for heat to be effectively discharged through traditional heat dissipation methods, especially in multi-layer stacking and tightly packaged designs, where local heat is easily accumulated. This not only reduces the performance of the chip, but may also cause irreversible damage. In addition, the uneven thermal conductivity between different layers in 3D ICs makes the heat diffusion path and heat dissipation capacity between layers vary greatly, exacerbating the difficulty of overall heat dissipation. Traditional thermal management methods are difficult to effectively control temperature distribution, especially in the case of high-density layout and complex packaging, which makes hot spots more likely to occur, affecting the long-term reliability and stability of the chip. The difference in thermal resistance between different materials inside the chip will also lead to an increase in temperature gradients, further increasing the complexity of thermal management.

[0003] Traditional thermal simulation methods, such as the finite element method (FEM) and the finite difference method (FDM), are usually used to solve the three-dimensional heat conduction equation. These methods rely on fine meshing and complex partial differential equation solutions to obtain temperature distribution, but due to the fineness of the meshing and the complexity of the solution process, the computational cost is high and time-consuming. This high computational complexity makes it difficult for these methods to adapt to the rapid iteration and optimization requirements in modern IC design, especially when the physical design stage is frequently changed, the simulation efficiency is particularly insufficient. As the design complexity increases, the number of grids grows rapidly, resulting in an exponential increase in the consumption of computing resources. In addition, since FEM and FDM rely on numerical solutions, every detail must be accurately modeled and calculated, so when dealing with complex 3D structures, it often takes a lot of time and computing resources, which makes these methods very inefficient when exploring and optimizing the design space. Therefore, 3D integrated circuit design requires a more efficient, fast and accurate thermal simulation method to adapt to the rapid iteration and ever-changing needs of modern chip design.

[0004] In recent years, artificial intelligence has been widely used in the field of thermal simulation, and machine learning-based methods have gradually become an alternative. For example, the Fourier neural operator (FNO) converts partial differential equations (PDEs) into the frequency domain for solution through fast Fourier transform (FFT), which significantly improves computational efficiency in some cases. However, these methods have two main challenges:

[0005] First, they have difficulty in effectively capturing high-frequency information in complex temperature distributions, especially in the details of hotspot areas;

[0006] Secondly, the training of these methods requires a large amount of high-precision data, and for complex systems such as 3D ICs, the cost of obtaining this data is very high. To obtain enough high-precision data, a large number of high-resolution simulations are usually required, which is time-consuming and requires powerful computing resources, becoming a major challenge.

[0007] In order to solve these problems, the present invention proposes a new thermal modeling framework based on self-attention mechanism and U-Net Fourier neural operator (SAU-FNO). By combining the self-attention mechanism and the U-Net architecture, the problems of high-frequency information loss and low computational efficiency in traditional methods are overcome, and the accuracy and efficiency of 3D IC thermal simulation are greatly improved. By introducing the self-attention mechanism, the model can better capture long-distance heat transfer relationships, and combined with the multi-scale feature extraction capability of U-Net, it ensures that the model has high flexibility in processing global and local information. This not only improves the prediction accuracy of the model, but also makes the thermal simulation calculation more efficient, meeting the needs of modern chip design for efficient simulation tools. Summary of the invention

[0008] The purpose of the present invention is to use a combination of self-attention mechanism and U-Net architecture to effectively overcome the problems of high-frequency information loss and low computational efficiency in traditional methods, and improve the accuracy and efficiency of 3D-IC thermal simulation. By introducing the self-attention mechanism, the model can better capture long-distance heat transfer relationships, and combined with the multi-scale feature extraction capability of U-Net, it ensures that the model has high flexibility in processing global and local information. This not only improves the prediction accuracy of the model, but also makes the thermal simulation calculation more efficient, meeting the needs of modern chip design for efficient simulation tools.

[0009] In order to achieve the above purpose, the present invention adopts the following technical means:

[0010] A 3D-IC thermal simulation method based on self-attention and operator learning includes the following steps:

[0011] Step 1: Dimensional transformation: Use a fully connected neural network P to lift the input observation a(x) to a higher-dimensional space;

[0012] Step 1.1, the input observation value a(x) is transformed P through the neural network to obtain the improved high-dimensional space representation;

[0013] Step 1.2: Use the high-dimensional space representation obtained in step 1.1 as input for subsequent processing.

[0014] Step 2, U-Fourier layer iteration: for the dimension-enhanced input v l0 (x) Apply an iterative Fourier layer, followed by an iterative U-Fourier layer, to obtain v mM (x), that is, the feature map V t (x);

[0015] Step 2.1, Initialize v l0 (x) is the input representation after dimensionality increase;

[0016] Step 2.2: v lL (x) is iterated through the U-Fourier layer to obtain v m0 (x);

[0017] Step 2.3, v mk (x) is iterated through the U-Fourier layer to obtain v m(k+1) (x), k=0, 1, .., M-1.

[0018] Step 2.3.1: In each U-Fourier layer, for the input function sequence v mk (x) Apply convolution kernel K to obtain the intermediate result Kv mk (x);

[0019] Step 2.3.2: For the input function sequence v mk (x) Apply U-Net to obtain the intermediate result Uv mk (x);

[0020] Step 2.3.3: Add the intermediate results obtained in step 2.3.1 and step 2.3.2 to get Kv mk (x)+Uv mk (x);

[0021] Step 2.3.4: v mk (x) multiplied by the learnable weight W and the result of step 2.3.3, we get Kv mk (x)+Uv mk (x)+Wv mk (x);

[0022] Step 2.3.5: Apply the activation function σ to the result of step 2.3.4 to obtain the updated function sequence v mk+1 (x);

[0023] Step 2.3.6: Repeat steps 2.3.1 to 2.3.5 until v is obtained after M iterations. mM (x), that is, the feature map V t (x).

[0024] Step 3: Self-attention layer: feature map V processed by U-Fourier layer t (x) Perform 1×1 convolution to calculate the spatial attention map A s And channel attention map A c , by element-wise multiplication Combined with A s and A c , get the attention enhanced feature map V t '(x);

[0025] Step 3.1, V t (x) Perform 1×1 convolution to calculate the channel attention map A c :

[0026] A c =W h V t (x)

[0027] Step 3.2: V t (x) Perform 1×1 convolution to calculate the spatial attention map A s :

[0028]

[0029]

[0030] Q=W q V t (x)

[0031] K=W k V t (x)

[0032] Where exp() is the exponential function, where s ij represents the i-th query feature vector Q i and the jth key feature vector K j The similarity or correlation between i is an eigenvector in Q, Indicates Q i The transpose of K j is a eigenvector in K, N represents the key eigenvector K j The number of h , W q , and W k are learnable parameters, representing vectors of values, queries, and keys, respectively;

[0033] Step 3.3: Element-wise multiplication Combined with A c and A s, get the attention enhanced feature map V t '(x):

[0034]

[0035] Step 4: Feature map projection: Use the fully connected neural network Q to project the enhanced feature map V t '(x) is projected back to the original space to obtain the thermal simulation result z(x);

[0036] Step 4.1: V t '(x) is used as input and the projection result z(x) is obtained by transforming Q through a fully connected neural network;

[0037] Step 4.2: Output z(x) as the final thermal simulation result.

[0038] Step 5: Transfer learning training: First use a large amount of low-resolution data D l =X l , Y l Pre-trained model M l , and then use a small amount of high-resolution data D h =X h , Y h Fine-tune the pre-trained model M1 to obtain model M h .

[0039] Step 5.1: Use data D l =X l , Y l Pre-trained low-resolution model M l , where Y1 is the prediction obtained by applying a low-resolution model M1 with learnable parameters to the input X1, using the L2 loss function to optimize the parameters of model M1:

[0040]

[0041] Where N is the number of samples, T true，i represents the true value of sample i, T pred，i represents the prediction of model M1. After training with low-precision data, model M1 builds l , mapping to Y1;

[0042] Step 5.2: Use data D h =X h , Y h For the pre-trained model M l Fine-tune to get the final model M h ;

[0043] Use data D h =X h, Y h Fine-tune the low-resolution model M l Get M h , where Y h is to transform a high-resolution model M with learnable parameters into h Applied to input X h The obtained predictions are optimized using the L2 loss function. h Parameters:

[0044]

[0045] Where N is the number of samples, T true，i represents the true value of sample i, T pred，i Represents model M h After fine-tuning M using high-precision data l Then get M h , Model M h Established from X h To Y h This completes the modeling of a more accurate predictive temperature model.

[0046] Because the present invention adopts the above technical means, it has the following beneficial effects:

[0047] 1. Significantly improved computational efficiency: Compared with traditional finite element methods (such as COMSOL and MTA), SAU-FNO achieves 842 times acceleration in thermal simulation tasks while maintaining prediction accuracy comparable to high-precision commercial software. This acceleration effect is due to the combination of Fourier transform and self-attention mechanism, which enables the model to significantly reduce computational complexity while maintaining high accuracy, especially when dealing with large-scale 3D IC designs, showing extremely high computational efficiency.

[0048] 2. Improved prediction accuracy: In comparative experiments with other machine learning methods (such as FNO, DeepOHeat, etc.), SAU-FNO performed better in all evaluation indicators, especially in terms of mean square error (MSE), which was reduced by more than 50%. Combining U-Net and self-attention mechanism, the model can capture global and local temperature changes at the same time, especially in areas with large temperature gradient changes, SAU-FNO can predict temperature distribution more accurately.

[0049] 3. Reduce the reliance on high-precision data: Through the application of transfer learning, the present invention can achieve the same effect as the model trained entirely with high-precision data while using a small amount of high-precision data, thereby reducing the cost of data acquisition. Transfer learning not only reduces the demand for high-precision data, but also speeds up the training of the model, so that SAU-FNO can still provide high-quality prediction results when data resources are limited. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 : Shows the overall architecture of SAU-FNO;

[0051] As input, after the preliminary features are extracted by the fully connected network P, it is processed through multiple U-Fourier layers, and a self-attention module is added to capture long-distance dependencies. Finally, the final output is calculated through the fully connected network Q.

[0052] U-Fourier layer: Each U-Fourier layer combines Fourier transform, U-Net module and inverse Fourier transform, and is parameterized in Fourier space by R to enhance the feature extraction of temperature distribution.

[0053] Self-attention module: The self-attention module captures long-range dependencies in features after the U-Fourier layer, enabling the model to better model complex thermal distributions;

[0054] p(x) is the input, P and Q are fully connected neural networks, and T(x) is the output. F represents the Fourier transform, R is the parameterization in Fourier space, F-1 is the inverse Fourier transform, W is a linear bias term, and σ is the activation function. U represents a U-Net.

[0055] Figure 2 : Structure diagram of self-attention module;

[0056] Figure 2 The internal structure of the self-attention module is shown. The input feature map is first convolved by 1×1 to obtain the query (Q), key (K) and value (V) feature maps. Then the similarity between the query and the key is calculated to obtain the attention weight matrix, which is then multiplied with the value feature map to generate the enhanced feature map. This method dynamically adjusts the relationship between features and is particularly suitable for capturing the mutual influence between different regions during heat conduction.

[0057] Figure 3 : The overall framework of our SAU-FNO-based transfer learning method. (a): Pre-training step; (b): Fine-tuning step;

[0058] Figure 4 : Geometry and thermal parameters of 3D-IC;

[0059] Figure 5 : Comparison of the method in this application with other machine learning methods;

[0060] Figure 6 : Performance comparison of Chip1 before and after transfer learning;

[0061] Figure 7 : Stacking structure and plan view of three 3D chips;

[0062] Figure 8 , Fig. 9 : Schematic diagram of the prediction and actual situation of the method of the present invention;

[0063] Fig.10 : Comparison of the maximum and minimum temperatures of COMSOL, MTA, Hotspot, and SAU-FNO on five steady-state samples. DETAILED DESCRIPTION

[0064] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions made to the present invention should all be included in the scope of the claims of the present invention.

[0065] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific embodiments. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0066] The present invention adopts a Fourier neural operator (SAU-FNO) architecture based on self-attention and U-Net. The overall technical solution is as follows:

[0067] Architecture Design

[0068] U-FNO design: SAU-FNO consists of multiple U-Fourier layers, each of which combines Fourier transform and U-Net's skip connection to enhance the representation of high-frequency information through local convolution operations. The U-Fourier layer combines the characteristics of U-Net, retains the multi-scale information of the input features, and effectively avoids the loss of information during deep propagation through skip connections, making the model more robust when processing complex features.

[0069] Self-attention mechanism integration: The self-attention mechanism is introduced in the U-Fourier layer to capture long-distance dependencies in the thermal distribution, thereby enhancing the understanding of the global temperature effect, especially in capturing local temperature changes. The self-attention mechanism dynamically adjusts the feature weights of each position by calculating the correlation between features, thereby better modeling the mutual influence between different regions in the temperature field. Especially in the case of uneven power distribution, self-attention can effectively capture the formation and propagation process of temperature hotspots, making the model more capable in dealing with complex thermal management problems.

[0070] Transfer learning strategy: First use low-precision data to pre-train the model, and then use a small amount of high-precision data for fine-tuning. This strategy can effectively reduce the dependence on high-precision data and significantly improve the prediction performance of the model. Transfer learning makes full use of the advantages of low-precision data, allowing the model to quickly establish a basic temperature distribution model, and further improve the accuracy of the model in details through fine-tuning of high-precision data, thereby achieving a good balance between data cost and computing efficiency.

[0071] Specific implementation steps

[0072] Dimensional transformation:

[0073] Step 1: Get the input data, use it as input features, and perform preliminary processing on the input features through a fully connected neural network. This step maps the input features from the original space to a high-dimensional feature space.

[0074] Step 2: Further process the dimensionally upgraded features through a fully connected network to ensure that the model can learn more temperature distribution details in the high-dimensional space.

[0075] Step 3: Regularize the features after dimensionality increase to avoid overfitting and ensure that the model has good generalization ability in complex environments.

[0076] U-Fourier layer iteration:

[0077] Step 1: Input the features after dimensionality increase, first perform Fourier transform to extract global features.

[0078] Step 2: Parameterize the features in the frequency domain and use the U-Net structure to extract local features.

[0079] Step 3: Perform inverse Fourier transform on the features and introduce nonlinear characteristics by combining the GELU activation function.

[0080] Self-Attention Layer:

[0081] Step 1: Perform self-attention calculation on the features after the U-Fourier layer and obtain the query, key, and value through convolution.

[0082] Step 2: Calculate the similarity between the query and the key to get the attention weight matrix.

[0083] Step 3: Multiply the weight matrix with the value feature to generate the enhanced feature.

[0084] Feature map projection:

[0085] Step 1: Input the features into the fully connected network for dimensionality reduction.

[0086] Step 2: After linear transformation and activation function processing, the final temperature field prediction value is obtained.

[0087] Transfer learning training:

[0088] Step 1: Use a low-precision dataset for pre-training to obtain preliminary temperature distribution features.

[0089] Step 2: Fine-tune the model using high-precision datasets to improve prediction accuracy.

[0090] Example 1

[0091] A 3D-IC thermal simulation method based on self-attention and operator learning includes the following steps:

[0092] Step 1: Dimensional transformation: Use a fully connected neural network P to lift the input observation a(x) to a higher-dimensional space;

[0093] Step 2, U-Fourier layer iteration: for the dimension-enhanced input v l0 (x) Apply an iterative Fourier layer, followed by an iterative U-Fourier layer, to obtain v mM (x), that is, the feature map V t (x);

[0094] Step 3: Self-attention layer: feature map V processed by U-Fourier layer t (x) Perform 1×1 convolution to calculate the spatial attention map A s And channel attention map A c , by element-wise multiplication Combined with A s and A c , get the attention enhanced feature map V t '(x);

[0095] Step 4: Feature map projection: Use the fully connected neural network Q to project the enhanced feature map V t'(x) is projected back to the original space to obtain the thermal simulation result z(x);

[0096] Step 5: Transfer learning training: First use a large amount of low-resolution data D l =X l , Y l Pre-trained model M l , and then use a small amount of high-resolution data D h =X h , Y h For the pre-trained model M l Make fine adjustments.

[0097] Furthermore, step 1 includes the following steps:

[0098] Step 1.1, the input observation value a(x) is transformed P through the neural network to obtain the improved high-dimensional space representation;

[0099] Step 1.2: Use the high-dimensional space representation obtained in step 1.1 as input for subsequent processing.

[0100] Further, step 2 includes the following steps:

[0101] Step 2.1, Initialize v l0 (x) is the input representation after dimensionality increase;

[0102] Step 2.2: v lL (x) is iterated through the U-Fourier layer to obtain v m0 (x);

[0103] Step 2.3, v mk (x) is iterated through the U-Fourier layer to obtain v m(k+1) (x), k=0, 1, .., M-1.

[0104] Step 2.3.1: In each U-Fourier layer, for the input function sequence v mk (x) Apply convolution kernel K to obtain the intermediate result Kv mk (x), where K is Figure 1 Medium FRF -1 The operation process, F stands for Fourier transform, F-1 is the inverse transform. R stands for recombining or performing nonlinear operations in the frequency domain, such as weighting, transforming, or processing through network layers.

[0105] Step 2.43.2: For the input function sequence v mk (x) Apply U-Net to obtain the intermediate result Uv mk (x);

[0106] Step 2.3.3: Add the intermediate results obtained in step 2.3.1 and step 2.3.2 to get Kv mk (x)+Uv mk (x);

[0107] Step 2.3.4: v mk (x) multiplied by the learnable weight W and the result of step 2.3.3, we get Kv mk (x)+Uv mk (x)+Wv mk (x);

[0108] Step 2.3.5: Apply the activation function σ to the result of step 2.3.4 to obtain the updated function sequence v mk+1 (x);

[0109] Step 2.3.6: Repeat steps 2.3.1 to 2.3.5 until v is obtained after M iterations. mM (x), that is, the feature map V t (x).

[0110] Further, step 3 includes the following steps:

[0111] Step 3.1, V t (x) Perform 1×1 convolution to calculate the channel attention map A c :

[0112] A c =W h V t (x)

[0113] Step 3.2: V t (x) Perform 1×1 convolution to calculate the spatial attention map A s :

[0114]

[0115]

[0116] Q=W q V t (x)

[0117] K=W k V t (x)

[0118] Where exp() is the exponential function, where s ij represents the i-th query feature vector Q i and the jth key feature vector K j The similarity or correlation between iis an eigenvector in Q, Indicates Q i The transpose of K j is a eigenvector in K, N represents the key eigenvector K j The number of h , W q , and W k are learnable parameters, representing vectors of values, queries, and keys, respectively;

[0119] Step 3.3: Element-wise multiplication Combined with A c and A s , get the attention enhanced feature map V t '(x):

[0120]

[0121] Further, step 4 includes the following steps:

[0122] Step 4.1: V t '(x) is used as input and the projection result z(x) is obtained by transforming Q through a fully connected neural network;

[0123] Step 4.2: Output z(x) as the final thermal simulation result.

[0124] Further, step 5 includes the following steps:

[0125] Step 5.1: Use data D l =X l , Y l Pre-trained low-resolution model M l , where Y1 is the prediction obtained by applying a low-resolution model M1 with learnable parameters to the input X1, using the L2 loss function to optimize the parameters of model M1:

[0126]

[0127] Where N is the number of samples, T true，i represents the true value of sample i, T pred,i represents the prediction of model M1. After training with low-precision data, model M1 builds l , mapping to Y1;

[0128] Step 5.2: Use data D h =X h , Y h For the pre-trained model M l Fine-tune to get the final model M h ;

[0129] Use data D h =X h , Y h Fine-tune the low-resolution model M l Get M h , where Y h is to transform a high-resolution model M with learnable parameters into h Applied to input X h The obtained predictions are optimized using the L2 loss function. h Parameters:

[0130]

[0131] Where N is the number of samples, T true，i represents the true value of sample i, T pred，i Represents model M h After fine-tuning M using high-precision data l Then get M h , Model M h Established from X h To Y h This completes the modeling of a more accurate predictive temperature model.

[0132] Experimental Implementation

[0133] Experimental setup: This application uses the Alpha 21264EV6 microprocessor architecture as the core structure, and three different 3D ICs are selected for experiments. These ICs have the same stacked components, including L1 / L2 cache layers, thermal interface materials (TIM), heat sinks, core layers, through silicon vias (TSVs), and packaging. Specifically, Chip1 is a single-core processor consisting of two layers of devices: one layer contains the core, two L1 caches, and one L2 cache, while the other layer consists of three L2 caches. Chip2 is a three-layer quad-core processor located on the upper layer close to the heat sink. The L2 cache is located on the other two device layers. The address and data buses between the L2 cache and the processing core are connected through TSVs. Chip3 is a two-layer eight-core microprocessor with a similar architecture to Chip1. 3D structures such as Figure 7 To simplify the floor plan, the TSVs through each layer have been omitted.

[0134] Data Generation: We randomly assign power levels to different functional blocks while ensuring that the total power remains within an appropriate range. By varying the power distribution, we aim to achieve significant and plausible temperature differences in the simulation results, reflecting real-world scenarios. Specifically, we use Synopsys Power Compiler to obtain power traces and modify these traces to create power profiles that serve as input to the dataset. The model output is a heat map; due to the slow processing speed of commercial software such as Celsius, Icepak, and COMSOL, we adopt Manchester Thermal Analysis (MTA) as the solver to derive the training data output. Figure 4 Table 1 in provides the detailed structure of the 3D IC chip.

[0135] Model settings: We refer to the previous work settings and configure SAU-FNO with a width of 2 for Chip1 and Chip2 and a model structure of [12, 12, 2]. For Chip3, due to the larger data scale and larger grid size, the width is doubled, resulting in a model structure of [24, 24, 2] with a width of 64, where [12, 12, 2] is n_modes, which is a tuple, such as (12, 12, 2), indicating the number of Fourier modes retained in each dimension in the Fourier layer.

[0136] n_modes = (12, 12, 2), the model will retain 12 Fourier modes in each dimension (such as height and width) for frequency domain transformation, and 2 in depth.

[0137] The number of Fourier modes retained in each dimension determines the frequency information that FNO can capture in that dimension. A higher number of modes means that the model can capture higher frequency features (finer details), while a lower number of modes focuses more on low frequencies (smoother features).

[0138] Width refers to the number of channels or feature dimensions of each layer in the model. It is similar to the number of channels of the convolutional layer in CNN, which determines the feature representation capability of each layer. Width controls the feature dimension of each layer of the FNO model when processing data. A larger width means that the model can process more features, thereby improving the model's expressiveness, but it also increases the computational and storage costs. Width controls the feature dimension of each layer of the FNO model when processing data. A larger width means that the model can process more features, thereby improving the model's expressiveness, but it also increases the computational and storage costs.

[0139] Training and testing: We used the Adam optimizer with decaying learning rate. The initial learning rate was set to 1e4, and a weight decay of 1e-5 was implemented to release model overfitting. When fine-tuning was used, we chose an initial learning rate of 1e-5 to ensure training stability. To ensure convergence, each training process exceeded 200 epochs.

[0140] Comparison with State-of-the-Art Machine Learning Methods

[0141] The method we proposed is essentially a neural operator method, which has properties that neural networks do not have, such as grid invariance. Directly comparing accuracy or time is not a completely fair approach. Therefore, we only compare some of the latest neural operator methods, including FNO, U-FNO, DeepOHeat, etc.

[0142] In order to verify the effectiveness of this method, we selected two groups of power trajectories with large differences for visualization, as shown in the figure. Figure 8 and Fig. 9 As shown. Note that the chip here has two heat-generating layers, so we use layer 1 and layer 2 to represent them respectively. It can also be seen from the visualization results that our thermal simulation results are very close to the actual situation, not only achieving good results in average temperature difference, but also performing well in temperature gradient and temperature variation.

[0143] In terms of quantitative experiments, to evaluate the performance, we used various standard metrics such as root mean square error (RMSE), mean temperature error (Mean), and maximum temperature error (Max). In addition, we also included mean absolute percentage error (MAPE) and peak absolute percentage error (PAPE) in the analysis. Compared with RMSE, junction temperature is more concerned in 3D IC thermal modeling. Figure 5 As can be seen in Table 2, our proposed SAU-FNO method outperforms the baseline method in all indicators. Specifically, RMSE and junction temperature can be reduced by more than 50% compared to FNO. The performance improvement is similar to that of other methods. This proves that our method has stronger thermal modeling capabilities than other state-of-the-art machine learning methods. Note that no transfer learning strategy is used here. Only the advantages and performance improvement of design thermal modeling are verified here.

[0144] Although transfer learning still requires a certain amount of high-precision data, it significantly reduces the need for high-precision data and speeds up model training. We choose Chip1 as the reporting result; the results of the three chip architectures are similar. Our experiments found that using full fine-tuning works better than freezing a few layers. Figure 6As shown in Table 3, we observe that RMSE and junction temperature, as well as other metrics, show only a slight loss in accuracy when using fine-tuning versus training from scratch on large-scale high-resolution data. It is worth noting that the low-resolution data model was trained on 4,000 cases, while the high-resolution data model was trained on only 1,000 cases. We also trained the high-resolution model from scratch using 4,000 cases to provide a fair comparison. Considering that acquiring high-resolution data takes about 4 to 6 times more time than low-resolution data, the total time cost is reduced by about 2.5 times.

[0145] We also performed complementary experiments to show that if the amount of high-precision data used for fine-tuning is increased to 4,000 cases (the same data used for training from scratch), the error can be further slightly reduced. At the same time, transfer learning is not only applicable to SAU-FNO; we also experimented with other machine learning methods such as FNO and U-FNO and showed that it is equally effective. This shows that transfer learning is well suited for applications that require high efficiency, especially in the context of 3D IC thermal simulation.

[0146] Comparison with professional thermal simulation tools

[0147] We compared the accuracy and speed of the proposed model with other mainstream commercial software or open source tools that support 3D IC thermal simulation, including COMSOL, MTA, and Hotspot. COMSOL is a powerful multi-physics simulation software based on the finite element method (FEM) and is a representative of commercial software suitable for complex engineering problems. MTA is a representative of open source academic tools, also based on FEM. Hotspot uses fast thermal network analysis technology based on the finite element difference method (FDM) and empirical models.

[0148] To achieve a certain fairness in the comparison, we performed simulation experiments using the finest grid available with these tools, i.e., the maximum resolution. We generated 20 sets of high-precision data power traces for chip1-chip3 using the method in Section 4, performed 20 simulations with these software or frameworks to obtain the maximum surface temperature and temperature gradient of each device layer, and compared the average error value with our algorithm. In 3D IC thermal simulation, junction temperature and temperature gradient are the two most important indicators, so we chose them for comparison. Fig.10Table 4 shows that the predictions provided by SAU-FNO are very close to the baseline method. The maximum temperature error is within 0.25K, which proves its usability in practical thermal simulation problems. In terms of speed, the commercial software COMSOL is relatively slow, while the open source tools MTA and Hotspot show superior performance. To ensure fairness, our speed comparison is only for these two tools. Based on the average time of 60 simulation sets from chip1 to chip3, the average prediction time of our SAU-FNO method is 0.27 seconds, while MTA is 227.31 seconds and Hotspot is 98.47 seconds, achieving 842 times and 365 times speed improvements respectively.

[0149] In summary, we propose SAU-FNO, a cutting-edge thermal modeling framework that effectively addresses the key challenges of thermal management of 3D integrated circuits. By integrating the self-attention mechanism and U-Net architecture into the Fourier neural operator, we achieve a powerful model that is able to capture both global and local thermal dynamics. Our results show a significant improvement in accuracy while drastically reducing the computation time—up to 842 times faster than traditional methods. The successful application of transfer learning further enhances the performance of our model by minimizing the need for extensive high-precision datasets. Looking ahead, we aim to expand the capabilities of SAU-FNO to cover a wider range of thermal analysis tasks, thereby contributing to a more efficient design process for advanced semiconductor technologies.

Claims

1. A 3D-IC thermal simulation method based on self-attention and operator learning, characterized in that: The following steps are involved: Step 1: Dimensional transformation: Use a fully connected neural network P to lift the input observation a(x) to a higher-dimensional space; Step 2, U-Fourier layer iteration: for the dimension-enhanced input v l0 (x) Apply the iterative U-Fourier layer to obtain v lM (x), that is, the feature map V t (x); Step 3: Self-attention layer: feature map V processed by U-Fourier layer t (x) Perform 1×1 convolution to calculate the spatial attention map A s And channel attention map A c , by element-wise multiplication Combined with A s and A c , get the attention enhanced feature map V t '(x); Step 4: Feature map projection: Use the fully connected neural network Q to project the feature map V after attention enhancement t '(x) is projected back to the original space to obtain the thermal simulation result z(x); Step 5: Transfer learning training: First use a large amount of low-resolution data D l =(X l , Y l ) Pre-trained model M l , and then use a small amount of high-resolution data D h =(X h , Y h ) for the pre-trained model M l Fine-tune and obtain model M h .

2. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1, the input observation value a(x) is transformed P through the neural network to obtain the improved high-dimensional space representation; Step 1.2: Use the high-dimensional space representation obtained in step 1.1 as input for subsequent processing.

3. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 1, characterized in that: Step 2 includes the following steps: Step 2.1, initialize v l0 (x) is the input representation after dimensionality increase; Step 2.2: v l0 (x) is iterated through the U-Fourier layer to obtain v m0 (x); Step 2.3, v mk (x) is iterated through the U-Fourier layer to obtain v m(k+1) (x),k=0,1,...,M-1.

4. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 3, characterized in that: Step 2.3 includes the following steps: Step 2.3.1: In each U-Fourier layer, for the input function sequence v mk (x) Apply the convolution kernel K to obtain the intermediate result Kv mk (x); Step 2.3.2: For the input function sequence v mk (x) Apply U-Net to obtain the intermediate result Uv mk (x); Step 2.3.3: Add the intermediate results obtained in step 2.3.1 and step 2.3.2 to get Kv mk (x)+Uv mk (x); Step 2.3.4: v mk (x) multiplied by the learnable weight W and the result of step 2.3.3, we get Kv mk (x)+Uv mk (x)+Wv mk (x); Step 2.3.5: Apply the activation function σ to the result of step 2.3.4 to obtain the updated function sequence v mk+1 (x); Step 2.3.6: Repeat steps 2.3.1 to 2.3.5 until v is obtained after M iterations. mM (x), that is, the feature map V is obtained t (x).

5. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 1, characterized in that: Step 3 includes the following steps: Step 3.1, V t (x) Perform 1×1 convolution to calculate the channel attention map A c : A c =W h V t (x) Step 3.2: V t (x) Perform 1×1 convolution to calculate the spatial attention map A s : Q=W q V t (x) K=W k V t (x) Where exp() is the exponential function, where s ij represents the i-th query feature vector Q i and the jth key feature vector K j The similarity or correlation between i is an eigenvector in Q, Indicates Q i The transpose of K j is a eigenvector in K, N represents the key eigenvector K j The number of h , W q , and W k are learnable parameters, representing vectors of values, queries, and keys, respectively; Step 3.3: Element-wise multiplication Combined with A c and A s , get the attention enhanced feature map V t '(x):

6. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 1, characterized in that: Step 4 includes the following steps: Step 4.1: V t '(x) is used as input and the projection result z(x) is obtained by transforming Q through a fully connected neural network; Step 4.2: Output z(x) as the final thermal simulation result.

7. The 3D-IC thermal simulation method based on self-attention and operator learning according to claim 1, characterized in that: Step 5 includes the following steps: Step 5.1: Use data D l =X l , Y l Pre-trained low-resolution model M l , where Y l is to transform the low-resolution model M with learnable parameters into l Apply the predictions obtained by input X1 to optimize the model M using the L2 loss function l Parameters: Where N is the number of samples, T true，i represents the true value of sample i, T pred，i Represents model M l After training with low-precision data, model M1 builds l , mapping to Y1; Step 5.2: Use data D h =X h , Y h For the pre-trained model M l Fine-tune to get the final model M h ; Use data D h =X h , Y h Fine-tune the low-resolution model M l Get M h , where Y h is to transform a high-resolution model M with learnable parameters into h Applied to input X h The obtained predictions are optimized using the L2 loss function. h Parameters: Where N is the number of samples, T true，i represents the true value of sample i, T pred，i Represents model M h After fine-tuning M using high-precision data l Then get M h , Model M h Established from X h To Y h This completes the modeling of a more accurate predictive temperature model.

Images