A method, device and storage medium for 3D integrated circuit thermal simulation

Abstract

The present application relates to the technical field of integrated circuits, and discloses a method and device for 3D integrated circuit thermal simulation and a storage medium. The existing method based on machine learning ignores the shared structure of heat conduction and diffusion type partial differential equations, resulting in the problem of needing to train from zero, relying on massive data and poor generalization ability. The method comprises the following steps: obtaining a pre-trained basic model on a diversified data set containing diffusion type partial differential equations; fine-tuning the pre-trained model to adapt to the 3D integrated circuit thermal simulation task, reinitializing the embedding layer and the recovery layer to process specific input and output features, and keeping the backbone network pre-training weight; applying different learning rates to different layers in fine-tuning, using a lower learning rate for the backbone network and a higher learning rate for the embedding layer and the recovery layer; using a combined loss function for optimization, including a global fidelity loss and a hotspot area loss emphasizing the prediction accuracy of high temperature areas; and using the fine-tuned model to predict the steady-state temperature field distribution of the 3D integrated circuit.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology and provides a method for thermal simulation of 3D integrated circuits. Background Technology

[0002] With the development of three-dimensional integrated circuit (3D-ICs) technology, modern 3D-ICs exhibit extremely high power density, making thermal management a core concern in chip reliability design [3,8]. Typical 3D-ICs contain thermal meshes with more than one million nodes, resulting in finite element method (FEM) solvers requiring several minutes to several hours per analysis [7,19]. In actual design workflows, hundreds to thousands of thermal assessments are typically required

[21] , resulting in huge computational costs. Therefore, accelerating 3D-ICs thermal simulation from hours to seconds has become a key requirement in the field of electronic design automation (EDA) [4,32,37].

[0003] Early machine learning methods for thermal acceleration mainly relied on convolutional neural networks (CNNs) [12,34,39] for thermal field prediction. While these methods reduced simulation time, they exhibited poor generalization ability when faced with changes in geometry, material properties, or operating conditions. Subsequently, physically-informed neural networks (PINNs) were introduced into the field of thermal analysis by incorporating the governing equations into the training process [5], but they faced problems of unstable optimization and difficulty in handling complex boundary conditions in multi-scale 3D-ICs structures [2,29].

[0004] Recently, neural operators have demonstrated strong performance on partial differential equation (PDE) problems through two representative architectures—Fourier neural operators (FNOs)

[20] and DeepONets

[27] . These methods provide resolution independence and fast inference capabilities by learning function space mappings. However, as Figure 1 As shown in (a), existing methods [15,25,26] still require a large amount of high-fidelity FEM data and typically train the model independently from scratch for each chip, failing to fully utilize the shared mathematical structure between thermal conduction and diffusion-type PDEs [1,9,41].

[0005] The application of basic models in scientific computing provides new ideas for solving the above problems. Large-scale pre-trained PDE basic models, such as DPot

[10] and Poseidon

[11] , have demonstrated that pre-training on diverse families of PDEs enables models to quickly adapt to downstream tasks with minimal data. These models learn transferable general physical priors by pre-training on idealized families of PDEs. However, whether these models pre-trained on idealized families of PDEs can be effectively transferred to real engineering problems with complex, heterogeneous material properties and complex geometries (such as 3D-IC thermal analysis) remains an open and critical question.

[0006] In summary, the main challenges facing existing 3D-ICs thermal simulation technology are:

[0007] (1) Traditional FEM methods are computationally expensive;

[0008] (2) Existing machine learning methods require a large amount of training data and have limited generalization ability;

[0009] (3) The shared PDE mathematical structure between different physical systems was not fully utilized. Therefore, there is an urgent need for a 3D-ICs thermal simulation method that can overcome data bottlenecks, maintain high accuracy, and be computationally efficient. Summary of the Invention

[0010] The purpose of this invention is to solve the problems of existing machine learning-based 3D integrated circuit thermal simulation methods, which ignore the shared mathematical structure between heat conduction and diffusion partial differential equations (PDEs), resulting in high data acquisition costs, poor model generalization ability, large training resource consumption, and difficulty in achieving high-precision hotspot prediction with limited data.

[0011] To achieve the above objectives, the present invention employs the following technical means:

[0012] This invention provides a method for thermal simulation of 3D integrated circuits, comprising the following steps:

[0013] Step 1: Obtain a base model pre-trained on a diverse PDE dataset including diffuse partial differential equations, the base model comprising a Transformer-based backbone network and convolutional neural network branches;

[0014] Step 2: Fine-tune the pre-trained base model to adapt to the 3D integrated circuit thermal simulation task. The fine-tuning includes re-initializing the embedding layer and recovery layer to handle the specific input and output features of the 3D integrated circuit, while maintaining the pre-trained weights of the backbone network. The target dataset used for fine-tuning contains no more than 20% of the samples of the complete training data.

[0015] Step 3: During fine-tuning, different learning rates are applied to different layers of the model, with a lower learning rate applied to the backbone network and a higher learning rate applied to the embedding and recovery layers.

[0016] Step 4: During the fine-tuning process, optimization is performed using a combined loss function, which includes global fidelity loss and hot spot region loss, where the hot spot region loss emphasizes the prediction accuracy of high-temperature regions in the chip domain;

[0017] Step 5: Use the fine-tuned model to predict the steady-state temperature field distribution of the 3D integrated circuit, paying particular attention to the temperature accuracy of the hot spot area of ​​the chip.

[0018] In the above scheme, step 1 includes:

[0019] Step 1.1: Obtain the base model pre-trained on a diverse PDE dataset, which includes PDE scenarios centered on diffusion-type dynamics, wherein the PDE scenarios contain diffusion-reaction partial differential equations with the following steady-state form:

[0020]

[0021] in, Represents a conserved quantity. The diffusion coefficient, representing spatial variation, Indicates the source term, Represents spatial coordinates;

[0022] Step 1.2: The basic model adopts a hybrid spectrum-attention architecture, including an embedding layer, a convolutional neural network branch composed of multiple ConvNeXt blocks, a Transformer-based backbone network composed of SwinV2 stages, and a recovery layer;

[0023] Step 1.3: The base model has learned a universal diffusion kernel applicable across domains during the pre-training phase, enabling the model to capture physical priors related to diffusion.

[0024] In the above scheme, step 2 includes:

[0025] Step 2.1: For the 3D integrated circuit thermal simulation task, the input features are determined to include power density. and thermal conductivity The output feature is determined to be a temperature field. ,in Representing spatial coordinates; the task is formulated as a parametric operator learning problem, with the goal of learning a solution operator:

[0026]

[0027] in, Represents the input function space. Represents the output function space. Indicates chip domain, Indicates learnable parameters, Satisfying the steady-state heat conduction equation ;

[0028] Step 2.2: Reinitialize the embedding layer of the pre-trained base model. and recovery layer , where the embedding layer Used to map input features to patch markers, recovery layer Used to map high-resolution markers back to the temperature field;

[0029] Step 2.3: Set the backbone network parameters of the pre-trained base model to pre-trained weights. The backbone network includes a SwinV2 stage and a ConvNeXt block.

[0030] Step 2.4: Fine-tune the re-initialized embedding layer, recovery layer, and backbone network using the target dataset. The number of samples in the target dataset does not exceed 20% of the complete training data. The embedding layer and recovery layer are trained from scratch, and the backbone network is lightly fine-tuned.

[0031] In the above scheme, step 3 includes:

[0032] Step 3.1: Set the learning rate of the backbone network to the first learning rate. Set the learning rates of the embedding and recovery layers to the second learning rate. ,in ;

[0033] Step 3.2: During fine-tuning, use the first learning rate. Update the parameters of the backbone network using the second learning rate. Update the parameters of the embedding layer and the recovery layer; where, , .

[0034] In the above scheme, step 4 includes:

[0035] Step 4.1: Calculate the global fidelity loss Its expression is: in, This represents the chip domain of a 3D integrated circuit. Indicates position The predicted temperature value at that location, Indicates position The actual temperature value at that location, Represents spatial coordinates, Represents chip domain Measure;

[0036] Step 4.2: Define the critical temperature set ,in Represents the temperature field of -Quantities, ;

[0037] Step 4.3: Calculate the loss in the hotspot area Its expression is: in, Represents the set of critical temperatures Measure;

[0038] Step 4.4: Combine the global fidelity loss and the hotspot region loss into a total loss function: in, Weighting parameters to balance overall accuracy and hotspot precision;

[0039] Step 4.5: During fine-tuning training, when training rounds... At that time, use the critical temperature set defined in step 4.2. Calculate the loss for hotspot regions; during training epochs At that time, an adaptive critical temperature set is adopted: replace and use Calculate the loss in the hotspot area.

[0040] In the above scheme, step 5 includes:

[0041] Step 5.1: Input the input features of the 3D integrated circuit into the fine-tuned model, wherein the input features include power density. and thermal conductivity ,in Represents spatial coordinates;

[0042] Step 5.2: Perform forward propagation using the fine-tuned model, sequentially through the embedding layers. Mapping to patch markers, extracting local structure from ConvNeXt blocks, performing window-based self-attention processing in the SwinV2 stage, gradually restoring spatial resolution in the decoder, and the recovery layer. Mapping to generate the predicted steady-state temperature field distribution ;

[0043] Step 5.3: Output the predicted steady-state temperature field distribution. And extract chip domain The highest temperature value in the chip is used as a key verification indicator for the accuracy of temperature in the hot spot area.

[0044] The present invention also provides an apparatus for thermal simulation of 3D integrated circuits, including a processor and a storage medium, wherein when the processor executes a program in the storage medium, it implements the method for thermal simulation of 3D integrated circuits.

[0045] The present invention also provides a storage medium in which the processor executes a program in the storage medium to implement the method for thermal simulation of 3D integrated circuits.

[0046] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0047] 1. This invention solves the technical problem of existing neural operator methods that "require training from scratch for each chip and rely on massive amounts of high-fidelity data" caused by neglecting the shared mathematical structure between the heat conduction equation and the diverse dataset of diffusion-type partial differential equations in step 1, and combining the techniques in step 2 of only fine-tuning the embedding layer and recovery layer, retaining the pre-trained weights of the backbone network and using a very small amount of target data (≤20%). This achieves the effect of effectively transferring the cross-domain general diffusion physics priors (such as diffusion kernels and gradient propagation laws) learned in the pre-training stage to the 3D integrated circuit thermal simulation task, significantly reducing the dependence on chip-specific high-cost simulation data, and enhancing the model's generalization adaptability to different chip geometries and material heterogeneity.

[0048] 2. This invention solves the contradiction between the easily corrupted physical priors of the pre-trained backbone network and the slow convergence of task-specific layers during fine-tuning by applying a low learning rate to the backbone network and a high learning rate to the re-initialized embedding and recovery layers in step 3. It achieves the effect of preserving the stability of the general diffusion physical structure encoded by the backbone network while enabling the embedding and recovery layers to quickly adapt to the unique input characteristics (power density, thermal conductivity spatial distribution) and output characteristics (high-resolution temperature field) of 3D integrated circuits. This avoids knowledge forgetting and training oscillations during fine-tuning and ensures the stability and efficiency of transfer learning.

[0049] 3. This invention solves the technical problem that the standard loss function, due to its spatial uniform weighting, neglects the high-temperature tail region during the optimization process and is difficult to meet the stringent requirements for hotspot accuracy in chip reliability design by designing a combined loss function (global fidelity loss and hotspot region loss) and introducing an adaptive hotspot set mechanism (step 4.5). It achieves the goal of explicitly strengthening the focus on the critical region of α-quantile in the temperature distribution (α=0.995) in the optimization objective, so that the model can effectively guide the gradient signal to the key hotspot region under the condition of limited training resources, and significantly improve the physical consistency and reliability verification value of the high-temperature region prediction.

[0050] This invention organically integrates three major technical approaches: "PDE basic model pre-training transfer (steps 1-2)," "hierarchical differentiated learning rate fine-tuning (step 3)," and "hotspot-aware combined loss optimization (step 4)," generating a nonlinear synergistic effect. The diffusion physical prior provided by the pre-trained model lays a solid inductive bias for fine-tuning with limited data; the hierarchical learning rate strategy ensures that this prior is stably retained during fine-tuning, while releasing the rapid adaptability of task-specific layers; and the combined loss function precisely directs limited optimization resources to the core pain point of engineering—hotspot regions. The synergy of these three approaches breaks through the traditional technical dilemma of "insufficient accuracy due to limited data, global imbalance due to focusing on hotspots, and weak adaptability due to retaining priors." It enables the model to simultaneously achieve high-fidelity reconstruction of the global temperature field and high-precision capture of hotspot regions with extremely low data dependence, and supports second-level steady-state temperature field prediction in step 5. It systematically solves the fundamental problem of balancing data cost, prediction accuracy (especially hotspots), and computational efficiency in 3D integrated circuit thermal simulation, providing a new paradigm of efficient simulation that deeply integrates physical mechanisms and data-driven approaches for the EDA field. Attached Figure Description

[0051] Figure 1 The approach shifted from independent training for each chip to fine-tuning a base model using less than 20% of the data.

[0052] Figure 2 The workflow of PNO-Therm (abbreviated as the present invention). The left figure shows the lightweight fine-tuning paradigm we built based on the pre-trained PDE base model, and the right figure shows the base model architecture used in this framework.

[0053] Figure 3 Comparison with other methods in the single-core case.

[0054] Figure 4 Sensitivity analysis of PNO-Therm to the number of training samples under three designs. The gray dashed line represents the root mean square error (RMSE) benchmark of SAU-FNO (state-of-the-art method). All experiments were generated using Ours-B configuration; Figure 5 Visualization of temperature field prediction using PNO-Therm in three chip designs and comparison with simulation tools. Detailed Implementation

[0055] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0056] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0057] To facilitate a better understanding by those skilled in the art, the relevant technical knowledge in this field is explained as follows:

[0058] 1. Thermal conduction of 3D-ICs

[0059] The steady-state temperature distribution in 3D ICs is governed by the heat conduction equation. For a chip domain... Spatial coordinates Temperature field satisfy:

[0060]

[0061] in Thermal conductivity, which represents spatial variation This indicates the volumetric power density. The top surface typically undergoes convective cooling.

[0062]

[0063] in Indicates the convective heat transfer coefficient. Indicates the temperature of the boundary surface. Indicates ambient temperature. Represents the temperature gradient;

[0064] The side boundaries are considered either adiabatic or periodic, depending on the encapsulation constraints. 3D-ICs exhibit significant material and geometric heterogeneity, requiring FEM solvers to resolve millions of mesh nodes

[30] . Solving the resulting large sparse systems typically takes minutes to hours, becoming a major bottleneck in the design workflow [17, 36].

[0065] 2. Neural Operators: Learning in Function Spaces

[0066] Neural operators provide a framework for learning mappings between infinite-dimensional function spaces while maintaining independence from discretization. Unlike CNNs, which operate on fixed-size inputs, they approximate function-to-function mappings. Given input and output spaces... and The goal is to approximate an operator:

[0067]

[0068] in This could represent coefficients of spatial variation, boundary conditions, or source terms, while This represents the corresponding PDE solution. Representative architectures include DeepONet

[27] and FNO

[20] , as well as multi-scale variants. Despite their strong performance on PDE benchmarks, existing neural operators still require a large amount of high-fidelity data and cannot reuse the learned physical structures when adapting to new tasks.

[0069] 3. Pre-trained PDE base model

[0070] The underlying models have recently been extended from natural language processing and computer vision to scientific machine learning [10, 11]. Unlike training separate models for each task, this paradigm pre-trains large-scale neural operators on a diverse family of PDEs, enabling them to learn general physical priors that can be efficiently transferred to downstream problems, requiring only minimal data.

[0071] Poseidon

[11] embodies this approach, employing a hybrid spectral-attention architecture and pre-training on thousands of PDE scenarios centered on diffusion-type dynamics. This pre-training exposes the model to underlying physical behaviors, including diffusion, gradient propagation, and steady-state convergence, resulting in strong zero-shot and few-shot generalization capabilities on previously unseen PDE types. Although previous work has shown that PDE base models can be transferred between different physical systems, their applicability in thermal simulations of 3D-ICs remains untested.

[0072] 4. Problem Formulation and Overall Framework

[0073] This invention formulates 3D-IC thermal simulation as a parameter operator learning problem. Given the encoded power density... and thermal conductivity input function space and the output space representing the temperature field The goal of this invention is to learn the solver:

[0074]

[0075] in Represents learnable parameters, chip domain , Satisfies the steady-state thermal equation:

[0076]

[0077] and boundary conditions exist Above represents the convective cooling of the top surface and the adiabatic conditions of the side boundaries, where Represents the boundary operator, Represents the boundary of the computational domain. This represents boundary condition functions, such as Neumann conditions.

[0078] Figure 2 This provides an overview of the framework of the present invention. The present invention (PNO-Therm for short) uses a pre-trained Poseidon architecture as its spectral operator backbone; the PDE model pre-training stage has been completed, and the present invention directly utilizes this pre-trained model for downstream 3D-IC thermal adaptation.

[0079] During forward propagation, the embedding layer... and recovery layer Both were reinitialized and trained from scratch. These two components account for only a small fraction of the total parameters, while the remaining spectral backbone retains its pre-trained weights and is updated only through light fine-tuning.

[0080] During forward propagation, the input channels for the downstream 3D-IC task differ from those used during pre-training, and the output resolution also changes. Therefore, the embedding layer... and recovery layer The pre-trained weights were reinitialized, while the remaining modules retained their pre-trained weights and underwent minor fine-tuning.

[0081] Input features Firstly, through Mapped to patch markers and processed through several ConvNeXt blocks

[24] to extract local structures, where The spatial distribution density of TSVs is then represented and labeled through the SwinV2

[22] stage, applying window-based self-attention:

[0082]

[0083] in , and It is a linear projection feature. This indicates a relative position offset.

[0084] At each encoder level, the SwinV2 output is downsampled via PatchMerging, while the ConvNeXt branch provides skip connections to the decoder. At the lowest spatial resolution, another SwinV2 module aggregates the global context. The decoder mirrors the encoder to progressively restore the spatial resolution. Finally, the recovery layer... Mapping high-resolution labels back to the temperature field generates predictions. .

[0085] 5. Transfer learning through operator domain adaptation

[0086] 5.1 Mathematical Structure Alignment

[0087] The pre-trained equations are structurally aligned with steady-state heat conduction in 3D-ICs. Many PDEbench systems follow a diffusion-reaction model:

[0088]

[0089] in This represents the field variable to be solved. The diffusion coefficient, representing spatial variation, It is the source term. In steady state ( This can be simplified to:

[0090]

[0091] By replacement , , This is isomorphic to the thermal equation. Both domains require solving elliptic PDEs with strong material heterogeneity and complex interface conditions, allowing for the natural transfer of diffusion-related priors learned during pre-training.

[0092] 5.2 Theoretical Basis of Transfer

[0093] Formalizing transfer learning from the perspective of operator field adaptation. Let... This represents the source distribution of the PDE family in PDEbench. This represents the target distribution for 3D-IC thermal problems. The present invention uses... This represents the input-output function pair used for operator supervision. Pre-trained operators. minimize Expected risk:

[0094]

[0095] in This represents an optimization problem that minimizes the expected risk over the source distribution. Denotes the loss function. This represents an operator learning model parameterized by parameter θ;

[0096] To achieve effective transfer, the learned operator must capture a universal diffusion kernel applicable across domains. Let... This represents the hypothesis class of the diffusion operator. Assuming both the source and target PDEs belong to the diffusion family, the generalization gap can be restricted as follows:

[0097]

[0098] in The hypothetical class differences between the measurement distributions This represents the combined error of the optimal assumption. Structural alignment ensures... The small value proves the rationality of the migration efficiency. Represents a parameterized neural operator.

[0099] 5.3 Multi-rate optimization for stable adaptation

[0100] The neural operator exhibits a hierarchical representation: early spectral layers encode a general diffusion kernel, while deeper layers capture task-specific variations. Let... Indicates the first There are one spectral layer, therefore:

[0101]

[0102] in Let represent the feature space of the l-th layer. This indicates the composition of functions.

[0103] A multi-rate optimization strategy is employed, where all spectral layers remain trainable but receive significantly smaller learning rates to preserve the pre-trained diffusion structure. Specifically, the spectral backbone uses... Update, while embedding and recovery modules use higher learning rates. This allows for rapid adaptation to the specific input distribution and output field of the chip.

[0104] Therefore, training is minimized:

[0105]

[0106] in This represents an optimization problem that minimizes the expected risk on the target distribution Dtarget.

[0107] Achieve stable adaptation while retaining the diffusion-oriented inductive bias of the pre-trained model.

[0108] 6. Hotspot-aware loss function

[0109] Standard mean squared error assigns uniform weights in the spatial domain, leading optimization to prioritize reducing errors in large low-temperature regions while tolerating larger residuals in local hotspots. We address this issue using a combined loss that integrates global fidelity and explicit tail region emphasis.

[0110] 6.1 Theoretical Motivation

[0111] The temperature distribution in 3D-ICs exhibits heavy-tailed characteristics. Let... This represents the probability density of temperature values. High-temperature regions satisfy... However, it is the dominant reliability constraint. (Standard) loss:

[0112]

[0113] in, This represents the critical temperature threshold that defines the high-temperature region, implicitly measured spatially. Weighted errors cause the gradient signal of hotspots to disappear relative to the main contribution. We instead construct a loss that emphasizes tail behavior through explicit hotspot identification and differential weighting.

[0114] 6.2 Formulation of Combined Loss

[0115] Global components ensure overall field fidelity:

[0116]

[0117] Regarding the emphasis on hotspots, we define the critical temperature set as the upper... -Quantities:

[0118]

[0119] in Indicates high-temperature hotspots. express -Quantities. The lower bound is represented by t, and the temperature threshold variable is represented by t. This represents the cumulative distribution function of the random variable T in temperature. Indicates quantile level;

[0120] Based on empirical analysis showing that high temperatures are concentrated within 0.5% of the range, we set... On this subset, we use Loss to achieve robustness:

[0121]

[0122] The norm provides robustness to occasional outliers in FEM numerical artifacts while preserving convexity.

[0123] The total loss combines two items:

[0124]

[0125] in Balances overall accuracy with hotspot precision. This value is selected through cross-validation.

[0126] 6.3 Adaptive Hotspot Definition

[0127] In early training (rounds) )period, From the true value only Exporting provides a clear gradient signal. For We adopt an adaptive approach:

[0128]

[0129] This prevents degradation that suppresses predicted temperatures to avoid hotspot penalties, while maintaining physical consistency. This adaptive mechanism ensures... Increased penalties for omissions and false hotspots.

[0130] 7. Experiment

[0131] We evaluated PNO-Therm on a standard 3D-IC thermal benchmark from the open-source simulator HotSpot

[13] . The planar plots were derived from three stacked configurations of the Alpha 21264 (EV6)

[16] processor: single-core (2 device layers), quad-core (3 device layers), and octa-core (2 device layers), covering different design scales and thermal complexities. Table 1 summarizes the benchmark parameters. Thermal interface material (TIM) layers are inserted between device layers, and through-silicon vias (TSVs) provide vertical connections. The power plots follow the EV6 paradigm under representative workload conditions. We adopted standard 3D-IC thermal boundary conditions: the side boundaries are modeled as adiabatic, the top surface is convectively cooled by a heatsink, and the bottom surface remains thermally coupled to the package layer.

[0132] Table 1: Detailed parameter settings for benchmark testing

[0133]

[0134] Following our previous work

[38] , we used the Manchester Thermal Analyzer (MTA)

[18] (an academic simulator based on FEM) to generate the temperature field. The structure was discretized using gmsh and then directly solved in steady state using a preconditioned Krylov solver (i.e., with the time derivative set to zero) to ensure numerical robustness.

[0135] We compare PNO-Therm with representative methods, including FNO

[20] , U-FNO

[35] , DeepOHeat

[23] , ARO

[33] , SAU-FNO

[14] , and T-Fusion

[38] . PINN-based methods[6, 31, 40] are excluded because they solve a single PDE instance rather than learning solution operators across a family of PDEs. For baselines with published results, we cite official figures; otherwise, we reproduce experiments to the best of our ability according to available descriptions.

[0136] 7.1 Experimental Setup

[0137] Experiments were conducted on an NVIDIA A100 GPU (80GB) and an Intel Xeon Gold 6246R CPU, using PyTorch 2.0.1 and CUDA 11.8. We employed the AdamW optimizer with a learning rate of 5×10⁻⁶. -5 Batch size 40, weight decay 1×10 -6 Hyperparameters were tuned using Optuna, and all experiments were tracked using Wandb.

[0138] The baseline implementation follows the configuration specified in SAU-FNO

[14] to ensure fair comparison.

[0139] PNO-Therm follows three pre-trained POSEIDON

[11] scales: T(21M), B(158M) and L(629M), and uses the same network architecture and hyperparameter settings as the corresponding POSEIDON models.

[0140] During data preprocessing, we applied a uniform normalization procedure to all samples: The mean and standard deviation used for normalization were pre-calculated from a representative dataset and then kept constant across all experiments. This set of normalization constants was consistently used for data collected from the same chip but at different resolutions. Experimental results show that this strategy significantly improves the model's generalization ability across different resolutions.

[0141] A total of 5000 steady-state samples were generated, following the SAU-FNO

[14] setup, and divided into 3600 / 400 / 1000 samples for training, validation, and testing. All three 3D-IC designs were evaluated at two spatial resolutions to ensure mesh independence and cross-scale consistency. Performance was evaluated using MAE, RMSE, MAX, MAPE, PAPE (peak absolute percentage error, emphasizing hotspot bias), and mean error. Computational efficiency was measured using training time and GPU memory.

[0142] We also report two improved metrics, Imp-RMSE and Imp-Max, which quantify the reduction in RMSE and MAX errors relative to the previous state-of-the-art method (SAU-FNO)

[14] .

[0143] 7.2 Comparison with other ML methods

[0144] We evaluated PNO-Therm on three chip designs and two spatial resolutions to assess its accuracy and scalability.

[0145] In the quad-core case (64×64) (Table 2), traditional operator-based baselines such as FNO (0.409 RMSE) and DeepOHeat (0.473) produced large prediction errors, while U-FNO (0.278) still performed worse than SAU-FNO (0.203). In contrast, Ours-B (158M) reduced the RMSE to 0.078, MAPE to 0.008, and the mean error to 0.022, achieving a 2.60x improvement in RMSE and a 2.25x improvement in maximum error compared to SAU-FNO. Training efficiency was also significantly improved: Ours-B required only 0.85 hours and 5.75 GB of GPU memory, while SAU-FNO required 4.24 hours and 15.1 GB, achieving approximately 5x speedup and 2.6x memory reduction. Although T-Fusion is faster and more memory efficient, its accuracy is insufficient for high-fidelity thermal analysis.

[0146] Table 2: Comparison of PNO-Therm with Existing Machine Learning-Based Modeling Methods

[0147]

[0148] In single-core case (88×88) Figure 3 On the resolution scale, FNO (0.157), U-FNO (0.133), and DeepOHeat (0.149) are still significantly inferior to SAU-FNO (0.089). Ours-L (629M) achieved an RMSE of 0.055, a MAPE of 0.007, and a mean error of 0.025, indicating a 3.58-fold improvement in RMSE and a 2.06-fold improvement in maximum error. Memory consumption decreased from 48.3 GB for SAU-FNO to 13.3 GB (a 72.5% reduction), while training costs remained almost constant across resolutions, demonstrating strong scalability.

[0149]

[0150] Figure 5The results from the three chip designs are further illustrated. PNO-Therm accurately reconstructs the global temperature field and reliably captures hotspot structures, with both visual and numerical errors remaining at low levels. All visualizations were generated using Ours-B configuration to reduce computational overhead.

[0151] 7.3 Sensitivity Analysis of Training Data

[0152] To evaluate the data efficiency of PNO-Therm, we conducted a training sample sensitivity study on three chip designs with resolutions ranging from 64×64 to 151×151.

[0153] like Figure 4 As shown in (a)-(c), all error metrics decrease monotonically with increasing training sample size, with the most significant improvement observed in the low data region of 100-1000 samples. For example, on single-core and quad-core tasks, RMSE and MAE decrease by more than 70% when the training set is increased from 100 to 500 samples, demonstrating that PNO-Therm can quickly adapt to downstream hot tasks even with limited data.

[0154] In all three designs, once Reaching approximately 500-1000, PNO-Therm achieves or surpasses the accuracy of SAU-FNO trained on a full 4000-sample dataset. For example, in a quad-core design, the RMSE increases from... The RMSE dropped from 0.474 to 0.179 with 1000 samples; in the eight-core design, the RMSE dropped below 0.4 with 200 to 500 samples; in the single-core design, the model achieved an RMSE below 0.089 with approximately 1000 samples.

[0155] Overall, PNO-Therm requires only 15-20% of the complete data to match or exceed the accuracy achieved by state-of-the-art methods using all 4000 training samples.

[0156] 7.4 Comparison with simulation tools

[0157] To verify the actual utility of PNO-Therm, we compared its accuracy and speed with mainstream 3D-ICs thermal simulation tools, including commercial FEM software COMSOL

[28] (which is widely recognized for its high-fidelity FEM), academic FEM tool MTA

[18] , and FDM-based open-source tool Hotspot

[13] .

[0158] All tools run at their finest mesh, with a maximum mesh size of 0.02 mm and a minimum of 0.2 μm, to ensure fairness. Furthermore, all simulators support parallel execution on 16 physics cores. COMSOL Multiphysics uses its default multi-core parallel scheme (i.e., shared-memory multi-core parallelism), while MTA employs the Krylov subspace iterative solver, and HotSpot uses the SuperLU direct solver library.

[0159] We generated 20 power distribution cases for three designs and measured the highest and lowest temperatures.

[0160] As shown in Table 3, PNO-Therm's predictions are extremely close to COMSOL's: the largest absolute error in all tests is only 0.063 K, demonstrating near-high fidelity accuracy even in engineering-critical areas.

[0161] In terms of speed, we calculated the total simulation time, and COMSOL was relatively slow, while MTA and Hotspot were faster academic / open-source tools. PNO-Therm stood out: it took only 0.25 seconds per prediction on average—compared to 235.16 seconds for MTA and 89.57 seconds for Hotspot—achieving speedups of approximately 940.6 times and 358.3 times, respectively.

[0162] 8. Conclusion

[0163] In this work, PNO-Therm achieves a 6-10x improvement in accuracy compared to state-of-the-art (SOTA) methods, while reducing training GPU memory by 3.5x and increasing inference speed by more than 3x. Notably, it achieves SOTA-level accuracy using less than 20% of the data. These results highlight the potential of fundamental neural operators in scientific computing scenarios characterized by data scarcity, structural diversity, and high generalization requirements, enabling more efficient ML-based physics simulations in EDA and other fields.

[0164] References

[0165] [1] Hao Ai, Liang Chen, Jianhua Zhang, Bei Yu, and Wenxing Zhu. 2025.Fast Steady-State Thermal Analysis with Separation of Variables and DiscreteCosine Transform. IEEE Transactions on Computer-Aided Design of IntegratedCircuits and Systems (2025).

[0166] [2] Shengze Cai, Zhicheng Wang, Sifan Wang, Paris Perdikaris, andGeorge Em Karniadakis. 2021. Physics-informed neural networks for heattransfer problems. Journal of Heat Transfer 143, 6 (2021), 060801.

[0167] [3] Kun Cao, Junlong Zhou, Tongquan Wei, Mingsong Chen, Shiyan Hu,and Keqin Li. 2019. A survey of optimization techniques for thermal-aware 3Dprocessors. Journal of Systems Architecture 97 (2019), 397–415.

[0168] [4] Liang Chen, Wentian Jin, and Sheldon X-D Tan. 2022. Fast thermalanalysis for chiplet design based on graph convolution networks. In 2022 27thAsia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 485–492.

[0169] [5] Liang Chen, Jincong Lu, Wentian Jin, and Sheldon X-D Tan. 2023.Fast full-chip parametric thermal analysis based on enhanced physics enforcedneural networks. In 2023 IEEE / ACM International Conference on Computer AidedDesign (ICCAD). IEEE, 1–8.

[0170] [6] Liang Chen, Wenxing Zhu, Min Tang, Sheldon X-D Tan, Jun-Fa Mao,and Jianhua Zhang. 2024. PISOV: Physics-Informed Separation of VariablesSolvers for Full-Chip Thermal Analysis. IEEE Transactions on Computer-AidedDesign of Integrated Circuits and Systems (2024).

[0171] [7] Yi-Chung Chen, Scott Ladenheim, Harry Kalargaris, Milan Mihajlović, and Vasilis F Pavlidis. 2017. Computationally efficient standard-cell FEM-based thermal analysis. In 2017 IEEE / ACM International Conference onComputer-Aided Design (ICCAD). IEEE, 490–495.

[0172] [8] Hamideh Delaram, Ali Dastfan, and Mahmoud Norouzi. 2018. Optimalthermal placement and loss estimation for power electronic modules. IEEETransactions on Components, Packaging and Manufacturing Technology 8, 2(2018), 236–243.

[0173] [9] Mohammadamin Hajikhodaverdian, Sherief Reda, and Ayse K Coskun.2025. Fast Machine Learning Based Prediction for Temperature Simulation UsingCompact Models. In 2025 Design, Automation & Test in Europe Conference(DATE). IEEE, 1–2.

[0174]

[10] Zhongkai Hao, Chang Su, Songming Liu, Julius Berner, ChengyangYing, Hang Su, Anima Anandkumar, Jian Song, and Jun Zhu. 2024. Dpot: Auto-regressive denoising operator transformer for large-scale pde pre-training.arXiv preprint arXiv:2403.03542 (2024).

[0175]

[11] Maximilian Herde, Bogdan Raonic, Tobias Rohner, Roger Käppeli,Roberto Molinaro, Emmanuel de Bézenac, and Siddhartha Mishra. 2024. Poseidon:Efficient foundation models for pdes. Advances in Neural InformationProcessing Systems 37 (2024), 72525–72624.

[0176]

[12] Yue Hua, Zhi-Qiao Wang, Xin-Yi Yuan, Yu Bai Li, Wei-Tao Wu, andNadine Aubry. 2023. Estimation of steady-state temperature field in MultichipModules using deep convolutional neural network. Thermal Science andEngineering Progress 40 (2023), 101755.

[0177]

[13] Wei Huang, Shougata Ghosh, Sivakumar Velusamy, KarthikSankaranarayanan, Kevin Skadron, and Mircea R Stan. 2006. HotSpot: A compactthermal modeling methodology for early-stage VLSI design. IEEE Transactionson very large scale integration (VLSI) systems 14, 5 (2006), 501–513.

[0178]

[14] Zhen Huang, Hong Wang, Wenkai Yang, Muxi Tang, Depeng Xie, Ting-Jung Lin, Yu Zhang, Wei W Xing, and Lei He. 2025. Self-Attention to OperatorLearning-based 3D-IC Thermal Simulation. In 2025 62nd ACM / IEEE DesignAutomation Conference (DAC). IEEE, 1–7.

[0179]

[15] Wentian Jin, Sheriff Sadiqbatcha, Jinwei Zhang, and Sheldon X-DTan. 2020. Full-chip thermal map estimation for commercial multi-core CPUswith generative adversarial learning. In Proceedings of the 39thInternational Conference on Computer-Aided Design. 1–9.

[0180]

[16] Richard E Kessler. 1999. The alpha 21264 microprocessor. IEEEmicro 19, 2 (1999), 24–36.

[0181]

[17] Akhilesh Kumar, Norman Chang, David Geb, Haiyang He, StephenPan, Jimin Wen, Saeed Asgari, Mehdi Abarham, and Chris Ortiz. 2022. Ml-basedfast on-chip transient thermal simulation for heterogeneous 2.5 d / 3D ICdesigns. In 2022 International Symposium on VLSI Design, Automation and Test(VLSI-DAT). IEEE, 1–8.

[0182]

[18] Scott Ladenheim, Yi-Chung Chen, Milan Mihajlović, and Vasilis FPavlidis. 2018. The MTA: An advanced and versatile thermal simulator forintegrated systems. IEEE Transactions on Computer-Aided Design of IntegratedCircuits and Systems 37, 12 (2018), 3123–3136.

[0183]

[19] Peng Li, Lawrence T Pileggi, Mehdi Asheghi, and Rajit Chandra.2004. Efficient full-chip thermal modeling and analysis. In IEEE / ACMInternational Conference on Computer Aided Design, 2004. ICCAD-2004. IEEE,319–326.

[0184]

[20] Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, BurigedeLiu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. 2020. Fourierneural operator for parametric partial differential equations. arXiv preprintarXiv:2010.08895 (2020).

[0185]

[21] Jai-Ming Lin, Wei-Yi Chang, Hao-Yuan Hsieh, Ya-Ting Shyu, Yeong-Jar Chang, and Juin-Ming Lu. 2021. Thermal-aware floorplanning and TSV-planning for mixed-type modules in a fixed-outline 3-D IC. IEEE Transactionson Very Large Scale Integration (VLSI) Systems 29, 9 (2021), 1652–1664.

[0186]

[22] Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, YixuanWei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. 2022. Swin transformerv2: Scaling up capacity and resolution. In Proceedings of the IEEE / CVFconference on computer vision and pattern recognition. 12009–12019.

[0187]

[23] Ziyue Liu, Yixing Li, Jing Hu, Xinling Yu, Shinyu Shiau, Xin Ai,Zhiyu Zeng, and Zheng Zhang. 2023. DeepOHeat: operator learning-based ultra-fast thermal simulation in 3D-IC design. In 2023 60th ACM / IEEE DesignAutomation Conference (DAC). IEEE, 1–6.

[0188]

[24] Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer,Trevor Darrell, and Saining Xie. 2022. A convnet for the 2020s. InProceedings of the IEEE / CVF conference on computer vision and patternrecognition. 11976–11986.

[0189]

[25] Jincong Lu, Jinwei Zhang, and Sheldon X-D Tan. 2023. Real-timeThermal Map Estimation for AMD Multi-Core CPUs Using Transformer. In 2023IEEE / ACM International Conference on Computer Aided Design (ICCAD). IEEE, 1–7.

[0190]

[26] Jincong Lu, Jinwei Zhang, and Sheldon X-D Tan. 2023. Real-timethermal map estimation for amd multi-core cpus using transformer. In 2023IEEE / ACM International Conference on Computer Aided Design (ICCAD). IEEE, 1–7.

[0191]

[27] Lu Lu, Pengzhan Jin, and George Em Karniadakis. 2019. Deeponet:Learning nonlinear operators for identifying differential equations based onthe universal approximation theorem of operators. arXiv preprint arXiv:1910.03193 (2019).

[0192]

[28] COMSOL Multiphysics. 1998. Introduction to comsol multiphysics®.COMSOL Multiphysics, Burlington, MA, accessed Feb 9, 2018 (1998), 32.

[0193]

[29] Maziar Raissi, Paris Perdikaris, and George E Karniadakis. 2019.Physics-informed neural networks: A deep learning framework for solvingforward and inverse problems involving nonlinear partial differentialequations. Journal of Computational physics 378 (2019), 686–707.

[0194]

[30] Rishikesh Ranade, Haiyang He, Jay Pathak, Norman Chang, AkhileshKumar, and Jimin Wen. 2022. A thermal machine learning solver for chipsimulation. In Proceedings of the 2022 ACM / IEEE Workshop on Machine Learningfor CAD. 111–117.

[0195]

[31] Yanliang Sha, Chengxin Zhang, and Quan Chen. 2025. PI-ONet: APhysics-Informed Operator Network for Efficient Thermal Analysis ofMultilayer Chiplets. IEEE Transactions on Components, Packaging andManufacturing Technology (2025).

[0196]

[32] Hameedah Sultan, Anjali Chauhan, and Smruti R Sarangi. 2019. Asurvey of chip-level thermal simulators. ACM Computing Surveys (CSUR) 52, 2(2019), 1–35.

[0197]

[33] Mingyue Wang, Yuanqing Cheng, Weiheng Zeng, Zhenjie Lu, VasilisF Pavlidis, and Wei Xing. 2024. ARO: Autoregressive operator learning fortransferable and multi-fidelity 3D-IC thermal analysis with active learning.In Proceedings of the 43rd IEEE / ACM International Conference on Computer-Aided Design. 1–9.

[0198]

[34] Zhi-Qiao Wang, Yue Hua, Hao-Ran Xie, Zhi-Fu Zhou, Yu-Bai Li, andWei-Tao Wu. 2024. Transfer learning of convolutional neural network model forthermal estimation of multichip modules. Case Studies in Thermal Engineering59 (2024), 104576.

[0199]

[35] Gege Wen, Zongyi Li, Kamyar Azizzadenesheli, Anima Anandkumar,and Sally M Benson. 2022. U-FNO—An enhanced Fourier neural operator-baseddeep-learning model for multiphase flow. Advances in Water Resources 163(2022), 104180.

[0200]

[36] Jimin Wen, Stephen Pan, Norman Chang, Wen-Tze Chuang, Wenbo Xia,Deqi Zhu, Akhilesh Kumar, En-Cih Yang, Karthik Srinivasan, and Ying-Shun Li.2020. DNN-based fast static on-chip thermal solver. In 2020 36thSemiconductor Thermal Measurement, Modeling & Management Symposium (SEMI-THERM). IEEE, 65–75.

[0201]

[37] Luqiao Yin, Ao Wang, Wenxing Zhu, Aiying Guo, Jingjing Liu, MinTang, Liang Chen, and Jianhua Zhang. 2024. A Stepwise Integration Separationof Variables Solver for Full-Chip Thermal Uncertainty Analysis. IEEETransactions on Components, Packaging and Manufacturing Technology (2024).

[0202]

[38] Bingrui Zhang, Wei Xing, Xin Zhao, and Yuquan Sun. 2025. T-Fusion: Thermal Modeling of 3D ICs with Multi-fidelity Fusion. In Proceedingsof the 30th Asia and South Pacific Design Automation Conference. 1406–1412.

[0203]

[39] Yunyang Zhang, Zhiqiang Gong, Weien Zhou, Xiaoyu Zhao, XiaohuZheng, and Wen Yao. 2023. Multi-fidelity surrogate modeling for temperaturefield prediction using deep convolution neural network. EngineeringApplications of Artificial Intelligence 123 (2023), 106354.

[0204]

[40] Zijian Zhou, Min Tang, and Liang Chen. 2025. ASRR-PINN: AdaptiveSub-Regional Random Resampling-Based PINN for Thermal Analysis of 3D-ICs. In2025 62nd ACM / IEEE Design Automation Conference (DAC). IEEE, 1–7.

[0205]

[41] Tianxiang Zhu, Qipan Wang, Yibo Lin, Runsheng Wang, and RuHuang. 2024. FaStTherm: Fast and Stable Full-Chip Transient Thermal PredictorConsidering Nonlinear Effects. In Proceedings of the 43rd IEEE / ACMInternational Conference on Computer-Aided Design. 1–9.

Claims

1. A method for thermal simulation of 3D integrated circuits, characterized in that, Includes the following steps: Step 1: Obtain a base model pre-trained on a diverse PDE dataset including diffuse partial differential equations, the base model comprising a Transformer-based backbone network and convolutional neural network branches; Step 2: Fine-tune the pre-trained base model to adapt to the 3D integrated circuit thermal simulation task. The fine-tuning includes re-initializing the embedding layer and recovery layer to handle the specific input and output features of the 3D integrated circuit, while maintaining the pre-trained weights of the backbone network. The target dataset used for fine-tuning contains no more than 20% of the samples of the complete training data. Step 3: During fine-tuning, different learning rates are applied to different layers of the model, with a lower learning rate applied to the backbone network and a higher learning rate applied to the embedding and recovery layers. Step 4: During the fine-tuning process, optimization is performed using a combined loss function, which includes global fidelity loss and hot spot region loss, where the hot spot region loss emphasizes the prediction accuracy of high-temperature regions in the chip domain; Step 5: Use the fine-tuned model to predict the steady-state temperature field distribution of the 3D integrated circuit, paying particular attention to the temperature accuracy of the hot spot area of ​​the chip.

2. The method according to claim 1, characterized in that, Step 1 includes: Step 1.1: Obtain the base model pre-trained on a diverse PDE dataset, which includes PDE scenarios centered on diffusion-type dynamics, wherein the PDE scenarios contain diffusion-reaction partial differential equations with the following steady-state form: in, Represents a conserved quantity. The diffusion coefficient, representing spatial variation, Indicates the source term, Represents spatial coordinates; Step 1.2: The basic model adopts a hybrid spectrum-attention architecture, including an embedding layer, a convolutional neural network branch composed of multiple ConvNeXt blocks, a Transformer-based backbone network composed of SwinV2 stages, and a recovery layer; Step 1.3: The base model has learned a universal diffusion kernel applicable across domains during the pre-training phase, enabling the model to capture physical priors related to diffusion.

3. The method according to claim 2, characterized in that, Step 2 includes: Step 2.1: For the 3D integrated circuit thermal simulation task, the input features are determined to include power density. and thermal conductivity The output feature is determined to be a temperature field. ,in Representing spatial coordinates; the task is formulated as a parametric operator learning problem, with the goal of learning a solution operator: in, Represents the input function space. Represents the output function space. Indicates chip domain, Indicates learnable parameters, Satisfying the steady-state heat conduction equation ; Step 2.2: Reinitialize the embedding layer of the pre-trained base model. and recovery layer , where the embedding layer Used to map input features to patch markers, recovery layer Used to map high-resolution markers back to the temperature field; Step 2.3: Set the backbone network parameters of the pre-trained base model to pre-trained weights. The backbone network includes a SwinV2 stage and a ConvNeXt block. Step 2.4: Fine-tune the re-initialized embedding layer, recovery layer, and backbone network using the target dataset. The embedding layer and recovery layer are trained from scratch, and the backbone network is lightly fine-tuned.

4. The method according to claim 3, characterized in that, Step 3 includes: Step 3.1: Set the learning rate of the backbone network to the first learning rate. Set the learning rates of the embedding and recovery layers to the second learning rate. ,in ; Step 3.2: During fine-tuning, use the first learning rate. Update the parameters of the backbone network using the second learning rate. Update the parameters of the embedding layer and the recovery layer; where, , .

5. The method according to claim 4, characterized in that, Step 4 includes: Step 4.1: Calculate the global fidelity loss Its expression is: in, This represents the chip domain of a 3D integrated circuit. Indicates position The predicted temperature value at that location, Indicates position The actual temperature value at that location, Represents spatial coordinates, Chip domain The measure; Step 4.2: Define the critical temperature set ,in Represents the temperature field of -Quantities, ; Step 4.3: Calculate the loss in the hotspot area Its expression is: in, Represents the set of critical temperatures The measure; Step 4.4: Combine the global fidelity loss and the hotspot region loss into a total loss function: in, Weighting parameters to balance overall accuracy and hotspot precision; Step 4.5: During fine-tuning training, when training rounds... At that time, use the critical temperature set defined in step 4.

2. Calculate the loss for hotspot regions; during training epochs At that time, an adaptive critical temperature set is adopted: replace and use Calculate the loss in the hotspot area.

6. The method according to any one of claims 1-5, characterized in that, Step 5 includes: Step 5.1: Input the input features of the 3D integrated circuit into the fine-tuned model, wherein the input features include power density. and thermal conductivity ,in Represents spatial coordinates; Step 5.2: Perform forward propagation using the fine-tuned model, sequentially through the embedding layers. Mapping to patch markers, extracting local structure from ConvNeXt blocks, performing window-based self-attention processing in the SwinV2 stage, gradually restoring spatial resolution in the decoder, and the recovery layer. Mapping to generate the predicted steady-state temperature field distribution ; Step 5.3: Output the predicted steady-state temperature field distribution. And extract chip domain The highest temperature value in the chip is used as a key verification indicator for the accuracy of temperature in the hot spot area.

7. An apparatus for thermal simulation of 3D integrated circuits, comprising a processor and a storage medium, characterized in that, When the processor executes a program in the storage medium, it implements a method for thermal simulation of 3D integrated circuits as described in any one of claims 1-5.

8. A storage medium, characterized in that, When the processor executes a program in the storage medium, it implements a method for thermal simulation of 3D integrated circuits as described in any one of claims 1-5.

Images