Integrated circuit ct image enhancement method and system based on deep super-resolution network

Abstract

The application relates to an integrated circuit CT image enhancement method and system based on a deep super-resolution network, which comprises the following steps: acquiring an integrated circuit CT original image and performing first preprocessing to generate an image to be enhanced; and performing super-resolution reconstruction on the image to be enhanced by using a super-resolution generative adversarial network model to obtain a high-resolution enhanced image; wherein the super-resolution generative adversarial network model is obtained through fine tuning training based on an integrated circuit special CT image dataset, and the integrated circuit special CT image dataset comprises representative ROI samples containing metal interconnection lines, vias, dielectric layer interfaces and / or layer structure labels. Compared with the prior art, the application realizes high-quality structure detail reconstruction of an integrated circuit CT image, and solves the problems of blurring, large noise and unclear structure details commonly existing in the integrated circuit CT image.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to an integrated circuit CT image enhancement method and system based on a deep super-resolution network. Background Technology

[0002] Integrated circuits (ICs) possess highly precise internal structures, consisting of multilayered metal interconnects, dielectric materials, via structures, and complex layouts. As chip manufacturing processes continue to shrink, the scale of critical structures has reached the micrometer or even nanometer level, making the demand for high-quality non-destructive imaging increasingly urgent in manufacturing inspection, failure analysis, and packaging verification. X-ray computed tomography (CT), as an important three-dimensional non-destructive testing technology, is widely used for visualizing the internal structure and identifying defects in ICs. However, IC CT imaging is limited by multiple factors, including imaging equipment resolution, detector pixel size, system optical structure, scanning energy, and noise levels. This often results in reconstructed images exhibiting structural blurring, severe noise interference, loss of texture details, and excessively smoothed edges. Furthermore, high-density metal interconnect areas can produce artifacts due to scattering effects, making it difficult to accurately identify critical structures.

[0003] Traditional image enhancement methods, such as bilinear interpolation, bicubic interpolation, sharpening filtering, or denoising filtering, mainly process existing pixels through mathematical transformations. They cannot compensate for the high-frequency information that is actually lost during CT imaging, and they are also difficult to maintain the authenticity of the structure while improving clarity.

[0004] With the rapid development of deep learning, super-resolution (SR) reconstruction technology based on convolutional neural networks has provided a new approach to image enhancement. In particular, generative adversarial network models such as ESRGAN (Enhanced Super-Resolution Generative Adversarial Network) can effectively recover high-frequency textures and details by learning the LR-HR correspondence in a large number of samples. However, general SR models are usually trained on natural image datasets, and the texture structure patterns they learn differ significantly from the highly regular metal interconnect morphology, interlayer media structure, and weak contrast details in IC CT images. Directly applying these models to IC CT imaging can lead to texture inconsistencies, unstable reconstruction, or artifact enhancement, making it difficult to meet the stringent requirements for high-fidelity detail reconstruction in industrial inspection. Furthermore, existing image processing workflows are inefficient and non-scalable, making them difficult to apply to practical industrial needs. Therefore, there is an urgent need for a deep super-resolution enhancement technology specifically optimized for the characteristics of IC CT images to improve image clarity and structural discernibility. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art by providing a high-efficiency, highly applicable, and easily integrated method and system for enhancing integrated circuit CT images based on deep super-resolution networks. This method and system achieve high-quality structural detail reconstruction of integrated circuit CT images, enhance the visualization and analytical value of the images, and solve the problems of blurriness, high noise, and unclear structural details that are common in integrated circuit CT images.

[0006] The objective of this invention can be achieved through the following technical solutions: A method for enhancing CT images of integrated circuits based on deep super-resolution networks includes the following steps: Acquire the original integrated circuit CT image and perform the first preprocessing to generate the image to be enhanced; A super-resolution generative adversarial network model is used to perform super-resolution reconstruction on the image to be enhanced, thereby obtaining a high-resolution enhanced image. The super-resolution generative adversarial network model is fine-tuned and trained based on a dedicated CT image dataset for integrated circuits. This dataset includes representative ROI samples with annotations of metal interconnect lines, vias, dielectric layer interfaces, and / or layered structures.

[0007] Furthermore, the first preprocessing includes multiple processes such as size normalization, noise suppression, and format conversion.

[0008] Furthermore, the image to be enhanced is an image sequence containing multiple images.

[0009] Furthermore, the super-resolution generative adversarial network model is a pre-trained deep super-resolution generative adversarial network model based on ESRGAN.

[0010] Furthermore, the construction of the dedicated CT image dataset for integrated circuits specifically includes: High-resolution IC CT images are collected as high-resolution images (HR). The imaging degradation of each high-resolution image (HR) is simulated to generate a corresponding low-resolution image (LR), forming a pair of (LR, HR) training pairs. Some of the (LR, HR) training pairs are marked with metal interconnect lines, vias, dielectric layer interfaces and / or layered structure information, forming the representative ROI sample. A second preprocessing is performed on each (LR, HR) training pair to construct the integrated circuit-specific CT image dataset.

[0011] Furthermore, the second preprocessing includes multiple processes such as normalization, elimination, and enhancement, and the registration consistency between LR and HR is maintained during the second preprocessing.

[0012] Furthermore, the process of fine-tuning the super-resolution generative adversarial network model based on an integrated circuit-specific CT image dataset includes: Obtain a pre-trained super-resolution generative adversarial network model, use the pre-trained weights as weights, and configure the model's input and output scales to match the CT image size / magnification. A phased fine-tuning strategy was adopted to train the super-resolution generative adversarial network model; The phased fine-tuning strategy includes: In the reconstruction fidelity fine-tuning stage, a loss function is constructed using pixel reconstruction loss for training. In the perceptual detail enhancement fine-tuning stage, adversarial loss and / or perceptual / texture loss are introduced to construct a loss function for training.

[0013] Furthermore, the fine-tuning training is performed multiple times, and the proportion of representative ROI samples is gradually increased during training sampling.

[0014] The present invention also provides an integrated circuit CT image enhancement system based on a deep super-resolution network, comprising: The image input and preprocessing module is used to preprocess raw images from integrated circuit CT scans to generate images to be enhanced. The dataset management and model training fine-tuning module is used to construct a dedicated CT image dataset for integrated circuits, and to fine-tune and train the super-resolution generative adversarial network model based on the dedicated CT image dataset for integrated circuits, so that the super-resolution generative adversarial network model can learn the internal metal interconnect structure, dielectric layer distribution and high-frequency texture features of integrated circuits. The super-resolution reconstruction inference module is used to load the super-resolution generative adversarial network model after fine-tuning and training, and to perform super-resolution reconstruction on the preprocessed image to be enhanced to generate a high-resolution enhanced image. The image visualization and comparison display module is used to visualize the high-resolution enhanced image and the corresponding original image. The system control and parameter configuration module is used to configure and control the magnification, image block size, number of inference threads, and model running parameters.

[0015] Furthermore, the system also includes an ROI cropping module for cropping CT images to extract ROI regions, which are then used for super-resolution reconstruction in the super-resolution reconstruction inference module.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention employs a super-resolution generative adversarial network model finely trained on a dedicated CT image dataset for integrated circuits for super-resolution reconstruction. The model can learn specific layered structures, high-frequency texture features, metal interconnect structures, and dielectric layer distribution patterns within the IC, solving problems such as reconstruction instability / artifacts caused by the mismatch between general super-resolution reconstruction models and IC CT texture patterns. CT images processed by this invention significantly outperform traditional image enhancement methods in detail restoration, noise suppression, and structural boundary sharpening, more accurately presenting the internal structure of the IC and providing reliable, high-quality image support for defect detection, structural analysis, packaging verification, and reverse engineering. This invention features high processing efficiency, strong applicability, and ease of integration, making it suitable for industrial inspection, electronic manufacturing, and related engineering fields. Specifically: (1) Significantly improves image clarity, enabling accurate identification of fine structures.

[0017] The super-resolution generative adversarial network model trained by this invention can compensate for the high-frequency information lost in CT imaging, so that small structures such as metal interconnect lines, via structures, and defect edges can present higher contrast and detail continuity after enhancement. It effectively avoids the blur diffusion and structural breakage problems caused by traditional algorithms, and provides a more accurate image basis for inspection personnel and algorithm analysis.

[0018] (2) By fine-tuning the IC-specific dataset, it is possible to reconstruct real and reliable IC-specific textures and structures.

[0019] This invention fine-tunes the model based on the material properties, structural distribution, and texture patterns of IC CT images. This not only enhances the clarity of the model but also accurately restores the unique regular circuit structure, layered features, and local geometry of ICs. This avoids the generation of textures or pseudo-structures that do not conform to physical laws by general GAN ​​models, thereby improving the reliability of the reconstruction results.

[0020] (3) Effectively restore key structural information such as interlayer interfaces and improve the recognizability of material boundaries and geometric details.

[0021] This invention supports deep texture modeling and edge enhancement of the multi-layer structure inside ICs, which can clearly restore the originally blurred dielectric layer interface, metal layer boundary and thin layer structure, providing more reliable visual basis for packaging inspection, material analysis, defect location and so on, and improving the diagnostic accuracy in engineering applications.

[0022] (4) Experiments show that the present invention can: improve the peak signal-to-noise ratio (PSNR) by 2–5 dB; improve the structural similarity index (SSIM) by 5–15%; significantly enhance the visibility of fine metal lines and interlayer structures; and improve the accuracy of defect detection by 20–40%.

[0023] 2. This invention integrates image preprocessing, super-resolution reconstruction, result visualization, and batch processing modules. It supports flexible adjustment of model types and parameters according to task requirements and provides the ability to compare original and enhanced images. CT images processed by this system show significantly improved edge sharpness, texture continuity, and structural details, more accurately presenting key structural features within the IC. This helps improve imaging quality and analytical reliability in applications such as defect detection, material analysis, and reverse engineering. This invention's system has advantages such as strong scalability, high processing efficiency, and significant enhancement effects, effectively overcoming the shortcomings of existing technologies in IC CT imaging. Specifically: (1) The software has a high degree of integration and supports efficient and scalable processing flow.

[0024] This invention constructs a complete image enhancement software system, integrating functions such as batch processing, model selection, parameter adjustment, real-time logging, before-and-after comparison display, and ROI cropping. It is suitable for various imaging scenarios and large-scale industrial inspection tasks. The system also possesses high robustness and high compatibility, scalable to multiple CT image formats and different hardware environments, enabling enterprise-level deployment.

[0025] (2) Enhanced results have higher visualization and analytical value.

[0026] The CT images enhanced by this invention are closer to real high-resolution images in terms of visual quality, with natural textures, stable edges, and significant noise suppression. This can significantly improve the accuracy and reliability of engineers' manual judgment, automatic defect detection algorithms, and reverse engineering processes. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the model training process of the present invention; Figure 2 This is a schematic diagram of the operation process of the system of the present invention. Detailed Implementation

[0028] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0029] Example 1 This embodiment provides an integrated circuit CT image enhancement method based on a deep super-resolution network, including the following steps: acquiring the original integrated circuit CT image and performing a first preprocessing, including size normalization, noise suppression, format conversion, etc., to generate an image to be enhanced, which can be an image sequence containing multiple images; using a super-resolution generative adversarial network model to perform super-resolution reconstruction on the image to be enhanced to obtain a high-resolution enhanced image.

[0030] Furthermore, the super-resolution generative adversarial network model is fine-tuned and trained based on a dedicated CT image dataset for integrated circuits, which includes representative ROI samples containing annotations of metal interconnect lines, vias, dielectric layer interfaces, and / or layered structures.

[0031] Specifically, in this embodiment, the super-resolution generative adversarial network model is a pre-trained deep super-resolution generative adversarial network model based on ESRGAN.

[0032] like Figure 1 As shown, the fine-tuning training process in this embodiment specifically includes: (1) Dataset construction High-resolution IC CT images are collected as high-resolution images (HR); corresponding low-resolution images (LR) are generated through imaging degradation simulation to form strictly paired (LR, HR) training pairs; additional annotation / screening is performed on representative ROIs containing "metal interconnect lines, vias, dielectric layer interfaces, and layered structures" to improve the structure learning density.

[0033] Specifically, high-resolution IC CT images can be images acquired by high-resolution equipment or results of high-quality reconstruction.

[0034] (2) Preprocessing and data augmentation A second preprocessing is performed on each (LR, HR) training pair to construct the integrated circuit-specific CT image dataset.

[0035] In this embodiment, the second preprocessing specifically includes: normalization, removal of obviously bad / abnormal slices; enhancement of samples by random cropping (patch), rotation / flipping, noise perturbation, etc., while maintaining LR-HR registration consistency.

[0036] (3) Initialization and migration Using pre-trained weights from the generic ESRGAN as initialization, the model's input and output scales are configured to match the CT image size / magnification.

[0037] (4) Phased fine-tuning strategy Phase 1 (Reconstruction Fidelity Fine-tuning): Primarily using pixel reconstruction loss (L1), the model first learns the consistency of CT structure boundaries and grayscale, avoiding the introduction of pseudo-structures in the early stages of GAN. Phase 2 (Perceptual Detail Enhancement Fine-tuning): Adversarial loss and perceptual / texture loss are gradually introduced to restore high-frequency textures in the model without compromising structural realism, thereby improving the clarity and continuity of fine metal interconnect lines, via edges, and dielectric layer interfaces.

[0038] (5) Training constraints for IC architecture Increasing the proportion of ROIs containing key structures during training sampling and weighting boundary / interlayer interface regions makes the model pay more attention to "continuity of interconnecting lines, sharpening of layered interfaces, and weak contrast details".

[0039] The beneficial effects of the above-mentioned fine-tuning training include: (1) Solve the problem of general model domain mismatch: By using the dedicated CT image dataset of integrated circuits and directional fine-tuning, the model learns the internal regular structure and high-frequency texture patterns of IC, avoiding the problems of texture inconsistency, artifact enhancement and unstable reconstruction of general GAN ​​on IC CT.

[0040] (2) Improve the recognizability of key structures: After fine-tuning, the model restores the structural boundaries of metal interconnects, vias, dielectric layer interfaces, etc. more sharply and with better continuity. While suppressing noise, it reduces the edge smoothing, which is more conducive to defect detection, structural analysis and other applications.

[0041] (3) Reduce computing power consumption and training cost: By adopting the strategy of fine-tuning training based on the pre-trained model, domain adaptation can be achieved by only performing a small number of iterations on the data of the integrated circuit CT scenario. There is no need to train the complete generative adversarial network model from scratch, which significantly reduces the amount of data, training rounds and computing resources required for training, reduces computing power consumption and overall training cost, and shortens the model adaptation and engineering deployment cycle.

[0042] Experiments show that the above methods can achieve the following: PSNR improvement of 2–5 dB; SSIM improvement of 5–15%; significantly enhanced visibility of fine metal lines and interlayer structures; and improved defect detection accuracy by 20–40%.

[0043] If the above methods are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0044] Example 2 This embodiment provides an integrated circuit CT image enhancement system based on a deep super-resolution network, including: The image input and preprocessing module is used to receive raw images from the integrated circuit CT and perform preprocessing operations such as size normalization, noise suppression, and format conversion on the raw images to form an image to be enhanced. The dataset management and model training fine-tuning module is used to construct a dedicated CT image dataset for integrated circuits, and to fine-tune and train a super-resolution generative adversarial network model based on the dataset, so that the model can learn the internal metal interconnect structure, dielectric layer distribution and high-frequency texture features of integrated circuits. The super-resolution reconstruction inference module is used to load the finely tuned super-resolution generative adversarial network model, perform super-resolution reconstruction on the preprocessed image to be enhanced, and generate a high-resolution enhanced image. The image visualization and comparison display module is used to visualize the high-resolution enhanced image and the corresponding original image, and to provide before-and-after image comparison function; The ROI cropping module is used to crop CT images to extract ROI regions. The extracted ROI regions can then be reconstructed in the super-resolution reconstruction inference module. The system control and parameter configuration module is used to configure and control the model type, magnification, inference parameters, and processing flow. The log recording and historical task management module fully records the model inference process, including running logs, error messages, processing time, and other information.

[0045] The above system has the following functions: (1) Supports batch image uploading and processing Multiple CT images can be imported at once, and a pre-trained super-resolution model can be loaded to achieve an efficient batch image enhancement workflow.

[0046] (2) Real-time display of model processing speed and task progress The system can dynamically display the processing time, average speed, and overall progress of the current image during operation, allowing users to keep track of the model's inference status in real time.

[0047] (3) Provide an interactive image interface for comparing before and after effects. The user interface integrates sliding vertical dividers, allowing users to visually view the differences between the enhanced and unenhanced images by dragging the comparison bars, making it easy to evaluate the super-resolution effect.

[0048] (4) Supports flexible selection and configuration of models and parameters. Users can select different super-resolution network models from the interface and customize relevant parameters (such as magnification, tile size, number of threads, etc.).

[0049] (5) It has log recording and history viewing functions. The system can fully record the model inference process, including running logs, error messages, processing time, and other information, and supports browsing historical task records for easy tracking and analysis.

[0050] (6) Provide image cropping functionality to extract Region of Interest (ROI) Users can crop CT images in the interface and use the target area as the model input to achieve refined enhancement of local structures.

[0051] like Figure 2 As shown, the operation flow of the above system includes: S1: The user uploads a batch of IC CT images to be analyzed and processed; S2: Preprocessing and selecting model parameters; S3: Load the finely tuned ESRGAN-CT model; S4: The system automatically performs batch inference and displays the progress in real time. Each image enhancement takes about 0.5 to 2 seconds. S5: After processing, users can view the enhanced results by sliding the separators for comparison; S6: Finally, export the enhanced high-resolution CT images for detection and analysis.

[0052] The rest is the same as in Example 1.

[0053] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0054] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0055] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for enhancing integrated circuit CT images based on deep super-resolution networks, characterized in that, Includes the following steps: Acquire the original integrated circuit CT image and perform the first preprocessing to generate the image to be enhanced; A super-resolution generative adversarial network model is used to perform super-resolution reconstruction on the image to be enhanced, thereby obtaining a high-resolution enhanced image. The super-resolution generative adversarial network model is fine-tuned and trained based on a dedicated CT image dataset for integrated circuits. This dataset includes representative ROI samples with annotations of metal interconnect lines, vias, dielectric layer interfaces, and / or layered structures.

2. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 1, characterized in that, The first preprocessing includes multiple processes such as size normalization, noise suppression, and format conversion.

3. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 1, characterized in that, The image to be enhanced is an image sequence containing multiple images.

4. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 1, characterized in that, The super-resolution generative adversarial network model is a pre-trained deep super-resolution generative adversarial network model based on ESRGAN.

5. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 1, characterized in that, The construction of the dedicated CT image dataset for integrated circuits specifically includes: High-resolution IC CT images are collected as high-resolution images (HR). The imaging degradation of each high-resolution image (HR) is simulated to generate a corresponding low-resolution image (LR), forming a pair of (LR, HR) training pairs. Some of the (LR, HR) training pairs are marked with metal interconnect lines, vias, dielectric layer interfaces and / or layered structure information, forming the representative ROI sample. A second preprocessing is performed on each (LR, HR) training pair to construct the integrated circuit-specific CT image dataset.

6. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 5, characterized in that, The second preprocessing includes various methods such as normalization, elimination, and enhancement, and the registration consistency between LR and HR is maintained during the second preprocessing.

7. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 1, characterized in that, The process of fine-tuning the super-resolution generative adversarial network model based on the integrated circuit-specific CT image dataset includes: Obtain a pre-trained super-resolution generative adversarial network model, use the pre-trained weights as weights, and configure the model's input and output scales to match the CT image size / magnification. A phased fine-tuning strategy was adopted to train the super-resolution generative adversarial network model; The phased fine-tuning strategy includes: In the reconstruction fidelity fine-tuning stage, a loss function is constructed using pixel reconstruction loss for training. In the perceptual detail enhancement fine-tuning stage, adversarial loss and / or perceptual / texture loss are introduced to construct a loss function for training.

8. The integrated circuit CT image enhancement method based on deep super-resolution networks according to claim 7, characterized in that, The fine-tuning training is performed multiple times, and the proportion of representative ROI samples is gradually increased during training sampling.

9. An integrated circuit CT image enhancement system based on a deep super-resolution network, characterized in that, include: The image input and preprocessing module is used to preprocess raw images from integrated circuit CT scans to generate images to be enhanced. The dataset management and model training fine-tuning module is used to construct a dedicated CT image dataset for integrated circuits, and to fine-tune and train the super-resolution generative adversarial network model based on the dedicated CT image dataset for integrated circuits, so that the super-resolution generative adversarial network model can learn the internal metal interconnect structure, dielectric layer distribution and high-frequency texture features of integrated circuits. The super-resolution reconstruction inference module is used to load the super-resolution generative adversarial network model after fine-tuning and training, and to perform super-resolution reconstruction on the preprocessed image to be enhanced to generate a high-resolution enhanced image. The image visualization and comparison display module is used to visualize the high-resolution enhanced image and the corresponding original image. The system control and parameter configuration module is used to configure and control the magnification, image block size, number of inference threads, and model running parameters.

10. The integrated circuit CT image enhancement system based on deep super-resolution networks according to claim 9, characterized in that, The system also includes an ROI cropping module for cropping CT images to extract ROI regions, which are then used for super-resolution reconstruction in the super-resolution reconstruction inference module.

Images