A defect detection method for high-density packaged chips

Abstract

The application discloses a kind of defect detection methods for high-density packaging chip, it is related to semiconductor detection technical field, including, to high-density packaging chip is pretreated;Utilize multispectral imaging equipment, the high-density packaging chip after pretreatment is imaged, obtains multispectral imaging image group;Sparse representation multispectral imaging image group is obtained by adopting the technology of multi-stage decomposition based on Haar wavelet discrete wavelet transform first, then thresholding processing;Compressed sensing is carried out to sparse representation multispectral imaging image group based on compressed sensing technology, and the multispectral imaging image group after compressed sensing is obtained;The multispectral imaging image group after compressed sensing is labeled, and the visual Transformer model of RGB input is transformed into the visual Transformer model of multi-band input, and the visual Transformer model of multi-channel input is trained.The application improves the accuracy of high-density packaging chip defect detection by multispectral imaging, sparse representation and compressed sensing technology.

Description

Technical Field

[0001] This invention relates to the field of semiconductor inspection technology, and in particular to a defect detection method for high-density packaged chips. Background Technology

[0002] In the semiconductor manufacturing industry, the production process of high-density packaged chips faces numerous challenges, especially defect detection. Traditional manual visual inspection and single electromechanical or optical inspection methods have significant limitations, such as reliance on operator experience, unstable inspection results, insufficient resolution, and slow inspection speed. As chip density increases, these methods can no longer meet the demands for high-precision and high-efficiency inspection.

[0003] Furthermore, traditional compressed sensing methods have shortcomings in image reconstruction and denoising, which may lead to image quality degradation and further affect the accuracy of defect detection. While deep learning methods perform well in image classification and object detection, traditional models are mainly based on RGB images and cannot fully utilize the multi-band information of multispectral imaging. These limitations severely restrict the defect detection performance and production efficiency of high-density packaged chips. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a defect detection method for high-density packaged chips to solve the problem of insufficient accuracy of traditional defect detection methods in the detection of high-density packaged chips.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a defect detection method for high-density packaged chips, comprising preprocessing the high-density packaged chip;

[0008] Using a multispectral imaging device, the pre-processed high-density packaged chip is imaged to obtain a multispectral imaging image set;

[0009] The discrete wavelet transform technique based on Haar wavelets is used to first perform multi-level decomposition, and then thresholding is performed to obtain a sparse representation of the multispectral imaging image group.

[0010] Compressed sensing technology is used to compress and sense the image group of sparse multispectral imaging to obtain the image group of multispectral imaging after compressed sensing.

[0011] The image group of multispectral imaging after compressed sensing is labeled, and the RGB input visual Transformer model is transformed into a multi-band input visual Transformer model. The multi-channel input visual Transformer model is then trained.

[0012] By using a trained multi-channel input visual Transformer model, the defect region of the image group after compressed sensing and multispectral imaging is located and identified to obtain the defect results of high-density packaged chip.

[0013] Based on the defect results of high-density packaged chips, the defect information is analyzed and a report is generated.

[0014] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the pretreatment includes surface cleaning, surface drying, surface planarization, optical optimization, and static electricity removal, with the specific steps as follows.

[0015] The surface of the high-density packaged chip was purged with pure nitrogen gas.

[0016] The high-density packaged chip is suspended and fixed by an acoustic levitation device, and low-temperature plasma is applied to the surface of the high-density packaged chip to clean organic pollutants.

[0017] Use deionized water to rinse the high-density packaged chips after they have been cleaned of organic contaminants.

[0018] A femtosecond laser drying device was used to perform a full-coverage scan of the surface of the high-density packaged chip after rinsing;

[0019] A high-precision micropolishing machine is used to planarize the surface of the high-density packaged chip after full-coverage scanning.

[0020] Optical optimization was performed on the high-density packaged chip after surface planarization using a physical vapor deposition coating device.

[0021] An ion gun is used to blow air through the optically optimized high-density packaged chip to remove static electricity, resulting in a pre-treated high-density packaged chip.

[0022] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, a multispectral imaging device is used to image the preprocessed high-density packaged chip to obtain a multispectral image set. The specific steps are as follows.

[0023] The pre-treated high-density packaged chip is fixed onto the stage of the multispectral imaging device using vacuum adsorption and mechanical clamps.

[0024] Multispectral imaging equipment automatically switches the light source and corresponding filter according to imaging requirements;

[0025] Using a multispectral imaging device, high-density packaged chips are imaged band by band for different combinations of light sources and filters, resulting in preliminary multispectral imaging images for each band of the visible light band, near-infrared light band, short-wave infrared light band, ultraviolet light band, and thermal infrared light band.

[0026] Image denoising is performed on the preliminary multispectral imaging images of each band based on nonlocal mean filtering;

[0027] The initial multispectral images of each band after image denoising are enhanced for contrast using histogram equalization;

[0028] The images of each band after contrast enhancement are aligned using an existing image registration algorithm based on feature point matching to obtain a group of multispectral images.

[0029] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the method employs Haar wavelet-based discrete wavelet transform technology to first perform multi-level decomposition, followed by thresholding processing to obtain a sparsely represented multispectral imaging image set. The specific steps are as follows:

[0030] The discrete wavelet transform based on Haar wavelets is used to perform multi-level decomposition on the multispectral imaging image group to obtain low-frequency approximate sub-bands and high-frequency detail sub-bands;

[0031] Thresholds are set based on the standard deviation of each band of the image, and wavelet coefficients smaller than the threshold in the high-frequency detail subband are set to zero for thresholding.

[0032] By fusing the low-frequency approximate subband and the thresholded high-frequency detail subband, a multispectral imaging image set with sparse representation is obtained.

[0033] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the following steps are taken: Compressed sensing technology is used to perform compressed sensing on a sparsely represented multispectral imaging image set to obtain a compressed-sensing multispectral imaging image set.

[0034] Random sampling is performed on the sparsely represented multispectral imaging image group using a Gaussian random matrix;

[0035] The orthogonal matching pursuit algorithm is used to reconstruct each multispectral image in a group of randomly sampled multispectral images;

[0036] By using total variational regularization, the reconstructed multispectral imaging image is processed to obtain a multispectral imaging image set after compressed sensing.

[0037] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the following steps are taken: The image group of multispectral imaging after compressed sensing is labeled, and the RGB input visual Transformer model is transformed into a multi-band input visual Transformer model. The multi-channel input visual Transformer model is then trained.

[0038] A large number of compressed sensing multispectral image sets are labeled to obtain an image dataset for multispectral imaging of high-density packaged chips.

[0039] Replace the input layer of the RGB input visual Transformer model with a multi-channel input input layer;

[0040] Learnable band weights are used to capture complementary information between bands, and a weighted loss function is combined to optimize the visual Transformer model.

[0041] Based on the image dataset of high-density packaged chip multispectral imaging, the optimized visual Transformer model is pre-trained.

[0042] Based on the pre-trained visual Transformer model, fine-tuning training is performed on the image dataset of high-density packaged chip multispectral imaging to obtain a trained visual Transformer model with multi-channel input.

[0043] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the defect regions of the high-density packaged chip are located and identified using a trained multi-channel input visual Transformer model, and the defect results are obtained from the multispectral imaging image group after compressed sensing. The specific steps are as follows.

[0044] The compressed sensing multispectral image group is input into the trained multi-channel input visual Transformer model to capture the global features of the image, locate and segment the defect region, and generate a defect segmentation map of the multispectral image.

[0045] The Canny edge detection algorithm is used to accurately locate defects in multispectral images to obtain defect results for high-density packaged chips.

[0046] As a preferred embodiment of the defect detection method for high-density packaged chips described in this invention, the method involves: using pandas to read the defect results of the high-density packaged chips, performing statistical analysis on all detected defects, and analyzing the defect trends existing in the chip manufacturing process.

[0047] Use the seaborn library to generate defect type distribution maps and defect trend maps;

[0048] Generate a PDF report using a defect type distribution map and a defect trend map.

[0049] In a second aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein the computer program, when executed by the processor, implements any step of the defect detection method for high-density packaged chips as described in the first aspect of the present invention.

[0050] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the defect detection method for high-density packaged chips as described in the first aspect of the present invention.

[0051] The beneficial effects of this invention are as follows: By employing multispectral imaging, sparse representation, compressed sensing, and deep learning technologies, the accuracy of defect detection in high-density packaged chips is improved. Multispectral imaging equipment acquires image information from different bands, providing more comprehensive defect features and overcoming the limitations of traditional single-band imaging, thus improving detection accuracy. Sparse representation based on Haar wavelet discrete wavelet transform, combined with compressed sensing technology, reduces data volume, accelerates processing speed, and maintains high image quality, solving the problem of low processing efficiency in traditional methods. The RGB-input visual Transformer model is transformed into a multi-band input model, and learnable band weights improve the model's performance and robustness, further enhancing detection accuracy. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the defect detection method for high-density packaged chips in Example 1.

[0054] Figure 2 This is a flowchart of the visual Transformer model for acquiring multi-channel input in Example 1. Detailed Implementation

[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0056] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0058] Example 1, referring to Figure 1 and Figure 2 This is the first embodiment of the present invention, which provides a defect detection method for high-density packaged chips, including the following steps:

[0059] S1. Preprocess the high-density packaged chip, including the following steps.

[0060] The surface of the high-density packaged chip is purged with pure nitrogen gas. Specifically, the nitrogen flow rate should be controlled at 10–20 L / min, and the purging time should be 30–60 seconds to ensure that the surface is free of dust and static electricity residue.

[0061] The high-density packaged chip is suspended and fixed after being purged using an acoustic levitation device, and then a low-temperature plasma is applied to the surface of the chip to clean organic contaminants. Specifically, the processing power of the low-temperature plasma should be set at 50–200W, the processing time at 10–30 seconds, and either argon or oxygen can be used as the plasma gas, with a flow rate of 50–100 sccm. During the cleaning process, the chip must be kept suspended to avoid contact with contaminants.

[0062] High-density packaged chips that have been cleaned of organic contaminants are rinsed with deionized water.

[0063] A femtosecond laser drying system was used to perform a full-coverage scan of the surface of the rinsed high-density packaged chip. Specifically, the laser wavelength of the femtosecond laser drying system was 800–1050 nm, the pulse width was 100–500 fs, the scanning speed was 1–5 mm / s, and the power density was 0.5–2 W / cm². 2 The drying process should cover the chip surface to ensure no moisture remains.

[0064] A high-precision micropolishing machine is used to planarize the surface of high-density packaged chips after full-coverage scanning. Specifically, the polishing material of the high-precision micropolishing machine should be nano-sized polishing paste, the polishing pressure should be 0.1–0.5 MPa, and the polishing time should be 30–90 seconds to ensure that the chip surface flatness reaches the nanometer level.

[0065] Optical optimization was performed on the high-density packaged chip after surface planarization using physical vapor deposition coating equipment.

[0066] An ion gun is used to blow air through the optically optimized high-density packaged chip to remove static electricity, resulting in a pre-treated high-density packaged chip.

[0067] It should be noted that low-temperature plasma is a partially ionized gas containing a large number of active particles (such as free radicals, ions, and excited-state molecules). These active particles can chemically react with organic contaminants on the chip surface, decomposing them into volatile substances, thereby removing the contaminants. Physical vapor deposition (PVD) is a technique that deposits materials onto the chip surface in a vacuum environment through physical processes (such as evaporation and sputtering). By selecting appropriate coating materials, the optical properties of the chip surface can be improved, thus enhancing image quality.

[0068] S2. Using a multispectral imaging device, image the preprocessed high-density packaged chip to obtain a multispectral image set, including the following steps.

[0069] The pre-treated high-density packaged chip is secured to the stage of the multispectral imaging equipment using vacuum adsorption and mechanical clamps. Specifically, ensure the stage of the multispectral imaging equipment is clean and dust-free. Place the pre-treated high-density packaged chip on the stage. Using the vacuum adsorption device, turn on the vacuum pump to ensure the chip is firmly attached to the stage. Use the mechanical clamps to gently secure the edges of the chip, ensuring it does not move during imaging.

[0070] Multispectral imaging equipment automatically switches between light sources and corresponding filters according to imaging requirements. It should be noted that the visible light band has a wavelength range of 400–700 nm, generated by a white LED and filtered using a broadband filter; the near-infrared band has a wavelength range of 700–1100 nm, generated by a near-infrared LED or halogen lamp and filtered using a near-infrared filter; the short-wave infrared band has a wavelength range of 1100–2500 nm, generated by a short-wave infrared LED or halogen lamp and filtered using a short-wave infrared filter; the ultraviolet band has a wavelength range of 200–400 nm, generated by an ultraviolet LED or mercury lamp and filtered using an ultraviolet filter; and the thermal infrared band has a wavelength range of 3000–5000 nm, generated by a heat source (such as a blackbody radiation source) and filtered using a thermal infrared filter.

[0071] Using a multispectral imaging device, high-density packaged chips are imaged band by band for different combinations of light sources and filters, resulting in preliminary multispectral images for each band: visible light, near-infrared light, short-wave infrared light, ultraviolet light, and thermal infrared light. It should be noted that proper exposure time and gain settings are crucial to ensure high-quality multispectral images are acquired in different bands.

[0072] Image denoising is performed on the preliminary multispectral images for each band based on nonlocal mean filtering. Specifically, the preliminary multispectral images for each band are converted into grayscale images, and the `cv2.fastNlMeansDenoising` function from the OpenCV library is used to denoise the preliminary multispectral images for each band. It should be noted that the `cv2.fastNlMeansDenoising` function removes noise by calculating the similarity of each pixel in the image and using a weighted average of similar pixels.

[0073] After denoising, the preliminary multispectral images of each band are enhanced for contrast using histogram equalization. Specifically, the `cv2.equalizeHist` function from the OpenCV library is used programmatically to perform histogram equalization on each band. It should be noted that the `cv2.equalizeHist` function enhances contrast by adjusting the grayscale distribution of image pixels, making the grayscale values ​​more uniform, especially effective when processing images with uneven brightness or low contrast.

[0074] The preliminary multispectral images of each band after contrast enhancement are aligned using existing feature point matching-based image registration algorithms to obtain a group of multispectral images. Specifically, the ORB feature point detection algorithm (Oriented Fast Feature and Rotated Binary Descriptor) is used to find salient feature points in the preliminary multispectral images of each band, and matching is performed using a combination of BFMatcher (brute force matcher) and K-nearest neighbor algorithm. The matching results are then filtered using Lowe's ratio test. A random sample consensus algorithm is used to estimate the global transformation matrix of the image. Using the estimated transformation matrix, images of other bands are aligned with the reference image to ensure that each band in the multispectral image is in the same coordinate system. It should be noted that the brute force matcher is a simple and direct feature point matching method. It finds the best match by comparing every possible pairing of one feature descriptor with another. The K-nearest neighbor algorithm is combined to improve the efficiency of the brute force matcher.

[0075] S3. A multi-level decomposition is first performed using Haar wavelet-based discrete wavelet transform technology, followed by thresholding to obtain a sparse representation of the multispectral imaging image set. This includes the following steps:

[0076] This paper utilizes Haar wavelet-based discrete wavelet transform to perform multi-level decomposition on a multispectral image set, obtaining low-frequency approximate sub-bands and high-frequency detail sub-bands. Specifically, the `pywt.wavedec2` function from the PyWavelets library is used programmatically to process the multispectral image set, thereby implementing multi-level decomposition of the multispectral image set using Haar wavelet-based discrete wavelet transform.

[0077] Thresholding is performed by setting a threshold based on the standard deviation of each band of the image, and setting wavelet coefficients smaller than the threshold in the high-frequency detail subbands to zero. Specifically, the standard deviation of the image pixels is first calculated, and a hard threshold is determined based on the standard deviation. Then, the `pywt.threshold` function from the PyWavelets library is used programmatically to set wavelet coefficients smaller than the hard threshold to zero. It should be noted that the `pywt.threshold` function is an important function in the PyWavelets library used for thresholding data (especially wavelet coefficients). It weakens or zeroes out the portion of the signal smaller than the threshold by setting a threshold, thereby achieving the purpose of noise reduction, compression, or feature extraction.

[0078] The low-frequency approximation subband and the thresholded high-frequency detail subband are fused to obtain a sparsely represented multispectral image set. Specifically, the low-frequency approximation subband and the thresholded high-frequency detail subband are recombinated, and then the image is reconstructed using the pywt.waverec2 function through inverse wavelet transform to obtain a sparsely represented multispectral image set. It should be noted that the pywt.waverec2 function is used for inverse wavelet transform of two-dimensional signals (such as images). It reconstructs the original image by merging the wavelet coefficients of the low-frequency approximation subband and the high-frequency detail subband layer by layer.

[0079] S4. Perform compressed sensing on the sparsely represented multispectral imaging image group based on compressed sensing technology to obtain the compressed sensing multispectral imaging image group, including the following steps.

[0080] Random sampling is performed on the sparsely represented multispectral image group using a Gaussian random matrix. Specifically, the `np.random.randn` function from the NumPy library generates a Gaussian random matrix, and the `random_gaussian_sampling` function calls the Gaussian random matrix to perform Gaussian random sampling on each band of the multispectral image group, obtaining each randomly sampled multispectral image. It should be noted that the `np.random.randn` function uses the Box-Muller transform and the Ziggurat algorithm to generate random numbers from a standard normal distribution N(0,1). The `random_gaussian_sampling` function is a user-defined function that accepts the input image bands and a Gaussian random matrix, and uses the Gaussian random matrix to sample the input image bands.

[0081] The Orthogonal Matching Pursuit (OMP) algorithm is used to reconstruct each multispectral image in a randomly sampled multispectral image group. Specifically, the Orthogonal Matching Pursuit method from the scikit-learn library is used to implement OMP and reconstruction. Furthermore, the `omp_reconstruction` function is used to implement the OMP algorithm based on the sampling matrix and the sampled signal for reconstruction. It should be noted that the `omp_reconstruction` function is a greedy algorithm that selects a small number of atoms from the dictionary to approximate the input signal, iteratively selecting the dictionary atoms that best explain the remaining error.

[0082] Total variational regularization is used to process the reconstructed multispectral imaging image to obtain a compressed sensing multispectral imaging image set. Specifically, total variational denoising is implemented using the `denoise_tv_chambolle` method (Chambeau's total variational algorithm for denoising) from the scikit-image library. Furthermore, the `tv_denoise_multispectral_images` function is used to obtain the compressed sensing multispectral imaging image set. It should be noted that the core idea of ​​the `tv_denoise_multispectral_images` function is to apply Chambeau's total variational algorithm for denoising to each band of the multispectral imaging image set, while considering the correlation between different bands to optimize the denoising effect.

[0083] S5. Label the image group after compressed sensing multispectral imaging, and transform the RGB input visual Transformer model into a multi-band input visual Transformer model. Train the multi-channel input visual Transformer model, including the following steps.

[0084] Based on the CVAT annotation platform, a large number of compressed sensing multispectral imaging image sets were annotated to obtain a high-density packaged chip multispectral imaging image dataset. Specifically, a large number of compressed sensing multispectral imaging image sets were uploaded to the CVAT platform's API interface through programming. The platform then annotated the uploaded large number of compressed sensing multispectral imaging image sets based on chip industry experience.

[0085] The input layer of the RGB-input visual Transformer model is replaced with a multi-channel input layer. Specifically, the input dimension of the RGB-input visual Transformer model's input layer is 3, supporting the visible light band. Based on the requirements of visible light, near-infrared, short-wave infrared, ultraviolet, and thermal infrared images, the input dimension is determined to be 7. Then, by setting the `in_channels` parameter of the PyTorch library to 7, the input layer of the RGB-input visual Transformer model is replaced with a multi-channel input layer.

[0086] This approach captures complementary information between bands by using learnable band weights and optimizes the visual Transformer model using a weighted loss function. Specifically, each channel in the visual Transformer model is modified to assign weights based on the feature importance of the bands. When calculating the loss, the band weights are used to weight the loss, causing the model to focus more on bands with higher weights during optimization. Finally, the model's weights are updated using backpropagation and gradient descent algorithms.

[0087] Based on an image dataset of multispectral imaging from a high-density packaged chip, an optimized visual Transformer model was pre-trained. Specifically, the visual Transformer model was trained to perform classification tasks using the category labels of the multispectral images, learning to identify different types of defects from multi-band data.

[0088] Based on the pre-trained visual Transformer model, fine-tuning training is performed on an image dataset of multispectral imaging of a high-density packaged chip to obtain a trained visual Transformer model with multi-channel input. Specifically, the loss function, optimizer, and learning rate scheduler are set, and a training loop is written to fine-tune the visual Transformer model.

[0089] It should be noted that the input layers of visual Transformer models with RGB input only support 3-channel RGB images (i.e., in_channels = 3) because they were originally designed for processing visible light images. When processing multispectral imaging (e.g., 7 bands), the number of input channels needs to be increased to 7. This is achieved by adjusting the in_channels parameter in the PyTorch library, changing the number of input channels from 3 to 7. It should be further noted that changing in_channels involves more than just modifying a single parameter; it also requires ensuring that the model's Patch Embedding layer (responsible for dividing the image into patches and embedding them into a high-dimensional space) can handle multi-channel input.

[0090] S6. Using a trained multi-channel input visual Transformer model, locate and identify defect regions in the multispectral imaging image set after compressed sensing to obtain defect results for high-density packaged chips. This includes the following steps.

[0091] The compressed sensing multispectral image set is input into a trained multi-channel input visual Transformer model to capture global features of the image, locate and segment defect regions, and generate a defect segmentation map of the multispectral image. It should be noted that although the defect segmentation map of the multispectral image already provides information about the defect region, introducing edge detection is highly beneficial for obtaining more accurate defect boundaries, removing noise, improving discontinuous boundaries, and providing higher accuracy for subsequent geometric analysis.

[0092] The Canny edge detection algorithm is used to accurately locate defects in multispectral images to obtain defect results for high-density packaged chips. Specifically, edge detection is performed by adjusting the threshold1 and threshold2 parameters (low and high thresholds) based on the dynamic range of the defect segmentation map in the multispectral image. Then, the cv2.findContours function is used to extract the contours of the defects from the Canny edge detection results, obtaining the defect results for the high-density packaged chip, which are then saved as a CSV file. It should be noted that the Canny edge detection result is a binary image containing the edges of the defect region. By combining the segmentation map and the edge detection results, more refined and accurate defect regions can be obtained in segmentation tasks, especially in complex scenarios such as multispectral imaging.

[0093] S7. Based on the defect results of the high-density packaged chip, analyze the defect information and generate a report. The specific steps are as follows.

[0094] This study utilizes pandas to retrieve defect results from high-density packaged chips, performs statistical analysis on all detected defects, and analyzes defect trends during chip manufacturing. Specifically, the `value_counts` method (a value frequency counting method) of the pandas library is used to count the number of each defect type. The `describe` method (a data summarization method) is used to count the defect area. The defect detection dates are summarized by day to analyze the trend of defect changes over time.

[0095] This section describes how to generate defect type distribution and defect trend plots using the seaborn library. Specifically, the `countplot` function from the seaborn library is used programmatically to generate a bar chart for defect type distribution. The `lineplot` function from the seaborn library is used to generate a line chart for defect trend. It should be noted that the `countplot` function works by counting categorical variables and then using a bar chart to display the frequency distribution of each category. The `lineplot` function works by plotting a line chart of continuous data points, and optionally uses error intervals to display the uncertainty of the data. Both the `countplot` and `lineplot` functions rely on the Matplotlib library for graphing.

[0096] A PDF report is generated using a defect type distribution map and a defect trend map. Specifically, the PDF file is created programmatically using the `reportlab.pdfgen.canvas` method (PDF generation canvas), the defect type distribution map and defect trend map are embedded in the PDF, and analytical text is added to obtain the PDF report.

[0097] It should be noted that this process is particularly suitable for quality control and defect analysis in the production of high-density packaged chips. It can help the production team quickly identify the main defect types, analyze the trend of defect occurrence, and locate potential problems in the production process.

[0098] This embodiment also provides a computer device applicable to a defect detection method for high-density packaged chips, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the defect detection method for high-density packaged chips as proposed in the above embodiment.

[0099] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0100] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the defect detection method for high-density packaged chips as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0101] In summary, this invention improves the accuracy of defect detection in high-density packaged chips by employing multispectral imaging, sparse representation, compressed sensing, and deep learning technologies. Multispectral imaging acquires image information from different spectral bands, providing more comprehensive defect features and overcoming the limitations of traditional single-band imaging, thus improving detection accuracy. Sparse representation based on Haar wavelet discrete wavelet transform, combined with compressed sensing technology, reduces data volume, accelerates processing speed, and maintains high image quality, solving the problem of low processing efficiency in traditional methods. The RGB-input visual Transformer model is transformed into a multi-band input model, and learnable band weights improve the model's performance and robustness, further enhancing detection accuracy.

[0102] Example 2, referring to Table 1, is the second embodiment of the present invention. To further verify the technical solution of the present invention, experimental simulation data for a defect detection method for high-density packaged chips are given.

[0103] All chip samples underwent pretreatment. The chip surface was purged with nitrogen gas to ensure it was free of dust and particles. Next, the chips were suspended and fixed using an acoustic levitation device, and low-temperature plasma was applied to clean organic contaminants. Subsequently, the chip surface was rinsed with deionized water to remove residual contaminants. After rinsing, the chip surface was dried using a femtosecond laser drying device to ensure no moisture residue remained. Finally, the chip surface was planarized using a micropolish machine, and optical optimization was performed using a physical vapor deposition coating. Static electricity was then removed using an ion gun.

[0104] After preprocessing, the chip is fixed on the stage of the multispectral imaging device. The device automatically switches between different light sources and filters to acquire preliminary multispectral images in the visible, near-infrared, short-wave infrared, ultraviolet, and thermal infrared bands. Non-local mean filtering is applied to the image data for noise reduction, and histogram equalization is used to enhance contrast. Subsequently, an image registration algorithm based on feature point matching is used to align the images in each band, resulting in a complete set of multispectral images.

[0105] The image is decomposed into multiple levels using the discrete wavelet transform technique based on Haar wavelets, extracting low-frequency approximations and high-frequency detail subbands. The wavelet coefficients of the high-frequency subbands are then processed using a hard thresholding method to obtain a sparsely represented multispectral image set. Next, compressed sensing technology is used to compress and reconstruct the sparsely represented image set. Finally, total variational regularization is applied for denoising to generate the compressed sensing multispectral image set.

[0106] In the data processing stage, the compressed sensing image set was labeled using the CVAT annotation platform to obtain a defect dataset of high-density packaged chips. The input layer of the visual Transformer model was modified to a multi-band input, and the model's weight allocation and loss function were optimized. The labeled defect dataset was used to pre-train and fine-tune the visual Transformer model, resulting in a trained model. This model is used to locate and identify defect regions in multispectral images, and combined with the Canny edge detection algorithm, the defects are further precisely located to generate the final defect results.

[0107] The details are shown in Table 1 below:

[0108] Table 1 Defect Detection Analysis Table

[0109]

[0110]

[0111] From the perspective of surface flatness, the flatness of all chips remained between 0.07 and 0.12 μm, indicating that the micro-polishing and flattening processes in the pretreatment step effectively improved the surface quality of the chips. This high flatness provides a good foundation for subsequent multispectral imaging, ensuring the stability of imaging quality. The improvement in image resolution plays a crucial role in the accuracy of defect detection. Higher resolution samples (e.g., 2048x2048 pixels) show higher detection accuracy, identifying more small defects (e.g., chips C003 and C005). Furthermore, the detection time is also proportional to the image resolution. Although increasing resolution prolongs the detection time, this time consumption is acceptable within a reasonable range. Regarding false positive and false negative rates, the method of this invention outperforms existing technologies. Experimental data shows that the false positive rate remains between 2.2% and 3.0%, while the false negative rate remains between 1.7% and 2.2%, both lower than the 5% false positive rate and 3% false negative rate of traditional methods. This demonstrates that by combining compressed sensing technology and a visual Transformer model, the present invention can effectively reduce false detections and false negatives. Regarding the final defect identification success rate, the detection method of the present invention achieves an average success rate of 96.2%-98.1%, which is superior to the approximately 90% success rate of traditional single-spectral imaging-based detection methods. This indicates that the detection process combining multispectral imaging with a visual Transformer model can more comprehensively capture chip defect information, enhancing the robustness and accuracy of detection.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A defect detection method for high-density packaged chips, characterized in that: include, Preprocessing of high-density packaged chips; Using a multispectral imaging device, the pre-processed high-density packaged chip is imaged to obtain a multispectral imaging image set; The discrete wavelet transform technique based on Haar wavelets is used to first perform multi-level decomposition, and then thresholding is performed to obtain a sparse representation of the multispectral imaging image group. Compressed sensing technology is used to perform compressed sensing on a group of sparsely represented multispectral images to obtain the compressed-sensing multispectral image group. The specific steps are as follows. Random sampling is performed on the sparsely represented multispectral imaging image group using a Gaussian random matrix; The orthogonal matching pursuit algorithm is used to reconstruct each multispectral image in a group of randomly sampled multispectral images; By using total variational regularization, the reconstructed multispectral imaging image is processed to obtain a multispectral imaging image set after compressed sensing. The compressed sensing multispectral imaging image set is labeled, and the RGB input visual Transformer model is transformed into a multi-band input visual Transformer model. The multi-channel input visual Transformer model is then trained. The specific steps are as follows. A large number of compressed sensing multispectral image sets are labeled to obtain an image dataset for multispectral imaging of high-density packaged chips. Replace the input layer of the RGB input visual Transformer model with a multi-channel input input layer; Learnable band weights are used to capture complementary information between bands, and a weighted loss function is combined to optimize the visual Transformer model. Based on the image dataset of high-density packaged chip multispectral imaging, the optimized visual Transformer model is pre-trained. Based on the pre-trained visual Transformer model, fine-tuning training is performed on the image dataset of high-density packaged chip multispectral imaging to obtain a trained visual Transformer model with multi-channel input. By using a trained multi-channel input visual Transformer model, the defect region of the image group after compressed sensing and multispectral imaging is located and identified to obtain the defect results of high-density packaged chip. Based on the defect results of high-density packaged chips, the defect information is analyzed and a report is generated.

2. The defect detection method for high-density packaged chips as described in claim 1, characterized in that: The pretreatment includes surface cleaning, surface drying, surface smoothing, optical optimization, and static electricity removal, with the specific steps as follows. The surface of the high-density packaged chip was purged with pure nitrogen gas. The high-density packaged chip is suspended and fixed by an acoustic levitation device, and low-temperature plasma is applied to the surface of the high-density packaged chip to clean organic pollutants. Use deionized water to rinse the high-density packaged chips after they have been cleaned of organic contaminants. A femtosecond laser drying device was used to perform a full-coverage scan of the surface of the high-density packaged chip after rinsing; A high-precision micropolishing machine is used to planarize the surface of the high-density packaged chip after full-coverage scanning. Optical optimization was performed on the high-density packaged chip after surface planarization using a physical vapor deposition coating device. An ion gun is used to blow air through the optically optimized high-density packaged chip to remove static electricity, resulting in a pre-treated high-density packaged chip.

3. The defect detection method for high-density packaged chips as described in claim 2, characterized in that: A multispectral imaging device was used to image the preprocessed high-density packaged chip, obtaining a group of multispectral images. The specific steps are as follows. The pre-treated high-density packaged chip is fixed onto the stage of the multispectral imaging device using vacuum adsorption and mechanical clamps. Multispectral imaging equipment automatically switches the light source and corresponding filter according to imaging requirements; Using a multispectral imaging device, high-density packaged chips are imaged band by band for different combinations of light sources and filters, resulting in preliminary multispectral imaging images for each band of the visible light band, near-infrared light band, short-wave infrared light band, ultraviolet light band, and thermal infrared light band. Image denoising is performed on the preliminary multispectral imaging images of each band based on nonlocal mean filtering; The initial multispectral images of each band after image denoising are enhanced for contrast using histogram equalization; The images of each band after contrast enhancement are aligned using an existing image registration algorithm based on feature point matching to obtain a group of multispectral images.

4. The defect detection method for high-density packaged chips as described in claim 3, characterized in that: A multi-level decomposition process based on Haar wavelet discrete wavelet transform is first performed, followed by thresholding, to obtain a sparse representation of the multispectral imaging image set. The specific steps are as follows. The discrete wavelet transform based on Haar wavelets is used to perform multi-level decomposition on the multispectral imaging image group to obtain low-frequency approximate sub-bands and high-frequency detail sub-bands; Thresholds are set based on the standard deviation of each band of the image, and wavelet coefficients smaller than the threshold in the high-frequency detail subband are set to zero for thresholding. By fusing the low-frequency approximate subband and the thresholded high-frequency detail subband, a multispectral imaging image set with sparse representation is obtained.

5. The defect detection method for high-density packaged chips as described in claim 1, characterized in that: Using a trained multi-channel input visual Transformer model, defect regions in multispectral imaging images after compressed sensing are located and identified to obtain defect results for high-density packaged chips. The specific steps are as follows. The compressed sensing multispectral image group is input into the trained multi-channel input visual Transformer model to capture the global features of the image, locate and segment the defect region, and generate a defect segmentation map of the multispectral image. The Canny edge detection algorithm is used to accurately locate defects in multispectral images to obtain defect results for high-density packaged chips.

6. The defect detection method for high-density packaged chips as described in claim 5, characterized in that: Based on the defect results of high-density packaged chips, the defect information is analyzed and a report is generated. The specific steps are as follows. Using pandas to read the defect results of high-density packaged chips, statistical analysis was performed on all detected defects, and the defect trends in the chip manufacturing process were analyzed. Use the seaborn library to generate defect type distribution maps and defect trend maps; Generate a PDF report using defect type distribution maps and defect trend maps.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the defect detection method for high-density packaged chips as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the defect detection method for high-density packaged chips as described in any one of claims 1 to 6.

Images