Defect detection method for high-density packaged chip

Abstract

The invention discloses a defect detection method for a high-density packaged chip, which relates to the technical field of semiconductor detection, and comprises the following steps: preprocessing the high-density packaged chip; imaging the preprocessed high-density packaged chip by using a multispectral imaging device to obtain a multispectral imaging image group; the method comprises the following steps: firstly, carrying out multi-level decomposition by adopting a discrete wavelet transform technology based on Haar wavelets, and then carrying out thresholding processing to obtain a multi-spectral imaging image group with sparse representation; carrying out compressed sensing on the sparse-represented multispectral imaging image group based on a compressed sensing technology to obtain a multispectral imaging image group after compressed sensing; the method comprises the following steps of: marking an image group of multispectral imaging after compressed sensing, transforming a visual Transform model input by RGB (Red, Green, Blue) into a visual Transform model input by a plurality of wavebands, and training the visual Transform model input by a plurality of channels; according to the method, the accuracy of defect detection of the high-density packaged chip is improved through multispectral imaging, sparse representation and compressed sensing technologies.

Description

Technical Field

[0001] The present invention relates to the field of semiconductor detection technology, and in particular to a defect detection method for high-density packaged chips. Background Art

[0002] In semiconductor manufacturing, the production of densely packaged chips presents numerous challenges, particularly in defect detection. Traditional methods using manual visual inspection and single-electromechanical or optical techniques have significant limitations, such as reliance on operator experience, unstable results, insufficient resolution, and slow inspection speeds. As chip density increases, these methods are no longer able to meet the demands for high-precision and efficient inspection.

[0003] Furthermore, traditional compressed sensing methods have shortcomings in image reconstruction and denoising, which can lead to degraded image quality and further impact defect detection accuracy. While deep learning methods have demonstrated excellent performance in image classification and object detection, traditional models are primarily based on RGB images and cannot fully utilize the multi-band information of multispectral imaging. These limitations severely restrict defect detection effectiveness and production efficiency for high-density packaged chips. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a defect detection method for high-density packaged chips to solve the problem of insufficient accuracy of traditional defect detection methods in high-density packaged chips detection.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a defect detection method for high-density packaged chips, comprising pre-processing the high-density packaged chips;

[0008] Using multispectral imaging equipment, the pre-processed high-density packaged chips are imaged to obtain a multispectral imaging image group;

[0009] The discrete wavelet transform technology based on Haar wavelet is used to perform multi-level decomposition first, and then threshold processing is performed to obtain a sparse representation of multispectral imaging image group;

[0010] Based on the compressed sensing technology, compressed sensing is performed on the sparsely represented multispectral imaging image group to obtain the compressed sensing multispectral imaging image group;

[0011] Label the multispectral image group after compressed sensing, transform the RGB input visual Transformer model into a multi-band input visual Transformer model, and train the multi-channel input visual Transformer model;

[0012] Using a trained multi-channel input visual Transformer model, we can locate and identify defective areas in compressed sensing multispectral imaging images and obtain defect results for high-density packaged chips.

[0013] Based on the defect results of high-density packaged chips, the defect information is analyzed and a report is generated.

[0014] As a preferred embodiment of the defect detection method for high-density packaged chips of the present invention, the pretreatment includes surface cleaning, surface drying, surface flattening, optical optimization and static removal. The specific steps are as follows:

[0015] Use pure nitrogen to purge the surface of high-density packaged chips;

[0016] The high-density packaged chips after purging are suspended and fixed using acoustic levitation equipment, and low-temperature plasma is applied to the surface of the high-density packaged chips to clean organic pollutants;

[0017] Use deionized water to rinse high-density packaged chips after cleaning of organic pollutants;

[0018] Use femtosecond laser drying equipment to perform full coverage scanning on the surface of the rinsed high-density packaged chips;

[0019] Use a high-precision micro-polishing machine to flatten the surface of high-density packaged chips after full coverage scanning;

[0020] Use physical vapor deposition coating equipment to perform optical optimization on high-density packaged chips after surface flattening;

[0021] The optically optimized high-density packaged chips were blown with an ion air gun to remove static electricity, thereby obtaining pre-treated high-density packaged chips.

[0022] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, a multispectral imaging device is used to image the pre-processed high-density packaged chips to obtain a multispectral imaging image group. The specific steps are as follows:

[0023] The pre-processed high-density packaged chip is fixed on the stage of the multispectral imaging device using vacuum adsorption and mechanical clamps;

[0024] Multispectral imaging equipment automatically switches the light source and corresponding filters according to imaging requirements;

[0025] Using multispectral imaging equipment, for different light source and filter combinations, high-density packaged chips are imaged band by band, obtaining preliminary multispectral imaging images for each band of visible light, near-infrared light, short-wave infrared light, ultraviolet light, and thermal infrared light.

[0026] Image denoising is performed on the preliminary multispectral imaging images of each band based on non-local mean filtering;

[0027] After image denoising, the preliminary multispectral imaging images of each band are contrast enhanced using histogram equalization;

[0028] The contrast-enhanced preliminary multispectral imaging images of each band are aligned using an existing image registration algorithm based on feature point matching to obtain a multispectral imaging image group.

[0029] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, a discrete wavelet transform technology based on Haar wavelet is first used to perform multi-level decomposition, and then threshold processing is performed to obtain a sparse representation multispectral imaging image group. The specific steps are as follows:

[0030] The multi-spectral imaging image group is decomposed into multiple levels using discrete wavelet transform based on Haar wavelet to obtain low-frequency approximate sub-bands and high-frequency detail sub-bands.

[0031] The threshold is set based on the standard deviation of each band of the image, and the wavelet coefficients in the high-frequency detail subband that are smaller than the threshold are set to zero for thresholding processing;

[0032] The low-frequency approximate subband and the thresholded high-frequency detail subband are fused to obtain a multispectral imaging image group after sparse representation.

[0033] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, wherein: based on the compressed sensing technology, the sparsely represented multispectral imaging image group is compressed sensing to obtain the compressed multispectral imaging image group, the specific steps are as follows:

[0034] The image group of multispectral imaging after sparse representation is randomly sampled using Gaussian random matrix;

[0035] Reconstruct each multispectral imaging image in the randomly sampled multispectral imaging image group based on the orthogonal matching pursuit algorithm;

[0036] The reconstructed multispectral imaging image is processed by total variation regularization to obtain a multispectral imaging image group after compressed sensing.

[0037] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, the image group of multispectral imaging after compressed sensing is annotated, and the visual Transformer model with RGB input is transformed into a visual Transformer model with multi-band input, and the visual Transformer model with multi-channel input is trained. The specific steps are as follows:

[0038] A large number of compressed sensing multispectral image groups are annotated to obtain an image dataset of multispectral imaging of high-density packaged chips;

[0039] Replace the input layer of the RGB input visual Transformer model with a multi-channel input input layer;

[0040] Capturing complementary information between bands through learnable band weights and optimizing the visual Transformer model in combination with a weighted loss function;

[0041] Pre-training the optimized visual Transformer model based on an image dataset of multispectral imaging of high-density packaged chips;

[0042] Based on the pre-trained visual Transformer model, fine-tuning training is performed on the image dataset of multispectral imaging of high-density packaged chips to obtain a trained multi-channel input visual Transformer model.

[0043] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, the following steps are used to locate and identify defect areas in a multispectral imaging image group after compressed sensing, and obtain defect results for high-density packaged chips.

[0044] The multispectral image group after compressed sensing is input into the trained multi-channel input visual Transformer model to capture the global features of the image, locate and segment the defect area, and generate a defect segmentation map of the multispectral image;

[0045] The Canny edge detection algorithm is used to accurately locate the defect segmentation map of the multispectral image and obtain the defect results of high-density packaged chips.

[0046] As a preferred solution of the defect detection method for high-density packaged chips described in the present invention, pandas is used to read the defect results of high-density packaged chips, perform statistical analysis on all detected defects, and analyze the defect trends existing in the chip production process;

[0047] Use the seaborn library to generate defect type distribution maps and defect trend maps;

[0048] Generate defect type distribution graphs and defect trend graphs, and generate PDF reports.

[0049] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the defect detection method for high-density packaged chips as described in the first aspect of the present invention is implemented.

[0050] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the defect detection method for high-density packaged chips as described in the first aspect of the present invention is implemented.

[0051] The beneficial effects of the present invention are: through multispectral imaging, sparse representation, compressed sensing and deep learning technology, the accuracy of defect detection of high-density packaged chips is improved. Multispectral imaging equipment obtains image information of different bands, provides more comprehensive defect characteristics, overcomes the limitations of traditional single-band imaging, and improves the accuracy of detection. The discrete wavelet transform based on Haar wavelet is used for sparse representation, combined with compressed sensing technology to reduce the amount of data, speed up the processing speed, and maintain the high quality of the image, solving the problem of low processing efficiency of traditional methods. The visual Transformer model with RGB input is transformed into a model with multi-band input. The performance and robustness of the model are improved through learnable band weights, further improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flow chart of the defect detection method for high-density packaged chips in Example 1.

[0054] Figure 2 This is a flowchart of the visual Transformer model for obtaining multi-channel input in Example 1. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0058] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides a defect detection method for high-density packaged chips, comprising the following steps:

[0059] S1. Pre-processing the high-density packaged chips, including the following steps:

[0060] Use pure nitrogen to purge the surface of high-density packaged chips. Specifically, use pure nitrogen to purge the surface of high-density packaged chips at a nitrogen flow rate of 10-20L / min and a purge time of 30-60 seconds to ensure that the surface is dust-free and free of static electricity.

[0061] After purging, high-density packaged chips are suspended and fixed using an acoustic levitation device. Low-temperature plasma is then applied to the surface of the high-density packaged chips to clean organic contaminants. Specifically, the low-temperature plasma treatment power should be set between 50–200W, the treatment time should be 10–30 seconds, and the plasma gas can be argon or oxygen, with a gas flow rate of 50–100 sccm. The chips must remain suspended during the cleaning process to avoid contact contamination.

[0062] Use deionized water to rinse high-density packaged chips after cleaning of organic pollutants.

[0063] The surface of the rinsed high-density packaged chip is fully scanned using a femtosecond laser drying device. Specifically, the laser wavelength of the femtosecond laser drying device is 800–1050 nm, the pulse width is 100–500 fs, the scanning speed is 1–5 mm / s, and the power density is 0.5–2 W / cm 2 The chip surface should be covered during the drying process to ensure no moisture remains.

[0064] After full-coverage scanning, high-density packaged chips are flattened using a high-precision micro-polishing machine. Specifically, the polishing material of the high-precision micro-polishing machine should be nano-grade polishing paste, with a polishing pressure of 0.1–0.5 MPa and a polishing time of 30–90 seconds to ensure that the chip surface flatness reaches the nanometer level.

[0065] The optical optimization of high-density packaged chips after surface planarization is performed using physical vapor deposition coating equipment.

[0066] The optically optimized high-density packaged chips were blown with an ion air gun to remove static electricity, thereby obtaining pre-treated high-density packaged chips.

[0067] It should be noted that low-temperature plasma is a partially ionized gas containing a large number of active species (such as free radicals, ions, and excited molecules). These active species can chemically react with organic contaminants on the chip surface, breaking them down into volatile substances and thus removing the contaminants. Physical vapor deposition (PVD) is a technology that deposits materials onto the chip surface through physical processes (such as evaporation and sputtering) in a vacuum environment. By selecting the right coating material, the optical properties of the chip surface can be improved, thereby enhancing imaging quality.

[0068] S2, using a multispectral imaging device to image the pre-processed high-density packaged chip to obtain a multispectral imaging image group, including the following steps:

[0069] Use vacuum suction and mechanical clamps to secure the pre-treated, high-density packaged chip to the stage of the multispectral imaging device. Specifically, ensure the stage of the multispectral imaging device is clean and dust-free. Place the pre-treated, high-density packaged chip on the stage. Using a vacuum suction device, turn on the vacuum pump to hold the chip firmly against the stage. Using a mechanical clamp, gently secure the edge of the chip to ensure it does not move during imaging.

[0070] Multispectral imaging equipment automatically switches the light source and corresponding filters according to imaging requirements. It should be noted that the wavelength range of the visible light band is 400–700 nm, generated by white light LEDs and filtered by broadband filters; the wavelength range of the near-infrared band is 700–1100 nm, generated by near-infrared LEDs or halogen lamps and filtered by near-infrared filters; the wavelength range of the short-wave infrared band is 1100–2500 nm, generated by short-wave infrared LEDs or halogen lamps and filtered by short-wave infrared filters; the wavelength range of the ultraviolet band is 200–400 nm, generated by ultraviolet LEDs or mercury lamps and filtered by ultraviolet filters; the wavelength range of the thermal infrared band is 3000–5000 nm, generated by heat sources (such as blackbody radiation sources) and filtered by thermal infrared filters.

[0071] Using multispectral imaging equipment, we image the high-density packaged chips band by band using different light source and filter combinations, obtaining preliminary multispectral images for each band: visible light, near-infrared light, short-wave infrared light, ultraviolet light, and thermal infrared light. It should be noted that the exposure time and gain are appropriately set to ensure high-quality multispectral images in different bands.

[0072] Image denoising is performed on the preliminary multispectral images of each band using non-local mean filtering. Specifically, the preliminary multispectral images of each band are converted to grayscale images and denoised using the cv2.fastNlMeansDenoising function in the OpenCV library. It should be noted that the cv2.fastNlMeansDenoising function calculates the similarity of each pixel in the image and uses the weighted average of similar pixels to remove noise.

[0073] After image denoising, the preliminary multispectral images for each band are contrast-enhanced using histogram equalization. Specifically, the cv2.equalizeHist function in the OpenCV library is used programmatically to perform histogram equalization on each band. It should be noted that the cv2.equalizeHist function adjusts the grayscale distribution of image pixels to achieve a more uniform distribution of grayscale values, thereby enhancing contrast. This is particularly effective when processing images with uneven brightness or low contrast.

[0074] After contrast enhancement, the preliminary multispectral images of each band are aligned using an existing image registration algorithm based on feature point matching to obtain a multispectral image set. Specifically, the ORB feature point detection algorithm (Oriented Fast Features and Rotated Binary Descriptors) is used to identify significant feature points in the preliminary multispectral images of each band. Matching is then performed using a combination of the BFMatcher (brute force matcher) and the K-nearest neighbor algorithm, with matching results screened using the Lowe's ratio test. A random sampling consistency algorithm helps estimate the global transformation matrix of the image. Using this estimated transformation matrix, the images of the remaining bands are aligned with the reference image, ensuring that each band in the multispectral image is in the same coordinate system. It should be noted that the brute force matcher is a simple and direct feature point matching method. It compares every possible pairing of one feature descriptor with another to find the best match. The K-nearest neighbor algorithm is used to improve the efficiency of the brute force matcher.

[0075] S3, using the discrete wavelet transform technology based on Haar wavelet to first perform multi-level decomposition, and then perform threshold processing to obtain a sparse representation of the multispectral imaging image group, including the following steps,

[0076] The Haar wavelet-based discrete wavelet transform is used to perform multi-level decomposition of multispectral imaging images to obtain low-frequency approximate subbands and high-frequency detail subbands. Specifically, the pywt.wavedec2 function in the PyWavelets library is used programmatically to process the multispectral imaging image group to achieve multi-level decomposition of the multispectral imaging image group using the Haar wavelet-based discrete wavelet transform.

[0077] Thresholds are set based on the standard deviation of each band of the image, and wavelet coefficients in the high-frequency detail subband that are less than the threshold are set to zero for thresholding. Specifically, the standard deviation of the image pixels is first calculated, and the hard threshold is determined based on the standard deviation. Then, the pywt.threshold function in the PyWavelets library is used programmatically to set the wavelet coefficients that are less than the hard threshold to zero. It should be noted that the pywt.threshold function is an important function in the PyWavelets library for thresholding data (especially wavelet coefficients). It sets a threshold to weaken or set to zero the part of the signal that is less than the threshold, thereby achieving the purpose of denoising, compression or feature extraction.

[0078] The low-frequency approximate subband and the thresholded high-frequency detail subband are fused to obtain a multispectral imaging image group after sparse representation. Specifically, the low-frequency approximate subband and the thresholded high-frequency detail subband are recombined, and then the image is reconstructed by inverse wavelet transform using the pywt.waverec2 function to obtain a multispectral imaging image group after sparse representation. It should be noted that the pywt.waverec2 function is an inverse wavelet transform for two-dimensional signals (such as images). It reconstructs the original image by merging the wavelet coefficients of the low-frequency approximate subband and the wavelet coefficients of the high-frequency detail subband layer by layer.

[0079] S4, performing compressed sensing on the sparsely represented multispectral imaging image group based on compressed sensing technology to obtain the compressed multispectral imaging image group, including the following steps:

[0080] A Gaussian random matrix is used to randomly sample the sparsely represented multispectral imaging image group. Specifically, the NumPy library's np.random.randn function is used to generate a Gaussian random matrix, and the random_gaussian_sampling function is used to call the Gaussian random matrix to perform Gaussian random sampling on each band of the multispectral image group, obtaining each multispectral imaging image after random sampling. It should be noted that the np.random.randn function uses the Box-Muller transform and the Ziggurat algorithm to generate random numbers from the standard normal distribution N(0,1). The random_gaussian_sampling function is a custom function that accepts input image bands and a Gaussian random matrix and uses the Gaussian random matrix to sample the input image bands.

[0081] Each multispectral image in the randomly sampled multispectral image group is reconstructed based on the orthogonal matching pursuit algorithm. Specifically, the OrthogonalMatchingPursuit method (orthogonal matching pursuit algorithm) of the scikit-learn library is used to implement orthogonal matching pursuit and reconstruction. Furthermore, the orthogonal matching pursuit algorithm is implemented through the omp_reconstruction function to reconstruct the sampling matrix and the sampled signal. It should be noted that the principle of the omp_reconstruction function is a greedy algorithm that is used to select a small number of atoms from the dictionary to approximately reconstruct the input signal, and gradually select the dictionary atoms that can best explain the residual error in an iterative manner.

[0082] The reconstructed multispectral imaging images are processed using total variation regularization to obtain a set of multispectral imaging images after compressed sensing. Specifically, total variation denoising is implemented using the denoise_tv_chambolle method (Chambolle total variation denoising) in the scikit-image library. Furthermore, the tv_denoise_multispectral_images function is used to obtain a set of multispectral imaging images after compressed sensing. It should be noted that the core idea of the tv_denoise_multispectral_images function is to apply the Chambolle total variation denoising algorithm to each band of the multispectral imaging image set, while taking into account the correlation between different bands to optimize the denoising effect.

[0083] S5. Label the image group of multispectral imaging after compressed sensing, transform the RGB input visual Transformer model into a multi-band input visual Transformer model, and train the multi-channel input visual Transformer model, including the following steps:

[0084] Using the CVAT annotation platform, we annotated a large number of compressed sensing multispectral imaging images to obtain a dataset of multispectral imaging images of high-density packaged chips. Specifically, we programmatically uploaded these compressed sensing multispectral images to the CVAT platform's API. The platform annotated these uploaded images based on chip industry experience.

[0085] Replace the input layer of the RGB input visual Transformer model with a multi-channel input input layer. Specifically, the input dimension of the RGB input visual Transformer model input layer is 3, and it supports the visible light band. Based on the requirements of visible light band images, near-infrared light band images, short-wave infrared light band images, ultraviolet light band images, and thermal infrared light band images, the input dimension can be determined to be 7. Then, by adjusting the in_channels parameter of the PyTorch library to 7, the input layer of the RGB input visual Transformer model is replaced with a multi-channel input input layer.

[0086] Learnable band weights capture complementary information between bands and are combined with a weighted loss function to optimize the visual Transformer model. Specifically, each channel in the visual Transformer model is restructured to assign weights based on the importance of the band's features. When calculating the loss, the band weights are used to weight the loss, causing the model to prioritize bands with higher weights during optimization. Finally, the model weights are updated through backpropagation and gradient descent.

[0087] The optimized visual Transformer model was pre-trained on a dataset of multispectral imaging of high-density packaged chips. Specifically, the model was trained on the category labels of the multispectral images to perform classification tasks, learning to identify different types of defects from multi-band data.

[0088] Based on the pre-trained visual Transformer model, fine-tune it on a dataset of multispectral imaging from high-density packaged chips to obtain a trained visual Transformer model with multi-channel input. Specifically, set the loss function, optimizer, and learning rate scheduler, and write a training loop to fine-tune the visual Transformer model.

[0089] It should be noted that the input layer of the visual Transformer model for RGB input only supports 3-channel RGB images (i.e., in_channels = 3) because they were originally designed to process visible light images. When processing multispectral imaging (such as 7 bands), the number of input channels of the model needs to be increased to 7. To achieve this, the specific operation is to change the input channels from 3 to 7 by adjusting the in_channels parameter in the PyTorch library. It should be further explained that changing in_channels is not just about modifying a single parameter, but also about ensuring that the model's Patch Embedding layer (responsible for dividing the image into small patches and embedding them into a high-dimensional space) can handle multi-channel inputs.

[0090] S6. Using the trained multi-channel input visual Transformer model, locate and identify defect areas in the multispectral imaging image group after compressed sensing, and obtain defect results of high-density packaged chips, including the following steps:

[0091] The compressed sensing multispectral image set is input into a trained multi-channel input visual Transformer model to capture the global features of the image, locate and segment the defect area, and generate a defect segmentation map for the multispectral image. It should be noted that although the defect segmentation map of the multispectral image already provides regional information of the defect, it is very beneficial to introduce edge detection to obtain more accurate defect boundaries, remove noise, improve incoherent boundaries, and provide higher accuracy for subsequent geometric analysis.

[0092] The Canny edge detection algorithm is used to accurately locate the defect segmentation map of the multispectral image and obtain the defect results of the high-density packaged chip. Specifically, the threshold1 parameter and threshold2 parameter that control the low threshold and high threshold are adjusted according to the dynamic range of the defect segmentation map of the multispectral image to perform edge detection. Then, the cv2.findContours function is used to extract the defect contour from the results of the Canny edge detection, obtain the defect results of the high-density packaged chip and save it as a CSV file. It should be noted that the result of the Canny edge detection is a binary image containing the edges of the defect area. By combining the segmentation map and the edge detection results, more refined and accurate defect areas can be obtained in the segmentation task, especially in complex scenarios such as multispectral imaging.

[0093] S7. Analyze the defect information and generate a report based on the high-density packaged chip defect results. The specific steps are as follows:

[0094] Using Pandas to read defect data for high-density packaged chips, we statistically analyzed all detected defects and analyzed defect trends during chip production. Specifically, we used the value_counts method (value frequency counting method) in the Pandas library to count the number of each defect type. We also used the describe method (data summarization method) to calculate defect area. We also summarized defect detection dates by day to analyze defect trends over time.

[0095] Use the seaborn library to generate defect type distribution charts and defect trend charts. Specifically, programmatically use the countplot function of the seaborn library to generate a bar chart as a defect type distribution chart. Use the lineplot function of the seaborn library to generate a line chart as a defect trend chart. It should be noted that the implementation principle of the countplot function is to count categorical variables and then use a bar chart to display the frequency distribution of each category. The implementation principle of the lineplot function is to draw a line chart of continuous data points and optionally use error intervals to display the uncertainty of the data. Both the countplot function and the lineplot function rely on the Matplotlib library for graphics drawing.

[0096] Generate a PDF report by generating a defect type distribution graph and a defect trend graph. Specifically, programmatically create a PDF file using the reportlab.pdfgen.canvas method (PDF generation canvas), embed the defect type distribution graph and defect trend graph into the PDF, and add analysis text to obtain a PDF report.

[0097] It should be noted that this process is particularly suitable for quality control and defect analysis in the production of high-density packaged chips. It can help the production team quickly identify the main defect types, analyze the occurrence trends of defects, and locate potential problems in the production process.

[0098] This embodiment also provides a computer device suitable for the defect detection method of high-density packaged chips, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the defect detection method for high-density packaged chips proposed in the above embodiment.

[0099] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0100] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the defect detection method for high-density packaged chips proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0101] In summary, the present invention improves the accuracy of defect detection in high-density packaged chips through multispectral imaging, sparse representation, compressed sensing and deep learning technologies. Multispectral imaging equipment obtains image information in different bands, provides more comprehensive defect features, overcomes the limitations of traditional single-band imaging, and improves the accuracy of detection. The discrete wavelet transform based on Haar wavelet is used for sparse representation, combined with compressed sensing technology to reduce the amount of data, speed up the processing speed, and maintain the high quality of the image, solving the problem of low processing efficiency of traditional methods. The visual Transformer model with RGB input is transformed into a model with multi-band input. The performance and robustness of the model are improved through learnable band weights, further improving the detection accuracy.

[0102] Example 2, referring to Table 1, is the second embodiment of the present invention. In order to further verify the technical solution of the present invention, experimental simulation data of a defect detection method for high-density packaged chips are provided.

[0103] All chip samples were pretreated. A nitrogen purge device was used to purge the chip surface to ensure that the surface was free of dust and particles. Next, an acoustic levitation device was used to suspend and fix the chip, and low-temperature plasma was applied to clean organic contaminants. Subsequently, the chip surface was rinsed with deionized water to remove residual contaminants. After rinsing, the chip surface was scanned for dryness using a femtosecond laser drying device to ensure that no moisture remained. Finally, a micro-polisher was used to flatten the chip surface, and optical optimization was performed using a physical vapor deposition coating. Finally, an ion air gun was used to remove static electricity.

[0104] After preprocessing, the chip is mounted on the stage of a multispectral imaging device. The device automatically switches between different light sources and filters to acquire preliminary multispectral images in the visible, near-infrared, short-wave infrared, ultraviolet, and thermal infrared bands. Non-local means filtering is used to denoise the image data, and histogram equalization is used to enhance contrast. Subsequently, an image registration algorithm based on feature point matching is used to align the images in each band to obtain a complete multispectral image set.

[0105] The image is decomposed at multiple levels using the Haar wavelet-based discrete wavelet transform (DWT) technique, extracting low-frequency approximation and high-frequency detail subbands. The high-frequency subband wavelet coefficients are then processed using a hard thresholding method to obtain a sparsely represented multispectral image set. This sparsely represented image set is then compressed and reconstructed using compressed sensing techniques. Finally, total variation regularization (TVR) is used to remove noise, resulting in a compressed sensing multispectral image set.

[0106] During the data processing phase, the compressed sensing image set was annotated using the CVAT annotation platform to obtain a defect dataset for high-density packaged chips. The input layer of the visual Transformer model was restructured to accommodate multi-band input, and the model's weight distribution and loss function were optimized. The visual Transformer model was pre-trained and fine-tuned using the annotated defect dataset, ultimately obtaining a trained model. This model was used to locate and identify defect areas in multispectral images. The Canny edge detection algorithm was then used to further accurately locate defects and generate the final defect results.

[0107] The details are shown in Table 1 below:

[0108] Table 1 Defect detection analysis table

[0109]

[0110]

[0111] Looking at the surface flatness parameter, all chips maintained a flatness between 0.07 and 0.12 μm, indicating that the micro-polishing and planarization treatments in the pretreatment step effectively improved the chip surface quality. This high flatness provides an excellent foundation for subsequent multispectral imaging, ensuring the stability of imaging quality. Improved image resolution plays a key role in defect detection accuracy. Samples with higher resolution (such as 2048x2048 pixels) showed higher detection accuracy, identifying more small defects (such as chips C003 and C005). Furthermore, detection time is proportional to image resolution. Although increased resolution increases detection time, this time consumption is acceptable within a reasonable range. The method of the present invention outperforms existing technologies in terms of false positive and false negative rates. Experimental data shows that the false positive rate remains between 2.2% and 3.0%, while the false negative rate remains between 1.7% and 2.2%, both lower than the 5% and 3% false positive rates of traditional methods. This demonstrates that by combining compressed sensing technology with the visual Transformer model, the present invention can effectively reduce false positives and missed detections. In terms of final defect identification success rate, the present invention's detection method achieved an average success rate of 96.2%-98.1%, surpassing the approximately 90% success rate of traditional single-spectral imaging-based detection methods. This demonstrates that the multispectral imaging combined with the visual Transformer model detection process can more comprehensively capture chip defect information, enhancing the robustness and accuracy of detection.

[0112] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A defect detection method for high-density packaged chips, characterized by: include, Pre-process high-density packaged chips; Using multispectral imaging equipment, the pre-processed high-density packaged chips are imaged to obtain a multispectral imaging image group; The discrete wavelet transform technology based on Haar wavelet is used to perform multi-level decomposition first, and then threshold processing is performed to obtain a sparse representation of multispectral imaging image group; Based on the compressed sensing technology, compressed sensing is performed on the sparsely represented multispectral imaging image group to obtain the compressed sensing multispectral imaging image group; Label the multispectral image group after compressed sensing, transform the RGB input visual Transformer model into a multi-band input visual Transformer model, and train the multi-channel input visual Transformer model; Using a trained multi-channel input visual Transformer model, we can locate and identify defective areas in compressed sensing multispectral imaging images and obtain defect results for high-density packaged chips. Based on the defect results of high-density packaged chips, the defect information is analyzed and a report is generated.

2. The defect detection method for high-density packaged chips according to claim 1, wherein: The pretreatment includes surface cleaning, surface drying, surface flattening, optical optimization and static removal. The specific steps are as follows: Use pure nitrogen to purge the surface of high-density packaged chips; The high-density packaged chips after purging are suspended and fixed using acoustic levitation equipment, and low-temperature plasma is applied to the surface of the high-density packaged chips to clean organic pollutants; Use deionized water to rinse high-density packaged chips after cleaning of organic pollutants; Use femtosecond laser drying equipment to perform full coverage scanning on the surface of the rinsed high-density packaged chips; Use a high-precision micro-polishing machine to flatten the surface of high-density packaged chips after full coverage scanning; Use physical vapor deposition coating equipment to perform optical optimization on high-density packaged chips after surface flattening; The optically optimized high-density packaged chips were blown with an ion air gun to remove static electricity, thereby obtaining pre-treated high-density packaged chips.

3. The defect detection method for high-density packaged chips according to claim 2, wherein: Use multispectral imaging equipment to image the pre-processed high-density packaged chips to obtain a multispectral imaging image group. The specific steps are as follows: The pre-processed high-density packaged chip is fixed on the stage of the multispectral imaging device using vacuum adsorption and mechanical clamps; Multispectral imaging equipment automatically switches the light source and corresponding filters according to imaging requirements; Using multispectral imaging equipment, for different light source and filter combinations, high-density packaged chips are imaged band by band, obtaining preliminary multispectral imaging images for each band of visible light, near-infrared light, short-wave infrared light, ultraviolet light, and thermal infrared light. Image denoising is performed on the preliminary multispectral imaging images of each band based on non-local mean filtering; After image denoising, the preliminary multispectral imaging images of each band are contrast enhanced using histogram equalization; The contrast-enhanced preliminary multispectral imaging images of each band are aligned using an existing image registration algorithm based on feature point matching to obtain a multispectral imaging image group.

4. The defect detection method for high-density packaged chips according to claim 3, wherein: The discrete wavelet transform technology based on Haar wavelet is used to perform multi-level decomposition first, and then threshold processing is performed to obtain a sparse representation of the multispectral imaging image group. The specific steps are as follows: The multi-spectral imaging image group is decomposed into multiple levels using discrete wavelet transform based on Haar wavelet to obtain low-frequency approximate sub-bands and high-frequency detail sub-bands. The threshold is set based on the standard deviation of each band of the image, and the wavelet coefficients in the high-frequency detail subband that are smaller than the threshold are set to zero for thresholding processing; The low-frequency approximate subband and the thresholded high-frequency detail subband are fused to obtain a multispectral imaging image group after sparse representation.

5. The defect detection method for high-density packaged chips according to claim 4, wherein: Based on the compressed sensing technology, the sparsely represented multispectral imaging image group is compressed to obtain the compressed multispectral imaging image group. The specific steps are as follows: The image group of multispectral imaging after sparse representation is randomly sampled using Gaussian random matrix; Reconstruct each multispectral imaging image in the randomly sampled multispectral imaging image group based on the orthogonal matching pursuit algorithm; The reconstructed multispectral imaging image is processed by total variation regularization to obtain a multispectral imaging image group after compressed sensing.

6. The defect detection method for high-density packaged chips according to claim 5, wherein: The image group of multispectral imaging after compressed sensing is annotated, and the visual Transformer model with RGB input is transformed into a visual Transformer model with multi-band input. The visual Transformer model with multi-channel input is trained. The specific steps are as follows: A large number of compressed sensing multispectral image groups are annotated to obtain an image dataset of multispectral imaging of high-density packaged chips; Replace the input layer of the RGB input visual Transformer model with a multi-channel input input layer; Capturing complementary information between bands through learnable band weights and optimizing the visual Transformer model in combination with a weighted loss function; Pre-training the optimized visual Transformer model based on an image dataset of multispectral imaging of high-density packaged chips; Based on the pre-trained visual Transformer model, fine-tuning training is performed on the image dataset of multispectral imaging of high-density packaged chips to obtain a trained multi-channel input visual Transformer model.

7. The defect detection method for high-density packaged chips according to claim 6, wherein: Using the trained multi-channel input visual Transformer model, we can locate and identify defective areas in the multispectral imaging image group after compressed sensing and obtain defect results for high-density packaged chips. The specific steps are as follows: The multispectral image group after compressed sensing is input into the trained multi-channel input visual Transformer model to capture the global features of the image, locate and segment the defect area, and generate a defect segmentation map of the multispectral image; The Canny edge detection algorithm is used to accurately locate the defect segmentation map of the multispectral image and obtain the defect results of high-density packaged chips.

8. The defect detection method for high-density packaged chips according to claim 7, wherein: According to the defect results of high-density packaged chips, the defect information is analyzed and a report is generated. The specific steps are as follows: Use pandas to read high-density packaged chip defect results, perform statistical analysis on all detected defects, and analyze defect trends in the chip production process; Use the seaborn library to generate defect type distribution maps and defect trend maps; Generate defect type distribution graphs and defect trend graphs, and generate PDF reports.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the defect detection method for high-density packaged chips according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the defect detection method for high-density packaged chips according to any one of claims 1 to 8 are implemented.

Images