Method and system for rapidly detecting defects of chip packaging process
By generating pseudo-color images through hyperspectral scanning and principal component analysis, and combining them with HSV color space threshold segmentation, the problem of difficult identification of adhesive overflow defects in transparent adhesive materials is solved, and the precise positioning and efficient detection of adhesive overflow areas are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-10
- Publication Date
- 2026-04-10
AI Technical Summary
Existing visible light visual inspection methods are difficult to effectively identify defects in transparent adhesive materials, resulting in a high rate of missed detections and difficulty in accurate positioning during chip packaging processes.
Hyperspectral scanning technology was used to screen out characteristic band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate. Principal component analysis was used to generate pseudo-color images through dimensionality reduction mapping, and HSV color space threshold segmentation was combined to achieve precise positioning of the adhesive overflow area.
It significantly reduced the missed detection rate of chip packaging adhesive overflow defects, and achieved precise positioning of the physical area and spatial location of the adhesive overflow area, thus improving the accuracy and efficiency of detection.
Smart Images
Figure CN121830682A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect recognition technology in image analysis, specifically a method and system for rapid detection of defects in chip packaging processes. Background Technology
[0002] In chip packaging, epoxy resin and other encapsulation materials are used to protect and fix the bare chip, ensuring sufficient mechanical strength and electrical reliability during subsequent use. During the packaging process, liquid epoxy resin is prone to overflow during dispensing or pressing due to factors such as fluctuations in process parameters, differences in substrate surface tension, and equipment precision limitations. This overflow occurs when the encapsulation resin extends beyond the pre-defined packaging boundary into the chip's functional areas or pad areas. If this overflow defect is not detected in time, it will lead to pads being covered by the resin, causing poor soldering, pin short circuits, or obstructed chip heat dissipation paths, severely affecting the electrical performance and lifespan of the packaged product.
[0003] Current detection methods generally employ machine vision approaches based on visible light imaging. These methods use industrial cameras to capture color or grayscale images of the chip packaging area and then utilize grayscale thresholding or edge detection algorithms to identify areas of adhesive overflow. However, transparent adhesive materials are highly transparent after curing, and the difference between their reflectivity and transmittance characteristics in the visible light band and those of the chip packaging substrate is extremely small. This results in highly similar grayscale responses in visible light images, making it difficult to establish effective grayscale contrast. The lack of spectral sensing capability for transparent adhesive materials leads to overflow areas exhibiting a response distribution similar to the background, making it ineffective in addressing the specific scenario of transparent adhesive overflow. Summary of the Invention
[0004] (1) Technical problems to be solved The purpose of this invention is to provide a rapid detection method and system for chip packaging process defects, in order to solve the problem that traditional visible light visual inspection suffers from high missed detection rate and difficulty in accurate location of transparent adhesive overflow defects in chip packaging process due to the high similarity of grayscale response between transparent adhesive material and chip packaging substrate in visible light images.
[0005] (2) Technical solution To achieve the above objectives, in one aspect, the present invention provides a rapid detection method for chip packaging process defects, the method comprising: S1. A three-dimensional hyperspectral data cube is obtained by performing hyperspectral scanning on the packaged area of the chip to be tested; the three-dimensional hyperspectral data cube is then subjected to radiometric calibration band by band to obtain a radiometrically calibrated hyperspectral data cube.
[0006] S2. Select characteristic band combinations from all bands of the radiometrically calibrated hyperspectral data cube to identify significant differences in spectral response between the transparent adhesive material and the chip packaging substrate; extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the characteristic band combinations, and construct a characteristic band subset dataset.
[0007] S3. After the feature band subset is processed by dimensionality reduction mapping, a pseudo-color image is generated; the pseudo-color image is processed by color threshold segmentation to extract the connected pixel set of the glue overflow area, and the physical area corresponding to the connected pixel set and its spatial position coordinates in the chip packaging area are calculated; if the physical area exceeds the preset glue overflow area threshold, it is determined that there is a glue overflow defect in the chip packaging process and the corresponding spatial position coordinates are output.
[0008] Furthermore, the method for selecting characteristic band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube includes: Pixel samples from the transparent adhesive material region and the chip packaging substrate region are collected from the radiometrically calibrated hyperspectral data cube. The spectral response values of each pixel sample in all bands are extracted. The average spectral response values of the transparent adhesive material pixel samples and the chip packaging substrate pixel samples in each band are calculated to obtain the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate.
[0009] The spectral response difference between the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate is calculated band by band to obtain the spectral response difference for each band; characteristic band combinations are selected based on the spectral response difference for each band.
[0010] Furthermore, the method for selecting characteristic band combinations based on the spectral response differences corresponding to each band includes: The spectral response differences corresponding to each band are sorted in descending order to obtain a descending sequence of spectral response differences. The original band index corresponding to each spectral response difference is recorded to obtain a band index arrangement corresponding to the descending sequence of spectral response differences. The cumulative value of the ratio of each spectral response difference in the descending sequence of spectral response differences to the sum of the spectral response differences of all bands is calculated step by step to form a sequence of cumulative contribution ratios of spectral response differences.
[0011] According to the descending order of spectral response difference, band indices are extracted one by one from the band index arrangement and the current cumulative contribution ratio is updated synchronously until the current cumulative contribution ratio reaches the preset threshold of spectral difference contribution ratio. The number of extracted band indices is recorded as N. Based on the determined number of characteristic bands N, the first N band indices are extracted from the band index arrangement, and the bands corresponding to the N band indices in the radiometrically calibrated hyperspectral data cube are extracted to form a characteristic band combination.
[0012] Furthermore, the method for generating a pseudo-color image after processing the feature band subset data through dimensionality reduction mapping includes: The spectral response value sequence of all pixels in the feature band subset is centered with zero mean to obtain a mean-free spectral matrix. Then, the inter-pixel covariance matrix is calculated on the mean-free spectral matrix. The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvectors, and the first three principal component directions are selected from the set of eigenvectors.
[0013] The spectral response vector of each pixel in the feature band subset is sequentially projected onto the three principal component directions to obtain the three-dimensional principal component score corresponding to each pixel, forming a three-channel score matrix composed of the principal component scores of all pixels; after linear stretching and normalization of each channel of the three-channel score matrix, a pseudo-color image is synthesized pixel by pixel.
[0014] Furthermore, the method of performing eigenvalue decomposition on the covariance matrix to obtain an eigenvector set, and selecting the first three principal component directions from the eigenvector set includes: The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvalues and their corresponding eigenvectors arranged in descending order of eigenvalues. The cumulative contribution ratio sequence is formed by calculating the ratio of the eigenvalue corresponding to each eigenvector to the sum of all eigenvalues.
[0015] Feature vectors are extracted sequentially from the feature vector set in descending order of feature value, and the cumulative contribution ratio is updated in real time until three feature vectors are extracted. The three extracted feature vectors are then used as the three principal component directions.
[0016] Furthermore, the method for synthesizing a pseudo-color image pixel-by-pixel after linearly stretching and normalizing each channel of the three-channel scoring matrix includes: The distribution of all pixel scores in each channel of the three-channel scoring matrix is statistically analyzed. All pixel scores are arranged in ascending order to form a score sequence. Low-end truncation scores are determined from the score sequence based on a preset low-end cumulative pixel ratio, and high-end truncation scores are determined from the score sequence based on a preset high-end cumulative pixel ratio. Pixels with scores below the low-end truncation score are forcibly set to the low-end truncation score, and pixels with scores above the high-end truncation score are forcibly set to the high-end truncation score, resulting in a truncated channel score distribution. Using the low-end truncation score as the mapping starting point and the high-end truncation score as the mapping ending point, the score of each pixel in the truncated channel score distribution is mapped linearly to the minimum and maximum values of a unified grayscale quantization interval, resulting in a normalized grayscale value distribution. The above truncation and linear scaling process is repeated for each channel of the three-channel scoring matrix to obtain the normalized grayscale value distribution for each channel. The normalized grayscale value distributions of the three channels are assigned to the red, green, and blue color components respectively, and a pseudo-color image is synthesized pixel by pixel.
[0017] Furthermore, the method for performing color thresholding segmentation on the pseudo-color image and extracting the connected pixel set of the glue overflow region includes: The pseudo-color image is converted from the RGB color space to the HSV color space to obtain an HSV image composed of hue, saturation and lightness components. The pixel distribution histograms of the hue and saturation components of the HSV image are calculated respectively. The threshold ranges of the hue and saturation components are determined according to the peak intervals corresponding to the areas of excess adhesive material in the pixel distribution histograms.
[0018] For each pixel in the HSV image, determine whether its hue component value falls within the hue component threshold range and whether its saturation component value falls within the saturation component threshold range. Pixels that satisfy both conditions are marked as candidate overflow pixels, and pixels that do not satisfy either condition are marked as background pixels, thus obtaining a binarized mask image. The binarized mask image is then processed by connected component labeling, and spatially connected candidate overflow pixels are merged into the same connected pixel set and denoted as the connected pixel set of the overflow region.
[0019] Furthermore, the method of calculating the pixel distribution histograms for the hue and saturation components of the HSV image respectively, and determining the threshold ranges for the hue and saturation components based on the peak intervals corresponding to the areas of excess adhesive material in the pixel distribution histograms, includes: The frequency of the hue component values of all pixels in the HSV image is statistically analyzed according to a preset equally spaced binning method to obtain a hue component pixel distribution histogram. Mean filtering is then applied to the frequency values of each bin in the hue component pixel distribution histogram for smoothing. The frequency value of the current bin is replaced with the arithmetic mean of the frequency values of several adjacent bins centered on that bin, resulting in a smoothed hue component pixel distribution histogram. The smoothed hue component pixel distribution histogram is then used to retrieve the bin locations of each local maxima according to the frequency value from high to low, with bins containing values exceeding a certain threshold being selected. The local maxima of the average frequency of all bins are selected as candidate peak bins. From these candidate peak bins, target peak bins that match the hue response of the adhesive overflow area are determined. The bin boundaries corresponding to the points where the frequency value of the target peak bins decreases to a predetermined proportion to the frequency value of the target peak bins are used as the lower and upper limits of the hue component threshold range, respectively. The above statistical, smoothing, and peak bin determination process is repeated for the saturation component, and the lower and upper limits of the saturation component threshold range are determined according to the same frequency decrease proportion.
[0020] On the other hand, based on the same inventive concept, the present invention also provides a rapid detection system for chip packaging process defects, the system comprising: The hyperspectral data acquisition and radiometric calibration module is used to acquire a three-dimensional hyperspectral data cube by performing hyperspectral scanning on the packaged area of the chip to be tested; and to perform radiometric calibration on the three-dimensional hyperspectral data cube band by band to obtain a radiometrically calibrated hyperspectral data cube.
[0021] The feature band sub-dataset construction module is used to screen out feature band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube; and to extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the feature band combinations to construct the feature band sub-dataset.
[0022] The glue overflow defect determination and output module is used to generate a pseudo-color image by performing dimensionality reduction mapping on the feature band subset; to perform color threshold segmentation on the pseudo-color image, extract the connected pixel set of the glue overflow region, calculate the physical area corresponding to the connected pixel set and its spatial position coordinates within the chip packaging area; if the physical area exceeds a preset glue overflow area threshold, it is determined that there is a glue overflow defect in the chip packaging process and the corresponding spatial position coordinates are output.
[0023] (3) Beneficial effects Compared with the prior art, the beneficial effects of the present invention are: 1. By performing hyperspectral scanning on the chip packaging area, and utilizing the characteristic spectral absorption and reflection differences between the transparent adhesive material and the chip packaging substrate in specific narrow bands such as near-infrared, the method calculates the spectral response difference between the average spectral curves of the two materials band by band and dynamically determines the number of characteristic bands based on the cumulative contribution ratio. This adaptively selects the combination of characteristic bands that contributes the most to the identification of adhesive overflow areas from all bands, fundamentally solving the problem of missed detection of transparent adhesive overflow in visible light images due to the high similarity of grayscale responses, and significantly reducing the missed detection rate of adhesive overflow defects in chip packaging.
[0024] 2. By performing principal component analysis on the feature band subset dataset for dimensionality reduction mapping, the subtle differences between the transparent epoxy resin overflow area and the chip packaging substrate, which are implicit in the high-dimensional spectral space, are extracted and transformed into color contrasts that can be directly perceived in the pseudo-color image. Combined with HSV color space threshold segmentation and connected component labeling, the physical area calculation and spatial coordinates of the overflow area are accurately located and output. This makes the entire detection process, from spectral information acquisition to quantitative defect determination, form a complete closed loop, thereby improving the detection rate of overflow defects. Attached Figure Description
[0025] Figure 1 This is a flowchart of a rapid detection method for chip packaging process defects according to Embodiment 1 of the present invention.
[0026] Figure 2 This is a schematic diagram of the module composition of a chip packaging process defect rapid detection system according to Embodiment 2 of the present invention. Detailed Implementation
[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0028] Example 1: As Figure 1 As shown, this embodiment provides a rapid detection method for chip packaging process defects, the method comprising: S1. A three-dimensional hyperspectral data cube is obtained by performing hyperspectral scanning on the packaged area of the chip to be tested; the three-dimensional hyperspectral data cube is then subjected to radiometric calibration band by band to obtain a radiometrically calibrated hyperspectral data cube. For example, after the detection is initiated, the industrial control platform drives a pushbroom hyperspectral camera to scan the packaged area of the chip under test row by row, acquiring a sequence of hyperspectral images covering the visible to near-infrared bands. After acquisition, the image sequence is organized into a three-dimensional hyperspectral data cube according to the spatial row dimension, spatial column dimension, and band dimension, denoted as . ,in and These correspond to the number of rows and columns of spatial pixels, respectively. This represents the total number of bands.
[0029] Due to the dark current response and inconsistent sensitivity across different bands in hyperspectral camera sensors, the raw digital quantization values cannot directly reflect the true radiation intensity, necessitating radiometric calibration. Radiometric calibration is performed on each band within the three-dimensional hyperspectral data cube. Perform the following linear transformation pixel by pixel: ; in, For spatial location In the band The original digital quantization value, This represents the quantization value of the corresponding position and band in the dark-field reference image. This represents the quantization value of the corresponding position and band in the whiteboard reference image. The reflectivity value after radiometric calibration is normalized to [value missing]. The interval. After band-by-band radiometric calibration, the radiometrically calibrated hyperspectral data cube is obtained. This eliminates the interference of sensor dark current and response inhomogeneity on subsequent spectral analysis.
[0030] S2. Select characteristic band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube. The method for selecting characteristic band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube includes: Pixel samples from the transparent adhesive material region and the chip packaging substrate region are collected from the radiometrically calibrated hyperspectral data cube. The spectral response values of each pixel sample in all bands are extracted. The average spectral response values of the transparent adhesive material pixel samples and the chip packaging substrate pixel samples in each band are calculated to obtain the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate.
[0031] The spectral response difference between the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate is calculated band by band to obtain the spectral response difference for each band; characteristic band combinations are selected based on the spectral response difference for each band.
[0032] The method for selecting characteristic band combinations based on the differences in spectral response corresponding to each band includes: The spectral response differences corresponding to each band are sorted in descending order to obtain a descending sequence of spectral response differences. The original band index corresponding to each spectral response difference is recorded to obtain a band index arrangement corresponding to the descending sequence of spectral response differences. The cumulative value of the ratio of each spectral response difference in the descending sequence of spectral response differences to the sum of the spectral response differences of all bands is calculated step by step to form a sequence of cumulative contribution ratios of spectral response differences.
[0033] According to the descending order of spectral response difference, band indices are extracted one by one from the band index arrangement and the current cumulative contribution ratio is updated synchronously until the current cumulative contribution ratio reaches the preset threshold of spectral difference contribution ratio. The number of extracted band indices is recorded as N. Based on the determined number of characteristic bands N, the first N band indices are extracted from the band index arrangement, and the bands corresponding to the N band indices in the radiometrically calibrated hyperspectral data cube are extracted to form a characteristic band combination.
[0034] Extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the combination of feature bands, and construct the feature band subset dataset; For example, because the reflectivity of transparent adhesive materials is almost indistinguishable from that of chip packaging substrates in the visible light band, but in the near-infrared short-wave region (typically concentrated around 1150 nm to 1250 nm), the overtone absorption of the CH bonds in the transparent epoxy resin molecules will produce a difference in reflectivity relative to the measurable reflectivity of the chip packaging substrate material. If all bands are introduced into subsequent analysis without screening, a large number of bands that do not contribute to distinguishing the two types of materials will introduce redundant noise, thereby weakening the final detection accuracy. Therefore, adaptive screening is needed to extract the bands with the most concentrated information.
[0035] Pixel samples can originate from known areas manually delineated by operators in the image to be inspected based on offline fluorescence illumination annotation results, or they can be automatically extracted by the system during the initialization phase based on standard samples. For example, in an actual packaged product, 100 pixels can be taken from a normal pad area confirmed to be free of adhesive overflow as chip package substrate pixel samples, and 100 pixels can be taken from an area known to be covered by adhesive overflow as transparent adhesive material pixel samples. Extracting each pixel sample in the entire... After calculating the spectral response values in each band, the average spectral response of the transparent adhesive material pixel sample and the chip packaging substrate pixel sample in each band was obtained, thus yielding the average spectral curve of the transparent adhesive material. Average spectral curve of the chip packaging substrate , of which The components are as follows: , ; in , These represent the number of pixels in the transparent adhesive material pixel sample and the chip packaging substrate pixel sample, respectively. Each of the corresponding categories Each pixel sample in the band The reflectivity value after radiometric calibration.
[0036] Subsequently, the difference in spectral response between the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate was calculated band by band. ; Difference in spectral response across all bands Arranged in descending order, the descending sequence of spectral response differences is obtained. Simultaneously, the original band index corresponding to each arrangement position is recorded to obtain the band index arrangement corresponding to the descending sequence of spectral response differences. The cumulative ratio of each spectral response difference in the descending sequence to the total spectral response differences of all bands is calculated step by step to form a sequence of cumulative contribution ratios of spectral response differences. ; Band indices are extracted one by one from the band index sequence according to the descending order of spectral response difference, and the current cumulative contribution ratio is updated synchronously until the current cumulative contribution ratio first reaches or exceeds the preset threshold for spectral difference contribution ratio. Up to this point, the number of band indices extracted is recorded as the number of feature bands. Threshold for the contribution percentage of spectral differences In practice, it is usually set to 0.90, meaning the cumulative contribution ratio is no less than 90%. For example, if a hyperspectral camera acquires 280 bands, and the sum of the spectral response differences across all bands reaches a certain value, and the cumulative contribution ratio of the top 12 bands exceeds 90% for the first time when the spectral response differences of the first 12 bands are gradually added together, then the number of characteristic bands... The number is determined to be 12, and the corresponding 12 band indices constitute the final characteristic band combination. This is based on the determined number of characteristic bands. Extracting the first band from the band index sort Each band index corresponds to the above in the radiometrically calibrated hyperspectral data cube. The bands of each band index are extracted to form a combination of characteristic bands.
[0037] After determining the feature band combination, the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the feature band combination are extracted to construct the feature band subset dataset. The feature band subset retains the band information that contributes most to distinguishing between transparent adhesive materials and chip packaging substrates, while removing redundant bands whose contribution to distinguishing the two types of materials is negligible.
[0038] S3. After processing the feature band subset data through dimensionality reduction mapping, a pseudo-color image is generated; The method for generating a pseudo-color image after processing the feature band subset data through dimensionality reduction mapping includes: The spectral response value sequence of all pixels in the feature band subset is centered with zero mean to obtain a mean-free spectral matrix. Then, the pixel covariance matrix is calculated on the mean-free spectral matrix. The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvectors, and the first three principal component directions are selected from the set of eigenvectors. The method of performing eigenvalue decomposition on the covariance matrix to obtain an eigenvector set, and selecting the first three principal component directions from the eigenvector set includes: The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvalues and their corresponding eigenvectors arranged in descending order of eigenvalues; the cumulative contribution ratio sequence is formed by calculating the ratio of the eigenvalue corresponding to each eigenvector to the sum of all eigenvalues. Feature vectors are extracted sequentially from the feature vector set in descending order of feature value, and the cumulative contribution ratio is updated in real time until three feature vectors are extracted. The three extracted feature vectors are then used as the three principal component directions.
[0039] The spectral response vector of each pixel in the feature band subset is sequentially projected onto the three principal component directions to obtain the three-dimensional principal component score corresponding to each pixel, forming a three-channel score matrix composed of the principal component scores of all pixels; after linear stretching and normalization of each channel of the three-channel score matrix, a pseudo-color image is synthesized pixel by pixel.
[0040] The method for synthesizing a pseudo-color image pixel-by-pixel after linearly stretching and normalizing each channel of the three-channel scoring matrix includes: The distribution of all pixel scores in each channel of the three-channel scoring matrix is statistically analyzed. All pixel scores are arranged in ascending order to form a score sequence. Low-end truncation scores are determined from the score sequence based on a preset low-end cumulative pixel ratio, and high-end truncation scores are determined from the score sequence based on a preset high-end cumulative pixel ratio. Pixels with scores below the low-end truncation score are forcibly set to the low-end truncation score, and pixels with scores above the high-end truncation score are forcibly set to the high-end truncation score, resulting in a truncated channel score distribution. Using the low-end truncation score as the mapping starting point and the high-end truncation score as the mapping ending point, the score of each pixel in the truncated channel score distribution is mapped linearly to the minimum and maximum values of a unified grayscale quantization interval, resulting in a normalized grayscale value distribution. The above truncation and linear scaling process is repeated for each channel of the three-channel scoring matrix to obtain the normalized grayscale value distribution for each channel. The normalized grayscale value distributions of the three channels are assigned to the red, green, and blue color components respectively, and a pseudo-color image is synthesized pixel by pixel.
[0041] For example, each pixel in the feature band subset carries a Since it is difficult to form a perceptible defect judgment by directly observing the spectral vector band by band, principal component analysis is used to perform dimensionality reduction mapping on the feature band subset data. The subtle differences between the transparent adhesive material and the chip packaging substrate hidden in the high-dimensional spectral space are concentrated and mapped to the three principal component directions, and finally synthesized into an intuitive pseudo-color image.
[0042] The first step in dimensionality reduction mapping is zero-mean centering. This involves sub-datasets of the feature bands. All The spectral response value sequence of each pixel is arranged into a matrix. Calculate the mean of the spectral response values of all pixels in each band dimension. Then subtract the spectral response values of all pixels in that band. The mean-removed spectral matrix is obtained. This process centers the values of each band to zero. The purpose of zero-mean centering is to eliminate the influence of differences in absolute response intensity across bands on subsequent covariance matrix calculations, ensuring that the covariance matrix accurately reflects the relative changes between the spectral response value sequences of each pixel.
[0043] After zero-mean centering, the mean-removed spectral matrix is processed. Calculate the inter-pixel covariance matrix For the covariance matrix Perform eigenvalue decomposition to obtain eigenvalues arranged in descending order of their values. With the corresponding unit orthogonal eigenvector The cumulative contribution ratio sequence is formed by successively calculating the ratio of the corresponding eigenvalue of each eigenvector to the sum of all eigenvalues. ; Feature vectors are extracted sequentially from the feature vector set in descending order of their eigenvalues, with the cumulative contribution ratio updated in real time, until three feature vectors are extracted. The three extracted feature vectors are then... The three principal component directions are chosen because the mathematical properties of principal component analysis ensure that they capture the three orthogonal variation directions with the largest variance in the feature band subset. The information carried by these three directions is usually sufficient to distinguish the adhesive overflow area from the chip packaging substrate area, and they also correspond exactly to the number of three color component channels required for pseudo-color images.
[0044] The spectral response value vector of each pixel in the feature band subset. (i.e., the mean-removed spectral matrix) The corresponding row vectors are projected sequentially onto the three principal component directions to obtain the 3D principal component score for each pixel: ; After performing the above projection on all pixels, a three-channel score matrix is formed, consisting of the principal component scores of all pixels. .
[0045] Because the numerical ranges of each channel in the three-channel scoring matrix differ significantly, directly mapping them to the grayscale quantization range would cause some extreme pixels to compress the dynamic range of a large number of normal pixels. Therefore, a linear stretching and normalization process is performed on each channel of the three-channel scoring matrix. Specifically, for each channel... All The pixel scores are arranged in ascending order to form a score sequence. The low-end truncation score is determined from the score sequence based on a preset low-end cumulative pixel ratio (usually 2%). (i.e., the 2nd percentile) The high-end truncation score is determined from the score sequence based on the preset high-end cumulative pixel ratio (usually 98%). (i.e., the 98th percentile). Pixels with scores below the low-end truncation score are forced to the low-end truncation score, and pixels with scores above the high-end truncation score are forced to the high-end truncation score, resulting in the truncated channel score distribution. Using the low-end truncation score as the mapping starting point and the high-end truncation score as the mapping ending point, the score of each pixel in the truncated channel score distribution is mapped to a uniform grayscale quantization range according to a linear proportional relationship. Between the minimum and maximum values: ; Obtain the normalized grayscale value distribution corresponding to this channel. In a pseudo-color image, the area where the transparent adhesive material overflows exhibits a color distinct from that of the chip packaging substrate. For example, the chip packaging substrate appears as a bluish-gray tone, while the area where the transparent adhesive material overflows appears as a yellowish-green tone. This color difference is a direct manifestation of the subtle differences in the spectral response of the transparent adhesive material and the chip packaging substrate in the characteristic bands, magnified and visualized by principal component projection.
[0046] The pseudo-color image is subjected to color threshold segmentation to extract the set of connected pixels in the glue overflow area. The method for performing color thresholding on the pseudo-color image and extracting the connected pixel set of the glue overflow region includes: The pseudo-color image is converted from the RGB color space to the HSV color space to obtain an HSV image composed of hue, saturation and lightness components. The pixel distribution histograms of the hue and saturation components of the HSV image are calculated respectively. The threshold ranges of the hue and saturation components are determined according to the peak intervals corresponding to the areas of excess adhesive material in the pixel distribution histograms.
[0047] The method of calculating the pixel distribution histograms for the hue and saturation components of the HSV image respectively, and determining the threshold ranges for the hue and saturation components based on the peak intervals corresponding to the areas of excess adhesive material in the pixel distribution histograms, includes: The frequency of the hue component values of all pixels in the HSV image is statistically analyzed according to a preset equally spaced binning method to obtain a hue component pixel distribution histogram. Mean filtering is then applied to the frequency values of each bin in the hue component pixel distribution histogram for smoothing. The frequency value of the current bin is replaced with the arithmetic mean of the frequency values of several adjacent bins centered on that bin, resulting in a smoothed hue component pixel distribution histogram. The smoothed hue component pixel distribution histogram is then used to retrieve the bin locations of each local maxima according to the frequency value from high to low, with bins containing values exceeding a certain threshold being selected. The local maxima of the average frequency of all bins are selected as candidate peak bins. From these candidate peak bins, target peak bins that match the hue response of the adhesive overflow area are determined. The bin boundaries corresponding to the points where the frequency value of the target peak bins decreases to a predetermined proportion to the frequency value of the target peak bins are used as the lower and upper limits of the hue component threshold range, respectively. The above statistical, smoothing, and peak bin determination process is repeated for the saturation component, and the lower and upper limits of the saturation component threshold range are determined according to the same frequency decrease proportion.
[0048] For each pixel in the HSV image, determine whether its hue component value falls within the hue component threshold range and whether its saturation component value falls within the saturation component threshold range. Pixels that satisfy both conditions are marked as candidate overflow pixels, and pixels that do not satisfy either condition are marked as background pixels, thus obtaining a binarized mask image. The binarized mask image is then processed by connected component labeling, and spatially connected candidate overflow pixels are merged into the same connected pixel set and denoted as the connected pixel set of the overflow region.
[0049] Calculate the physical area corresponding to the connected pixel set and its spatial coordinates within the chip packaging area; if the physical area exceeds a preset glue overflow area threshold, determine that there is a glue overflow defect in the chip packaging process and output the corresponding spatial coordinates.
[0050] For example, the hue components of the HSV color space are not sensitive to changes in light intensity and only reflect color category attributes. Therefore, the difference in hue components between the area of the adhesive overflow in the transparent adhesive material and the chip packaging substrate area is more stable and reliable than the grayscale difference of any single channel in the RGB color space.
[0051] Taking the hue component as an example, the hue component values of all pixels in the HSV image are statistically analyzed according to a preset equally spaced binning method, resulting in a hue component pixel distribution histogram. The equally spaced binning typically divides the hue component range... The image was divided into 72 bins, each with a width of 5°. Mean filtering was applied sequentially to the frequency values of each bin in the histogram of the hue component pixel distribution, smoothing the result with a window width of [missing information]. Typically, five bins are used. A moving average is applied to the frequency values of each bin, and boundary indices are filled with boundary values to obtain a smoothed histogram of hue component pixel distribution. Mean filtering smoothing suppresses random frequency fluctuations in the hue component pixel distribution histogram caused by the limited number of pixels, making local maximum retrieval more robust. The smoothed hue component pixel distribution histogram is then used to retrieve the bin locations of local maxima in descending order of frequency value. Bins with local maxima whose frequency values exceed the average of all bins are selected as candidate peak bins. Combining this with the known hue response range of the transparent adhesive material overflow area (determined offline through standard sample calibration during system initialization; for example, the hue component peak of the transparent adhesive material overflow area falls around 60° in the yellow-green segment), target peak bins matching the hue response of the transparent adhesive material overflow area are selected from the candidate peak bins. Extending outwards to the left and right from the target peak bin, find the bin boundaries corresponding to the points where the frequency value drops to a preset percentage (usually 50%) of the target peak bin frequency value. These boundaries serve as the lower limits of the hue component threshold range. and upper limit The above statistical, mean filtering, and candidate peak binning processes are repeated for the saturation component. The lower limit of the saturation component threshold range is determined by the same frequency decrease ratio. and upper limit .
[0052] After determining the threshold ranges for the hue component and saturation component, the hue component value of each pixel in the HSV image is checked to see if it falls within the hue component threshold range. Within, and whether the saturation component value falls within the saturation component threshold range. Within the image, pixels that simultaneously meet both of the above conditions are marked as candidate glue overflow pixels (assigned a value of 1), and pixels that do not meet either condition are marked as background pixels (assigned a value of 0), resulting in a binarized mask image. The binarized mask image is then processed using connected component labeling (using a four-neighbor or eight-neighbor connectivity criterion), merging spatially connected candidate glue overflow pixels into the same connected pixel set, which is denoted as the connected pixel set of the transparent adhesive material overflow region.
[0053] The calculation of physical area depends on the spatial resolution parameter of the hyperspectral camera, that is, the actual physical side length corresponding to each pixel. (Unit is) (This parameter is determined using a standard scale during the system calibration phase.) ; in, This represents the total number of pixels in the connected pixel set. The unit is Spatial location coordinates are represented by the centroid coordinates of this connected pixel set: ; in, This represents the set of connected pixels, with coordinates originating from the starting point of the chip package area scan, and the unit is . If the calculated physical area The overflow area exceeds the preset threshold (this threshold is set according to the product process specifications; for example, for a packaged product with pad sizes of 200μm × 200μm, the overflow area threshold can be set to 500μm). 2 If the region corresponding to the connected pixel set is determined to be a real glue overflow defect, the system outputs the defect determination conclusion and the corresponding spatial coordinates. If the physical area does not exceed the glue overflow area threshold, the connected pixel set will be treated as noise or minor impurity points and ignored, and defect judgment will not be triggered.
[0054] Example 2: Based on the same inventive concept, such as Figure 2As shown, this embodiment also provides a rapid detection system for chip packaging process defects, the system comprising: The hyperspectral data acquisition and radiometric calibration module is used to acquire a three-dimensional hyperspectral data cube by performing hyperspectral scanning on the packaged area of the chip to be tested; and to perform radiometric calibration on the three-dimensional hyperspectral data cube band by band to obtain a radiometrically calibrated hyperspectral data cube.
[0055] The feature band sub-dataset construction module is used to screen out feature band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube; and to extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the feature band combinations to construct the feature band sub-dataset.
[0056] The glue overflow defect determination and output module is used to generate a pseudo-color image by performing dimensionality reduction mapping on the feature band subset; to perform color threshold segmentation on the pseudo-color image, extract the connected pixel set of the glue overflow region, calculate the physical area corresponding to the connected pixel set and its spatial position coordinates within the chip packaging area; if the physical area exceeds a preset glue overflow area threshold, it is determined that there is a glue overflow defect in the chip packaging process and the corresponding spatial position coordinates are output.
[0057] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.
[0058] Finally, it should be noted that although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for rapid detection of defects in chip packaging processes, characterized in that, The method includes: A three-dimensional hyperspectral data cube is obtained by performing hyperspectral scanning on the packaged area of the chip to be tested; the three-dimensional hyperspectral data cube is then subjected to radiometric calibration band by band to obtain a radiometrically calibrated hyperspectral data cube. From all bands of the radiometrically calibrated hyperspectral data cube, we screen out the characteristic band combinations that show significant differences in spectral response between the transparent adhesive material and the chip packaging substrate; we extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the characteristic band combinations, and construct the characteristic band subset dataset. After the feature band subset is processed by dimensionality reduction mapping, a pseudo-color image is generated. The pseudo-color image is then processed by color threshold segmentation to extract the connected pixel set of the glue overflow area. The physical area corresponding to the connected pixel set and its spatial coordinates within the chip packaging area are calculated. If the physical area exceeds the preset glue overflow area threshold, it is determined that there is a glue overflow defect in the chip packaging process and the corresponding spatial coordinates are output.
2. The method for rapid detection of defects in chip packaging process according to claim 1, characterized in that, The method for selecting characteristic band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube includes: Pixel samples from the transparent adhesive material region and the chip packaging substrate region are collected from the radiometrically calibrated hyperspectral data cube. The spectral response values of each pixel sample in all bands are extracted. The average spectral response values of the transparent adhesive material pixel samples and the chip packaging substrate pixel samples in each band are calculated to obtain the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate. The spectral response difference between the average spectral curve of the transparent adhesive material and the average spectral curve of the chip packaging substrate is calculated band by band to obtain the spectral response difference for each band; characteristic band combinations are selected based on the spectral response difference for each band.
3. The method for rapid detection of defects in chip packaging process according to claim 2, characterized in that, The method for selecting characteristic band combinations based on the differences in spectral response corresponding to each band includes: The spectral response differences corresponding to each band are sorted in descending order to obtain a descending sequence of spectral response differences. The original band index corresponding to each spectral response difference is recorded to obtain a band index arrangement corresponding to the descending sequence of spectral response differences. The cumulative value of the ratio of each spectral response difference in the descending sequence of spectral response differences to the sum of the spectral response differences of all bands is calculated step by step to form a sequence of cumulative contribution ratios of spectral response differences. According to the descending order of spectral response difference, band indices are extracted one by one from the band index arrangement and the current cumulative contribution ratio is updated synchronously until the current cumulative contribution ratio reaches the preset threshold of spectral difference contribution ratio. The number of extracted band indices is recorded as N. Based on the determined number of characteristic bands N, the first N band indices are extracted from the band index arrangement, and the bands corresponding to the N band indices in the radiometrically calibrated hyperspectral data cube are extracted to form a characteristic band combination.
4. The method for rapid detection of defects in chip packaging process according to claim 1, characterized in that, The method for generating a pseudo-color image after processing the feature band subset data through dimensionality reduction mapping includes: The spectral response value sequence of all pixels in the feature band subset is centered with zero mean to obtain a mean-free spectral matrix. Then, the pixel covariance matrix is calculated on the mean-free spectral matrix. The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvectors, and the first three principal component directions are selected from the set of eigenvectors. The spectral response vector of each pixel in the feature band subset is sequentially projected onto the three principal component directions to obtain the three-dimensional principal component score corresponding to each pixel, forming a three-channel score matrix composed of the principal component scores of all pixels; after linear stretching and normalization of each channel of the three-channel score matrix, a pseudo-color image is synthesized pixel by pixel.
5. The method for rapid detection of defects in chip packaging process according to claim 4, characterized in that, The method of performing eigenvalue decomposition on the covariance matrix to obtain an eigenvector set, and selecting the first three principal component directions from the eigenvector set includes: The covariance matrix is decomposed into eigenvalues to obtain a set of eigenvalues and their corresponding eigenvectors arranged in descending order of eigenvalues; the cumulative contribution ratio sequence is formed by calculating the ratio of the eigenvalue corresponding to each eigenvector to the sum of all eigenvalues. Feature vectors are extracted sequentially from the feature vector set in descending order of feature value, and the cumulative contribution ratio is updated in real time until three feature vectors are extracted. The three extracted feature vectors are then used as the three principal component directions.
6. The method for rapid detection of defects in chip packaging process according to claim 4, characterized in that, The method for synthesizing a pseudo-color image pixel-by-pixel after linearly stretching and normalizing each channel of the three-channel scoring matrix includes: The distribution of all pixel scores in each channel of the three-channel scoring matrix is statistically analyzed. All pixel scores are arranged in ascending order to form a score sequence. Low-end truncation scores are determined from the score sequence based on a preset low-end cumulative pixel ratio, and high-end truncation scores are determined from the score sequence based on a preset high-end cumulative pixel ratio. Pixels with scores below the low-end truncation score are forcibly set to the low-end truncation score, and pixels with scores above the high-end truncation score are forcibly set to the high-end truncation score, resulting in a truncated channel score distribution. Using the low-end truncation score as the mapping starting point and the high-end truncation score as the mapping ending point, the score of each pixel in the truncated channel score distribution is mapped linearly to the minimum and maximum values of a unified grayscale quantization interval, resulting in a normalized grayscale value distribution. The above truncation and linear scaling process is repeated for each channel of the three-channel scoring matrix to obtain the normalized grayscale value distribution for each channel. The normalized grayscale value distributions of the three channels are assigned to the red, green, and blue color components respectively, and a pseudo-color image is synthesized pixel by pixel.
7. The method for rapid detection of defects in chip packaging process according to claim 1, characterized in that, The method for performing color thresholding on the pseudo-color image and extracting the connected pixel set of the glue overflow region includes: The pseudo-color image is converted from the RGB color space to the HSV color space to obtain an HSV image composed of hue, saturation and lightness components. The pixel distribution histograms of the hue and saturation components of the HSV image are calculated respectively. The threshold range of the hue component and the threshold range of the saturation component are determined according to the peak intervals corresponding to the overflow area of the transparent adhesive material in the pixel distribution histogram. For each pixel in the HSV image, determine whether its hue component value falls within the hue component threshold range and whether its saturation component value falls within the saturation component threshold range. Pixels that satisfy both conditions are marked as candidate overflow pixels, and pixels that do not satisfy either condition are marked as background pixels, thus obtaining a binarized mask image. The binarized mask image is then processed by connected component labeling, and spatially connected candidate overflow pixels are merged into the same connected pixel set and denoted as the connected pixel set of the overflow region.
8. The method for rapid detection of defects in chip packaging process according to claim 7, characterized in that, The method of calculating the pixel distribution histograms for the hue and saturation components of the HSV image respectively, and determining the threshold ranges for the hue and saturation components based on the peak intervals corresponding to the areas of excess adhesive material in the pixel distribution histograms, includes: The frequency of the hue component values of all pixels in the HSV image is statistically analyzed according to a preset equally spaced binning method to obtain a hue component pixel distribution histogram. Mean filtering is then applied to the frequency values of each bin in the hue component pixel distribution histogram for smoothing. The frequency value of the current bin is replaced with the arithmetic mean of the frequency values of several adjacent bins centered on that bin, resulting in a smoothed hue component pixel distribution histogram. The smoothed hue component pixel distribution histogram is then used to retrieve the bin locations of each local maxima according to the frequency value from high to low, with bins containing values exceeding a certain threshold being selected. The local maxima of the average frequency of all bins are selected as candidate peak bins. From these candidate peak bins, target peak bins that match the hue response of the adhesive overflow area are determined. The bin boundaries corresponding to the points where the frequency value of the target peak bins decreases to a predetermined proportion to the frequency value of the target peak bins are used as the lower and upper limits of the hue component threshold range, respectively. The above statistical, smoothing, and peak bin determination process is repeated for the saturation component, and the lower and upper limits of the saturation component threshold range are determined according to the same frequency decrease proportion.
9. A rapid detection system for chip packaging process defects, characterized in that, The system includes: The hyperspectral data acquisition and radiometric calibration module is used to acquire a three-dimensional hyperspectral data cube by performing hyperspectral scanning on the packaged area of the chip to be tested; and to perform radiometric calibration on the three-dimensional hyperspectral data cube band by band to obtain a radiometrically calibrated hyperspectral data cube. The feature band sub-dataset construction module is used to screen out feature band combinations with significant differences in spectral response between the transparent adhesive material and the chip packaging substrate from all bands of the radiometrically calibrated hyperspectral data cube; and to extract the spectral response values of each pixel in the radiometrically calibrated hyperspectral data cube under the feature band combinations to construct the feature band sub-dataset. The glue overflow defect determination and output module is used to generate a pseudo-color image by performing dimensionality reduction mapping on the feature band subset; to perform color threshold segmentation on the pseudo-color image, extract the connected pixel set of the glue overflow region, calculate the physical area corresponding to the connected pixel set and its spatial position coordinates within the chip packaging area; if the physical area exceeds a preset glue overflow area threshold, it is determined that there is a glue overflow defect in the chip packaging process and the corresponding spatial position coordinates are output.