An intelligent multi-dimensional defect detection system for chip packaging

Abstract

The present invention belongs to the field of visual inspection, in particular to an intelligent multi-dimensional defect detection system for chip packaging, including multiple modules and a subsystem; among them, the electro-thermal linkage detection module is composed of an ATE automatic test equipment and an infrared thermal imager, which collects the electrical parameter matrix and thermal imaging matrix of the chip; the multi-dimensional visual detection module includes an X-ray tomography imaging unit, a 3D structured light imaging unit and a planar vision unit; it is used to reconstruct the internal structure of the chip, obtain the 3D surface topography and extract the planar defect features; the multi-modal feature fusion module is based on the cross-modal attention mechanism and multi-task joint learning model to fuse the electro-thermal data, X-ray voxel features and 3D topography features; the hierarchical defect decision subsystem locates the defect area through FastR-CNN, classifies the defect types through the graph convolutional network, and outputs the final defect determination result based on the Bayesian inference network by integrating multi-modal evidence; through the multi-modal data linkage, the joint detection of voids, heat dissipation and appearance defects can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and particularly relates to an intelligent multi-dimensional defect detection system for chip packaging. Background Art

[0002] With the continuous improvement of integrated circuit manufacturing technology, chip packaging defect detection has become a key link to ensure product quality. However, the traditional detection methods have the following problems:

[0003] 1. Single detection dimension: The existing technologies mostly adopt the method of separating physical detection (appearance defects) from electrical performance detection, resulting in low detection efficiency and inability to detect cross-modal associated defects, such as abnormal heat dissipation caused by internal voids;

[0004] 2. Insufficient internal defect detection: Conventional X-ray or optical detection is difficult to accurately identify internal structure defects of the package, such as micron-level voids and pin soldering defects, and cannot perform linkage analysis with electro-thermal characteristics;

[0005] 3. Poor algorithm adaptability: Existing visual detection algorithms, such as traditional LBP feature extraction, have insufficient classification accuracy for complex printing defects (blurred, tilted, broken words) and lack the ability of multi-modal data fusion.

[0006] Therefore, we propose a solution to make up for the deficiencies of the traditional solution to meet the market demand. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent multi-dimensional defect detection system for chip packaging to solve the problems raised in the background art.

[0008] To achieve the above purpose, the present invention provides the following technical solution: An intelligent multi-dimensional defect detection system for chip packaging, the system includes the following modules:

[0009] Electro-thermal linkage detection module: Composed of an ATE automatic test equipment and an infrared thermal imager, synchronously collecting the electrical parameter matrix and thermal imaging matrix of the chip;

[0010] Multi-dimensional visual detection module: Including an X-ray tomography unit, a 3D structured light imaging unit and a planar vision unit, respectively used for reconstructing the internal structure of the chip, obtaining the 3D surface topography and extracting planar defect features;

[0011] Multi-modal feature fusion module: Based on the cross-modal attention mechanism and multi-task joint learning model, fusing electro-thermal data, X-ray voxel features and 3D topography features;

[0012] Hierarchical Defect Decision Subsystem: Locate the defect area through FastR-CNN, classify the defect type using a graph convolutional network, and output the final defect determination result based on a Bayesian inference network by integrating multi-modal evidence.

[0013] In the multi-dimensional defect intelligent detection system for chip packaging of the present invention, the electrical parameter matrix is ;

[0014] The thermal imaging matrix is ;

[0015] where V i is the voltage sequence, I j is the current value, T k is the environmental temperature parameter, H l is the humidity parameter, and i, j, k, and l are all positive integers; is the spatio-temporal temperature distribution.

[0016] In the multi-dimensional defect intelligent detection system for chip packaging of the present invention, the cross-modal attention mechanism is: ;

[0017] where Kj and Qi are the query vector and key vector within the image sampling range respectively, and d is the feature dimension.

[0018] In the multi-dimensional defect intelligent detection system for chip packaging of the present invention, the electro-thermal linkage detection module further includes:

[0019] An electro-thermal coupling analysis model that establishes the correlation between the current density J and heat diffusion through a partial differential equation to infer internal voids or heat dissipation defects;

[0020] where is the Laplace operator, is the heat generation rate per unit volume of the chip, m is the thermal conductivity of the packaging material, is the thermal diffusion coefficient.

[0021] In the multi-dimensional defect intelligent detection system for chip packaging of the present invention, the X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip to detect pin soldering defects or internal micro-cracks, and its reconstruction formula is: ;

[0022] where μ(r) is the linear attenuation coefficient of the voxel, D is the distance from the X-ray source to the detector, s is the coordinate on the detector plane, r is the voxel position vector in three-dimensional space, is the projection angle, is at the angle The projection value measured at the position s of the lower detector, and h(s’-s) is the ramp filter.

[0023] In the chip package multi-dimensional defect intelligent detection system of the present invention, the 3D structured light imaging unit calculates the absolute phase value by the three-step phase-shift method , and combines the point cloud registration algorithm to detect pin bending or package deformation, where as follows: ;

[0024] In the formula, I1, I2, and I3 are the intensities of the phase-shifted fringe images.

[0025] In the chip package multi-dimensional defect intelligent detection system of the present invention, the planar vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify misspelled words, blurring, and tilting defects of printed symbols. The LBP operator is as follows: ;

[0026] In the formula, P is the number of neighborhood sampling points, riu2 is the rotation-invariant uniform pattern, R is the sampling radius, c represents the gray value of the central pixel, p is the index of the neighborhood pixel, and the value range is p = 0, 1, 2, …, .

[0027] In the chip package multi-dimensional defect intelligent detection system of the present invention, the total loss function of the multi-task joint learning model is: ;

[0028] where L cls is the cross-entropy classification loss, L reg is the regression loss of electrical performance parameters, L recon is the X-ray voxel reconstruction loss, and λ1, λ2, and λ3 are weight coefficients.

[0029] In the chip package multi-dimensional defect intelligent detection system of the present invention, the hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning by combining the GCN node update formula.

[0030] In the chip package multi-dimensional defect intelligent detection system of the present invention,

[0031] the alignment formula is , where S is the feature map downsampling rate;

[0032] The GCN node update formula is ; in the formula is the activation function, N(i) is the set of neighborhood nodes, is the learnable weight matrix of the th layer, is the number of layers of the neural network, is the feature vector of node j in the layer, where i represents the current node and j represents the neighbor node of node i.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] Through the linkage of electro-thermal-X-ray multimodal data, multi-dimensional joint detection of internal cavities, heat dissipation defects and appearance defects is realized. In particular, by combining electrical parameters (V, I) with thermal imaging data, real defects and noise interference can be finely distinguished, greatly reducing the false alarm rate and achieving a higher comprehensive detection rate. Moreover, the cross-modal attention mechanism is used to improve the feature fusion efficiency, and the false alarm rate can be reduced synchronously. In addition, the system supports real-time detection and can be applied to high-precision semiconductor packaging production lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0036] Figure 1 is a schematic structural diagram of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present invention and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally further include steps or units not listed, or may optionally further include other steps or units inherent to these processes, methods, products or devices.

[0038] Referring to "embodiments" herein means that a particular feature, structure or characteristic described in connection with the embodiments can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0039] "Multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0040] Moreover, terms indicating directions such as "up, down, left, right, upper end, lower end, longitudinal" etc. are all referenced based on the posture position of the device or equipment described in this solution during normal use.

[0041] In order to make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be described clearly and completely below. Obviously, the described embodiments are partial embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present invention.

[0042] This embodiment discloses a multi-dimensional defect intelligent detection system for chip packaging as Figure 1 shown, and this system includes the following modules:

[0043] Electro-thermal linkage detection module: Composed of an ATE automatic test equipment and an infrared thermal imager, it synchronously collects the electrical parameter matrix and thermal imaging matrix of the chip, and can dynamically adjust the ATE test parameters according to the electrical performance detection results. The voltage adjustment formula is V new =V base +ΔV*sign(I dev ), where ;

[0044] Multi-dimensional vision detection module: Includes an X-ray tomography imaging unit, a 3D structured light imaging unit, and a planar vision unit, which are respectively used to reconstruct the internal structure of the chip, obtain the 3D surface topography, and extract planar defect features; among them, the resolution of the X-ray microfocus source ≤ 5μm, the thermal sensitivity of the thermal imager ≤ 20mK, the Z-axis accuracy of the 3D structured light camera ± 2μm, and the planar vision unit uses a global shutter CMOS camera with a frame rate ≥ 500fps / 5MP resolution and is equipped with a coaxial light source to eliminate reflection;

[0045] Multi-modal feature fusion module: Based on a cross-modal attention mechanism and a multi-task joint learning model, it fuses electro-thermal data, X-ray voxel features, and 3D topography features;

[0046] Hierarchical defect decision subsystem: Locates the defect area through FastR-CNN, classifies the defect type by a graph convolutional network, and outputs the final defect determination result based on a Bayesian inference network by synthesizing multi-modal evidence.

[0047] Through three processing modules and a subsystem, an electro-thermal-X-ray multi-modal data linkage is formed to achieve the joint detection of internal voids, heat dissipation defects, and appearance defects. In particular, by combining electrical parameters and thermal imaging data, real defects can be finely distinguished from noise interference, greatly reducing the false alarm rate and achieving a higher comprehensive detection rate. Moreover, the cross-modal attention mechanism is used to improve the feature fusion efficiency, and the false alarm rate can be reduced synchronously. In addition, the system supports real-time detection and is applicable to high-precision semiconductor packaging production lines.

[0048] In this embodiment, the electrical parameter matrix is ;

[0049] The thermal imaging matrix is ;

[0050] Among them, V i is the voltage sequence, I j is the current value, T k is the ambient temperature parameter, H l is the humidity parameter, and i, j, k, and l are all positive integers; is the spatio-temporal temperature distribution.

[0051] In this embodiment, the cross-modal attention mechanism is: ;

[0052] Among them, Q i is the query vector from the source modality (such as thermal imaging data), and K j is the key vector from the target modality (such as X-ray data), and represents the dot product (inner product) of the two vectors, used to measure their similarity. The role of T in the formula is to convert the row vector Q i into a column vector for matrix multiplication, and d is the feature dimension.

[0053] In this embodiment, the electro-thermal linkage detection module further includes:

[0054] An electro-thermal coupling analysis model, which establishes the correlation between current density J and heat diffusion through the partial differential equation ; establishing a physical model of current density and heat diffusion can infer the abnormal heat dissipation caused by internal voids in the package, improving the detection rate of internal defects. In particular, by combining electrical parameters (V, I) and thermal imaging data, real defects can be finely distinguished from noise interference, greatly reducing the false alarm rate;

[0055] Among them, is the Laplace operator, is the heat generation rate per unit volume of the chip, calculated by the product of V, I, and t; m is the thermal conductivity of the packaging material, is the thermal diffusivity, , is the density, is the specific heat capacity.

[0056] Specifically, when the current density J passes through the chip conductor, the Joule heat power density is:

[0057] In the formula, J = I / A (current density, I is the current, and A is the cross-sectional area of the conductor);

[0058] σ is the electrical conductivity of the material; the current I and voltage V obtained through electrical performance detection, combined with the geometric parameters of the conductor, can be used to calculate J and ;

[0059] In this embodiment, the X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip, which is used to detect pin soldering defects or internal microcracks. Its reconstruction formula is: ;

[0060] where μ(r) is the linear attenuation coefficient of the voxel;

[0061] D is the distance from the X-ray source to the detector;

[0062] s is the coordinate on the detector plane, representing the specific position on the X-ray detector;

[0063] r is the voxel position vector in three-dimensional space, that is, the spatial coordinates (x, y, z) of the point to be reconstructed;

[0064] is the projection angle, representing the angle at which the X-ray source and the detector rotate around the object (range 0 ≤ < 2π);

[0065] is the projection value measured at the detector position s at the angle , that is, the X-ray attenuation integral;

[0066] h(s’ - s) is the ramp filter;

[0067] The meaning of the formula is to reconstruct the linear attenuation coefficient μ(r) at the voxel r by integrating the data of all projection angles and combining the geometric correction factor .

[0068] X-ray tomography uses the FDK algorithm for reconstruction, which can achieve internal structure analysis at the sub-micron level, accurately detect hidden defects such as pin soldering defects and microcracks. Compared with traditional iterative reconstruction algorithms, the calculation efficiency is higher, meeting the real-time detection requirements of the production line.

[0069] In this embodiment, the 3D structured light imaging unit calculates the absolute phase value through the three-step phase-shifting method , and combines the point cloud registration algorithm to detect pin bending or package deformation, where as follows: ;

[0070] In the formula, I1, I2, and I3 are the intensities of the phase-shifted fringe images.

[0071] The three-step phase-shifting method can resist ambient light interference, with a 3D topography reconstruction accuracy of ±2μm, a single imaging time of ≤10ms, and supports real-time detection of pin bending and package warping in high-speed production lines.

[0072] In this embodiment, the planar vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify misspelled words, blurring, and tilting defects of printed symbols. Among them, the LBP operator is as follows: ;

[0073] In the formula, P is the number of neighborhood sampling points, such as P = 8;

[0074] riu2 is the rotation-invariant uniform pattern;

[0075] R is the sampling radius, that is, g c ;

[0076] c represents the gray value of the central pixel, and its value range is determined by the image bit depth. For example, for an 8-bit gray image g c ∈[0, 255], or for a normalized image: g c ∈[0, 1];

[0077] p is the index of the neighborhood pixel, and its value range is p = 0, 1, 2, …, .

[0078] The rotation invariance in the operator can improve the robustness to printed defects such as tilting and displacement, further improving the classification accuracy, and is faster than the traditional CNN method.

[0079] In this embodiment, the total loss function of the multi-task joint learning model is: ;

[0080] where L cls is the cross-entropy classification loss, L reg is the electrical performance parameter regression loss, L recon is the X-ray voxel reconstruction loss, and λ1, λ2, and λ3 are weight coefficients.

[0081] The multi-task joint learning model can achieve collaborative optimization and defect correlation analysis. During the process, by jointly optimizing classification, regression, and reconstruction tasks, the cross-modal feature consistency is improved, and the generalization ability of the model is enhanced. And the electrical performance prediction error L reg is linked with the appearance defect classification L cls to realize the tracing of the root cause of defects.

[0082] In this embodiment, the hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning by combining the GCN node update formula. Specifically:

[0083] The alignment formula is , where S is the feature map downsampling rate;

[0084] The GCN node update formula is ;

[0085] In the formula is the activation function;

[0086] N(i) is the set of neighbor nodes;

[0087] is the learnable weight matrix of the th layer, which is used for feature transformation;

[0088] is the number of layers of the neural network. For example, =0 is the input layer, =L is the output layer;

[0089] is the feature vector of node j in the th layer;

[0090] i represents the current node, and its value range is all nodes in the graph (which is the node set);

[0091] j represents the neighbor node of node i, and its value range is j∈N(i), where N(i) is the set of adjacent nodes of node i;

[0092] By adopting the ROI alignment and GCN node update formula through the hierarchical defect decision system, it is ensured that the ROI alignment error ≤ 1 pixel, avoiding missing detection of defect areas. Moreover, GCN aggregates neighbor nodes to identify the spatial distribution pattern of defects (such as continuous bending of multiple pins), and the classification accuracy of complex defects can be further improved.

[0093] In addition, this system can also be additionally integrated with a data preprocessing module to perform wavelet transform denoising on thermal imaging data and anisotropic diffusion filtering on X-ray projection data.

[0094] It should be understood that those of ordinary skill in the art can make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the protection scope of the appended claims of the present invention.

Claims

1. A chip packaging multi-dimensional defect intelligent detection system, characterized by: The system includes the following modules: Electrical-thermal linkage detection module: composed of ATE automatic test equipment and infrared thermal imager, synchronously collects the chip's electrical parameter matrix and thermal imaging matrix; Multi-dimensional visual inspection module: including X-ray tomography unit, 3D structured light imaging unit and planar vision unit, which are used to reconstruct the internal structure of the chip, obtain 3D surface topography and extract planar defect features respectively; Multimodal feature fusion module: Based on the cross-modal attention mechanism and multi-task joint learning model, it integrates electro-thermal data, X-ray voxel features and 3D topographic features; Hierarchical defect decision subsystem: Fast R-CNN is used to locate defect areas, a graph convolutional network is used to classify defect types, and a Bayesian inference network is used to integrate multimodal evidence to output the final defect determination result. Among them, the electrical parameter matrix is ; The thermal imaging matrix is ; Among them, V i is the voltage sequence, I j is the current value, T k is the ambient temperature parameter, H l is the humidity parameter, i, j, k and l are all positive integers; is the spatiotemporal temperature distribution; The cross-modal attention mechanism is: ； Among them, Kj and Qi are the query vector and key vector within the image sampling range, respectively, and d is the feature dimension; The electric-thermal linkage detection module further includes: Electro-thermal coupling analysis model, through partial differential equations Establish the correlation between current density J and heat diffusion, used to infer internal voids or heat dissipation defects; in, is the Laplace operator, is the heat generation rate per unit volume of the chip, m is the thermal conductivity of the packaging material, is the thermal diffusivity; The X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip to detect pin solder joints or internal microcracks. The reconstruction formula is: ； where μ(r) is the linear attenuation coefficient of the voxel, D is the distance from the X-ray source to the detector, s is the coordinate on the detector plane, r is the voxel position vector in three-dimensional space, is the projection angle, is the projection value measured at the detector position s at the angle and h(s’ - s) is the ramp filter.

2. The chip packaging multi-dimensional defect intelligent detection system according to claim 1, characterized in that: The 3D structured light imaging unit calculates the absolute phase value ϕ(x, y) through a three-step phase shift method and combines it with a point cloud registration algorithm to detect lead bending or package deformation, where ϕ(x, y) is as follows: ； Where I1, I2, and I3 are the phase-shifted fringe image intensities.

3. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, wherein The plane vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify typos, blurs, and tilt defects of printed symbols, wherein the LBP operator is as follows: ； Where P is the number of neighborhood sampling points, riu2 is the rotation-invariant uniform pattern, R is the sampling radius, c represents the grayscale value of the center pixel, and p is the index of the neighborhood pixel, ranging from p=0,1,2,…,p-1.

4. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, wherein, The total loss function of the multi-task joint learning model is: ； Among them, L cls is the cross-entropy classification loss, and L reg is the regression loss of electrical performance parameters, and L recon is the X-ray voxel reconstruction loss, where λ1, λ2, and λ3 are weight coefficients.

5. The chip packaging multi-dimensional defect intelligent detection system according to claim 1, characterized in that: The hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning in combination with the GCN node update formula.

6. The chip package multi-dimensional defect intelligent detection system according to claim 5, characterized in that, in, The alignment formula is , where S is the feature map downsampling rate; The GCN node update formula is ; where is the activation function, N(i) is the set of neighboring nodes, is the learnable weight matrix of the layer, is the feature vector of node j in the layer, i represents the current node, and j represents the neighbor node of node i.

Images