Chip packaging multi-dimensional defect intelligent detection system

Abstract

The invention belongs to the field of visual inspection, and particularly relates to a chip packaging multi-dimensional defect intelligent detection system which comprises a plurality of modules and a subsystem. Wherein the electric-thermal linkage detection module consists of ATE (automatic test equipment) and a thermal infrared imager, and is used for acquiring an electric parameter matrix and a thermal imaging matrix of a chip; the multi-dimensional vision detection module comprises an X-ray tomography unit, a 3D structured light imaging unit and a plane vision unit; the reconstruction module is used for reconstructing an internal structure of the chip, acquiring 3D surface topography and extracting plane defect features; the multi-modal feature fusion module is based on a cross-modal attention mechanism and a multi-task joint learning model to fuse electric-thermal data, X-ray voxel features and 3D morphology features; the hierarchical defect decision-making subsystem locates a defect area through FastR-CNN, classifies defect types through a graph convolutional network, and outputs a final defect judgment result based on Bayesian reasoning network by integrating multi-modal evidence; through multi-modal data linkage, joint detection of cavities, heat dissipation and appearance defects can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of visual detection, and particularly relates to an intelligent multi-dimensional defect detection system for chip packaging. Background Art

[0002] With the continuous improvement of integrated circuit manufacturing technology, chip packaging defect detection has become a key link to ensure product quality. However, the traditional detection methods have the following problems: 1. Single detection dimension: The existing technologies mostly adopt the method of separating physical detection (appearance defects) from electrical performance detection, resulting in low detection efficiency and inability to detect cross-modal related defects, such as abnormal heat dissipation caused by internal voids; 2. Insufficient internal defect detection: Conventional X-ray or optical detection is difficult to accurately identify internal structure defects of the package, such as micron-level voids and pin soldering defects, and cannot perform linkage analysis with electro-thermal characteristics; 3. Poor algorithm adaptability: Existing visual detection algorithms, such as traditional LBP feature extraction, have insufficient classification accuracy for complex printing defects (blurred, tilted, broken words) and lack the ability of multi-modal data fusion.

[0003] Therefore, we propose a solution to make up for the deficiencies of the traditional solution to meet the market demand. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent multi-dimensional defect detection system for chip packaging to solve the problems raised in the background art.

[0005] To achieve the above purpose, the present invention provides the following technical solution: An intelligent multi-dimensional defect detection system for chip packaging, the system includes the following modules: Electro-thermal linkage detection module: Composed of an ATE automatic test equipment and an infrared thermal imager, synchronously collecting the electrical parameter matrix and thermal imaging matrix of the chip; Multi-dimensional visual detection module: Including an X-ray tomography unit, a 3D structured light imaging unit, and a planar vision unit, respectively used for reconstructing the internal structure of the chip, obtaining the 3D surface topography, and extracting planar defect features; Multi-modal feature fusion module: Based on a cross-modal attention mechanism and a multi-task joint learning model, fusing electro-thermal data, X-ray voxel features, and 3D topography features; Hierarchical defect decision subsystem: Locating the defect area through FastR-CNN, classifying the defect type by a graph convolutional network, and outputting the final defect determination result based on a Bayesian inference network to synthesize multi-modal evidence.

[0006] For the intelligent multi-dimensional defect detection system for chip packaging of the present invention, wherein, the electrical parameter matrix is ; The thermal imaging matrix is ; where V i is the voltage sequence, I j is the current value, T k is the ambient temperature parameter, H l is the humidity parameter, and i, j, k, and l are all positive integers; is the spatio-temporal temperature distribution.

[0007] For the multi-dimensional defect intelligent detection system of chip packaging according to the present invention, wherein the cross-modal attention mechanism is: ; where Kj and Qi are the query vector and the key vector within the image sampling range respectively, and d is the feature dimension.

[0008] For the multi-dimensional defect intelligent detection system of chip packaging according to the present invention, wherein the electro-thermal coupling detection module further includes: An electro-thermal coupling analysis model, which establishes the correlation between the current density J and heat diffusion through the partial differential equation to infer internal voids or heat dissipation defects; where is the Laplace operator, is the heat generation rate per unit volume of the chip, m is the thermal conductivity of the packaging material, is the thermal diffusion coefficient.

[0009] For the multi-dimensional defect intelligent detection system of chip packaging according to the present invention, wherein the X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip for detecting pin soldering defects or internal micro-cracks, and its reconstruction formula is: ; where μ(r) is the linear attenuation coefficient of the voxel, D is the distance from the X-ray source to the detector, s is the coordinate on the detector plane, r is the voxel position vector in the three-dimensional space, is the projection angle, is at the angle the projection value measured at the detector position s under, and h(s’-s) is the ramp filter.

[0010] For the multi-dimensional defect intelligent detection system of chip packaging according to the present invention, wherein the 3D structured light imaging unit calculates the absolute phase value by the three-step phase-shift method, combined with the point cloud registration algorithm to detect pin bending or package deformation, where is as follows: In the formula, I1, I2, and I3 are the intensities of the phase-shifted fringe images.

[0011] The intelligent multi-dimensional defect detection system for chip packaging according to the present invention, wherein the planar vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify misspelled words, blurring, and tilting defects of printed symbols. The LBP operator is as follows: ; In the formula, P is the number of neighborhood sampling points, riu2 is the rotation-invariant uniform pattern, R is the sampling radius, c represents the gray value of the central pixel, and p is the index of the neighborhood pixel, with the value range p = 0, 1, 2, …, .

[0012] The intelligent multi-dimensional defect detection system for chip packaging according to the present invention, wherein the total loss function of the multi-task joint learning model is: ; where L cls is the cross-entropy classification loss, L reg is the regression loss of electrical performance parameters, L recon is the X-ray voxel reconstruction loss, and λ1, λ2, and λ3 are weight coefficients.

[0013] The intelligent multi-dimensional defect detection system for chip packaging according to the present invention, wherein the hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning by combining the GCN node update formula.

[0014] The intelligent multi-dimensional defect detection system for chip packaging according to the present invention, wherein the alignment formula is , where S is the feature map downsampling rate; The GCN node update formula is ; in the formula is the activation function, N(i) is the set of neighborhood nodes, is the learnable weight matrix of the th layer, is the number of layers of the neural network, is the th layer, and the feature vector of node j, where i represents the current node and j represents the neighbor node of node i.

[0015] Compared with the prior art, the beneficial effects of the present invention are: Through the linkage of electro-thermal-X-ray multi-modal data, multi-dimensional joint detection of internal voids, heat dissipation defects, and appearance defects is realized. In particular, by combining electrical parameters (V, I) and thermal imaging data, real defects can be finely distinguished from noise interference, greatly reducing the false alarm rate and achieving a higher comprehensive detection rate; moreover, the cross-modal attention mechanism is used to improve the feature fusion efficiency, and the false alarm rate can be reduced synchronously; in addition, the system supports real-time detection and is applicable to high-precision semiconductor packaging production lines. Brief Description of the Drawings

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 It is a schematic structural diagram of the present invention. Specific embodiments

[0018] The terms "first", "second", "third", "fourth", etc. in the description and claims of the present invention and the drawings thereof are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0019] Referring to "embodiments" herein means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0020] "Plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0021] Moreover, the terms indicating directions, such as "upper", "lower", "left", "right", "upper end", "lower end", "longitudinal", etc., are all referenced based on the attitude position of the device or equipment described in this solution during normal use.

[0022] In order to make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the protection scope of the present invention.

[0023] This embodiment discloses a multi-dimensional defect intelligent detection system for chip packaging as shown in Figure 1 , and this system includes the following modules: Electro-thermal linkage detection module: Composed of an ATE automatic test equipment and an infrared thermal imager, it synchronously acquires the electrical parameter matrix and thermal imaging matrix of the chip, and can dynamically adjust the ATE test parameters according to the electrical performance detection results. The voltage adjustment formula is V new =V base +ΔV*sign(I dev ), where ; Multi-dimensional vision detection module: Includes an X-ray tomography imaging unit, a 3D structured light imaging unit, and a planar vision unit, which are respectively used to reconstruct the internal structure of the chip, obtain the 3D surface topography, and extract planar defect features; among them, the resolution of the X-ray microfocus source ≤5μm, the thermal sensitivity of the thermal imager ≤20mK, the Z-axis accuracy of the 3D structured light camera ±2μm, and the planar vision unit uses a global shutter CMOS camera with a frame rate ≥500fps / 5MP resolution and is equipped with a coaxial light source to eliminate reflection; Multi-modal feature fusion module: Based on the cross-modal attention mechanism and multi-task joint learning model, it fuses electro-thermal data, X-ray voxel features, and 3D topography features; Hierarchical defect decision subsystem: Locates the defect area through FastR-CNN, classifies the defect type through a graph convolutional network, and outputs the final defect determination result based on the Bayesian inference network by synthesizing multi-modal evidence.

[0024] Through three processing modules and one subsystem, an electro-thermal-X-ray multi-modal data linkage is formed to realize the joint detection of internal voids, heat dissipation defects, and appearance defects. In particular, by combining electrical parameters and thermal imaging data, real defects and noise interference can be finely distinguished, greatly reducing the false alarm rate and achieving a higher comprehensive detection rate; moreover, the cross-modal attention mechanism is used to improve the feature fusion efficiency, and the false alarm rate can be reduced synchronously; in addition, the system supports real-time detection and is applicable to high-precision semiconductor packaging production lines.

[0025] In this embodiment, the electrical parameter matrix is ; The thermal imaging matrix is ; Among them, V i is the voltage sequence, I j is the current value, T k is the environmental temperature parameter, H l is the humidity parameter, and i, j, k, and l are all positive integers; is the spatio-temporal temperature distribution.

[0026] In this embodiment, the cross-modal attention mechanism is as follows: ; where Q i is the query vector from the source modality (such as thermal imaging data), and K j is the key vector from the target modality (such as X-ray data), and represents the dot product (inner product) of the two vectors, used to measure their similarity. The role of T in the formula is to convert the row vector Q i into a column vector for matrix multiplication operation, and d is the feature dimension.

[0027] In this embodiment, the electro-thermal linkage detection module further includes: An electro-thermal coupling analysis model that establishes the correlation between the current density J and heat diffusion through partial differential equations ; Establish a physical model of current density and heat diffusion, which can infer abnormal heat dissipation caused by voids inside the package, improve the internal defect detection rate. Especially by combining electrical parameters (V, I) and thermal imaging data, it can finely distinguish real defects from noise interference, greatly reducing the false alarm rate; where is the Laplace operator, is the heat generation rate per unit volume of the chip, calculated by the product of VIt; m is the thermal conductivity of the packaging material, is the thermal diffusion coefficient, , is the density, is the specific heat capacity.

[0028] Specifically, when the current density J passes through the chip conductor, the Joule heat power density is: In the formula, J = I / A (current density, I is the current, A is the cross-sectional area of the conductor); σ is the electrical conductivity of the material; the current I and voltage V obtained through electrical performance detection, combined with the geometric parameters of the conductor, can calculate J and ;

[0029] In this embodiment, the X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip for detecting pin soldering defects or internal microcracks, and its reconstruction formula is: ; where μ(r) is the linear attenuation coefficient of the voxel; D is the distance from the X-ray source to the detector; s is the coordinate on the detector plane, representing the specific position on the X-ray detector; r is the voxel position vector in three-dimensional space, that is, the spatial coordinates (x, y, z) of the point to be reconstructed; is the projection angle, representing the angle by which the X-ray source and the detector rotate around the object (range 0 ≤ <2π); is the projection value measured at the detector position s at the angle , i.e., the X-ray attenuation integral; h(s’ - s) is the ramp filter; The meaning of the formula is to reconstruct the linear attenuation coefficient μ(r) at the voxel r by integrating the data of all projection angles and combining the geometric correction factor .

[0030] X-ray tomography uses the FDK algorithm for reconstruction, which can achieve the analysis of internal structures at the sub-micron level, accurately detect hidden defects such as pin soldering defects and micro-cracks. Compared with traditional iterative reconstruction algorithms, it has higher computational efficiency and meets the real-time detection requirements of the production line.

[0031] In this embodiment, the 3D structured light imaging unit calculates the absolute phase value through the three-step phase-shift method , and combines the point cloud registration algorithm to detect pin bending or package deformation, where is as follows: ; In the formula, I1, I2, and I3 are the intensities of the phase-shifted fringe images.

[0032] The three-step phase-shift method can resist ambient light interference. The 3D topography reconstruction accuracy reaches ±2μm, and the single-shot imaging time ≤ 10ms, supporting the real-time detection of pin bending and package warping in high-speed production lines.

[0033] In this embodiment, the planar vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify misspelled words, blurring, and tilting defects of printed symbols. Among them, the LBP operator is as follows: ; In the formula, P is the number of neighborhood sampling points, such as P = 8; riu2 is the rotation-invariant uniform pattern; R is the sampling radius, i.e., g c ; c represents the gray value of the central pixel, and its value range is determined by the image bit depth. For example, for an 8-bit gray image g c ∈[0, 255], or for a normalized image: g c ∈[0, 1]; p is the index of the neighborhood pixel, and its value range is p = 0, 1, 2,..., .

[0034] The rotational invariance in the operator can enhance the robustness against printing defects such as tilt and displacement, further improve the classification accuracy, and is faster than the traditional CNN method.

[0035] In this embodiment, the total loss function of the multi-task joint learning model is: ; where L cls is the cross-entropy classification loss, L reg is the regression loss of electrical performance parameters, L recon is the X-ray voxel reconstruction loss, and λ1, λ2, and λ3 are weight coefficients.

[0036] The multi-task joint learning model can achieve collaborative optimization and defect correlation analysis. During the process, by jointly optimizing the classification, regression, and reconstruction tasks, it improves the cross-modal feature consistency and enhances the model generalization ability; and the electrical performance prediction error L reg is linked with the appearance defect classification L cls to realize the traceability of the defect root cause.

[0037] In this embodiment, the hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning by combining the GCN node update formula. Specifically: The alignment formula is , where S is the feature map downsampling rate; The GCN node update formula is ; where is the activation function; N(i) is the set of neighbor nodes; is the learnable weight matrix of the th layer for feature transformation; is the number of layers of the neural network. For example, =0 is the input layer, =L is the output layer; is the feature vector of node j in the th layer; i represents the current node, and its value range is all nodes in the graph (the set of nodes); j represents the neighbor node of node i, and its value range is j∈N(i), where N(i) is the set of adjacent nodes of node i; By adopting the ROI alignment and GCN node update formula through the hierarchical defect decision system, it ensures that the ROI alignment error ≤ 1 pixel and avoids missing detection of the defect area; moreover, GCN aggregates neighbor nodes to identify the spatial distribution pattern of defects (such as continuous bending of multiple pins), and the classification accuracy of complex defects can be further improved.

[0038] In addition, the system can also be additionally integrated with a data preprocessing module to perform wavelet transform denoising on the thermal imaging data and anisotropic diffusion filtering on the X-ray projection data.

[0039] It should be understood that those of ordinary skill in the art can make improvements or transformations according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. An intelligent multi-dimensional defect detection system for chip packaging, characterized in that, The system includes the following modules: The electro-thermal linkage detection module: It consists of an ATE automatic test equipment and an infrared thermal imager, and synchronously acquires the electrical parameter matrix and thermal imaging matrix of the chip; The multi-dimensional vision detection module: It includes an X-ray tomography imaging unit, a 3D structured light imaging unit and a planar vision unit, which are respectively used to reconstruct the internal structure of the chip, obtain the 3D surface topography and extract the planar defect features; The multi-modal feature fusion module: Based on the cross-modal attention mechanism and the multi-task joint learning model, it fuses the electro-thermal data, X-ray voxel features and 3D topography features; The hierarchical defect decision subsystem: It locates the defect area through FastR-CNN, classifies the defect types by the graph convolutional network, and outputs the final defect determination result based on the Bayesian inference network by synthesizing multi-modal evidence.

2. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, wherein The electrical parameter matrix is ; The thermal imaging matrix is ; Among them, V i is the voltage sequence, I j is the current value, T k is the ambient temperature parameter, H l is the humidity parameter, and i, j, k, and l are all positive integers; is the spatio-temporal temperature distribution.

3. The intelligent multi-dimensional defect detection system for chip packaging according to claim 1, wherein, The cross-modal attention mechanism is as follows: ; Among them, Kj and Qi are the query vector and the key vector within the image sampling range respectively, and d is the feature dimension.

4. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, characterized in that The electro-thermal linkage detection module further includes: Electro-thermal coupling analysis model, through partial differential equations Establish the correlation between current density J and heat diffusion, used to infer internal voids or heat dissipation defects; wherein, is the Laplace operator, is the heat generation rate per unit volume of the chip, m is the thermal conductivity of the packaging material, is the thermal diffusivity.

5. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, characterized in that, The X-ray tomography unit uses the FDK algorithm to reconstruct the internal structure of the chip for detecting pin soldering defects or internal microcracks, and its reconstruction formula is as follows: ; where μ(r) is the linear attenuation coefficient of the voxel, D is the distance from the X-ray source to the detector, s is the coordinate on the detector plane, r is the voxel position vector in three-dimensional space, is the projection angle, is the projection value measured at the detector position s at the angle and h(s’ - s) is the ramp filter.

6. The chip package multi-dimensional defect intelligent detection system according to claim 1, wherein The 3D structured light imaging unit calculates the absolute phase value through the three-step phase-shifting method , and combines the point cloud registration algorithm to detect pin bending or package deformation, where as follows: ; In the formula, I1, I2, and I3 are the intensities of the phase-shifted fringe images.

7. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, wherein, The planar vision unit uses an improved rotation-invariant uniform LBP operator combined with a BPNN classifier to identify the misspelling, blur, and tilt defects of printed symbols. Among them, the LBP operator is as follows: ； Wherein, P is the number of neighborhood sampling points, riu2 is the rotation-invariant uniform pattern, R is the sampling radius, c represents the gray value of the central pixel, p is the index of the neighborhood pixel, and the value range is p = 0, 1, 2, …, .

8. The intelligent multi-dimensional defect detection system for chip packaging according to claim 1, characterized in that The total loss function of the multi-task joint learning model is as follows: ; Among them, L cls is the cross-entropy classification loss, L reg is the regression loss of electrical performance parameters, L recon is the X-ray voxel reconstruction loss, and λ1, λ2, and λ3 are weight coefficients.

9. The multi-dimensional defect intelligent detection system for chip packaging according to claim 1, wherein The hierarchical defect decision subsystem locates the defect area through the ROI alignment formula and realizes defect relationship reasoning by combining the GCN node update formula.

10. The multi-dimensional defect intelligent detection system for chip packaging according to claim 9, wherein Among them, The alignment formula is , where S is the downsampling rate of the feature map; The GCN node update formula is ; where is the activation function, N(i) is the set of neighboring nodes, is the learnable weight matrix of the layer, is the feature vector of node j in the layer, i represents the current node, and j represents the neighbor node of node i.

Images