A chip appearance defect detection method based on laser holographic imaging technology

Through the combination of laser holographic imaging technology and machine learning models, efficient chip appearance defect detection is achieved, solving the problems of traditional low detection efficiency and high error detection rate, and achieving automated and high reliability chip detection.

CN114494139BActive Publication Date: 2024-11-22XIDIAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111624835.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-28
Publication Date
2024-11-22
Estimated Expiration
2041-12-28

AI Technical Summary

Technical Problem

The existing chips have low appearance defect detection efficiency and cannot meet the requirements of high-speed production, which can easily lead to missed inspections and missed inspections, and the traditional detection methods cannot adapt to chip miniaturization.

Method used

Using a detection method based on laser holographic imaging technology, a hologram is formed through a laser, and the image processing module performs numerical transformation and three-dimensional morphology reconstruction, combined with a support vector machine and a three-dimensional convolutional neural network for chip defect detection, and automatically uses a parallel connecting rod robot. Remove defective chips.

Benefits of technology

It improves detection efficiency, reduces the false detection rate, realizes automated detection, improves the reliability and production efficiency of detection results, and solves the problems of manual detection subjectivity and loss of two-dimensional image information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114494139B_ABST
    Figure CN114494139B_ABST
Patent Text Reader

Abstract

The present invention discloses a chip appearance defect detection method based on laser holographic imaging technology, which improves the low detection efficiency of chip appearance defect detection in the prior art. The invention contains the following steps: step 1, start the laser to form a hologram, the image acquisition module receives the hologram and sends it to the image processing module; step 2, perform numerical transformation on the hologram to obtain a digital hologram, and reconstruct the three-dimensional morphology through image processing; step 3, judge whether the chip is abnormal according to the classification result of the support vector machine; step 4, if the judgment result is abnormal, the parallel linkage robot removes the defective chip, and if the result is normal, the test of the next chip is ended. This technology solves the problem of unstable quality caused by changes in subjective standards of manual judgment, reduces detection costs, and improves detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of chip packaging appearance defect detection, and in particular to a chip appearance defect detection method based on laser holographic imaging technology. Background Art

[0002] Semiconductor chips are widely used in various fields and have become a top priority for economic development and national information security. Semiconductor chips shipped out of the factory inevitably have various defects on the surface of the chip, which directly affects the operating performance and life of the chip. Traditional chip appearance defect detection uses manual measurement and human eye observation and recognition methods. Manual detection is prone to visual fatigue, leading to false detection and misjudgment, and manual detection requires a lot of manpower and material resources. Therefore, it is necessary to continuously promote the development of chip appearance defect detection technology. Most of the solutions currently proposed require the acquisition of two-dimensional image signals of the chip to be inspected, and then obtain the detection results through complex machine vision algorithm processing. When the three-dimensional product is projected as a two-dimensional image, the information of the depth and invisible part will be lost, resulting in possible errors in the detection results.

[0003] The existing chip appearance defect detection technology has low detection efficiency, cannot meet the requirements of high-speed production, and is prone to missed detection and false detection. At the same time, with the miniaturization of chip size, traditional detection methods cannot be implemented. Therefore, at this stage, it is necessary to design a chip appearance defect detection system to solve the above problems. Summary of the invention

[0004] The present invention improves the problem of low detection efficiency and inability to meet high-speed production in the prior art of chip appearance defect detection, and provides a chip appearance defect detection method based on laser holographic imaging technology which has low detection efficiency and can easily meet high-speed production.

[0005] The technical solution of the present invention is to provide a chip appearance defect detection method based on laser holographic imaging technology with the following steps:

[0006] Step 1, start the laser to form a hologram, the image acquisition module receives the hologram and sends it to the image processing module;

[0007] Step 2, numerically transforming the hologram to obtain a digital hologram, and reconstructing the three-dimensional morphology through image processing;

[0008] Step 3: Determine whether the chip is abnormal based on the classification result of the support vector machine;

[0009] Step 4: If the result is abnormal, the parallel linkage robot removes the defective chip; if the result is normal, the test of the next chip ends.

[0010] Preferably, the laser in step 1 is a He-Ne laser, and the input He-Ne laser is divided into two beams by a polarization beam splitter PBS, one beam of laser is expanded and collimated by a beam expansion and collimation system BE1 and then passes through an object to form an object light; the other beam of laser is expanded and collimated by a beam expansion and collimation system BE2 and then serves as a reference light, and after the object light and the reference light pass through the beam splitter BS, interference fringes are generated at the CCD target surface to form a hologram, and data is collected by an image detection and acquisition module and transmitted to an image processing module by an image acquisition card.

[0011] The complex amplitude distribution of the object light wave irradiated to the CCD target surface is:

[0012] The complex amplitude distribution of the reference light wave irradiated to the CCD target surface is:

[0013] ox,y and rx,y represent the intensity distribution of object light and reference light respectively. and Represent the phase distribution of object light and reference light respectively. The total light field of interference fringes reaching the CCD target surface is:

[0014]

[0015] , the CCD target surface size is Lx×Ly, the pixel size is Δx, Δy, the number of pixels is Nx=Lx / Δx, Ny=Ly / Δy, then the discrete hologram recorded on the CCD target surface is expressed as:

[0016]

[0017] Preferably, after the hologram in step 2 is read into the computer, the computing unit of the computer simulates the optical diffraction process in the CPU according to the principle of Fresnel diffraction integral to digitally reproduce the hologram, which mainly includes the following three steps:

[0018] Step 2.1, set the wavelength of the reproduced light to E R (x, y), use this light wave to simulate the illumination of the hologram. According to the Rayleigh-Sommerfeld diffraction formula, the complex amplitude at the reconstruction plane with a reconstruction distance of d is:

[0019]

[0020]

[0021] ρ represents the distance from the holographic target surface to the corresponding point on the reconstruction plane, λ is the wavelength, and the discretized expression obtained by sampling Uξ',η' is:

[0022]

[0023]

[0024] Δξ', Δη' represent the sampling interval of the reconstructed image plane, and the intensity and phase distribution of the reconstructed image plane are respectively:

[0025] Im,n=|Um,n| 2

[0026]

[0027] Step 2.2: Substituting into Uξ',η' we get:

[0028]

[0029] Transforming the position of the exponential function and following the meaning of Fourier transform, we get:

[0030]

[0031] Step 2.3, use fast Fourier transform to calculate the result of the previous step and use the row-column point-by-point method to perform phase unwrapping to obtain the reconstructed image.

[0032] Preferably, in step 2, when improving the spatial resolution of the hologram reconstruction image, image fusion is combined with an image filtering algorithm, including the following four steps:

[0033] Step 2-1, extracting a certain number of sub-holograms of equal size from a hologram;

[0034] Step 2-2, digitally reproducing these sub-holograms to obtain a certain number of reconstructed images;

[0035] Step 2-3, averaging the grayscale values ​​of these reconstructed images to obtain a reconstructed image with preliminary noise reduction;

[0036] Step 2-4: Perform non-local mean filtering on the reconstructed image to obtain the final reconstructed image, that is, the three-dimensional morphology of the chip to be inspected.

[0037] Preferably, the image processing and three-dimensional shape reconstruction in step 2 are trained on a training set using a defined three-dimensional convolutional neural network structure to determine a set of optimal parameters, including the following three steps:

[0038] Step 2-1), normalize and enhance the training set data, the data enhancement includes flipping and rotation operations;

[0039] Step 2-2), for the input data of the chip 3D morphology, first divide it into 4*4*4 sub-blocks, and extract the chip defect features in each sub-block;

[0040] Step 2-3) Input the training set data into the network, perform forward propagation, calculate the error loss between the defect classification result and the label value, and use the back propagation algorithm to update the network parameters. Repeat multiple iterations until the network converges to the local optimum or stops training when a certain number of training times is reached. This means that a set of optimal network parameters has been found.

[0041] Step 2-4), input the reconstructed three-dimensional morphology data of the chip to be inspected into the three-dimensional convolutional neural network for feature extraction, input the features extracted by the three-dimensional convolutional neural network into the support vector machine model for classification, and realize chip defect detection based on the classification results of the support vector machine.

[0042] Preferably, the three-dimensional convolutional neural network includes 4 three-dimensional convolutional layers, 2 pooling layers, 1 Dropout layer to prevent overfitting, 1 Flatten layer to realize one-dimensionalization of multi-dimensional input features and 2 fully connected layers.

[0043] Preferably, the defect types detected by the chip appearance detection are scratches on the chip surface, pits on the chip surface, chip size defects, missing pins or tilted pins.

[0044] Compared with the prior art, the chip appearance defect detection method based on laser holographic imaging technology of the present invention has the following advantages: the present invention has a simple structure, reasonable design, and is easy to implement. Since the machine can automatically detect according to a unified standard, the quality instability problem caused by changes in subjective standards of manual judgment is solved, the detection cost is reduced, and the detection efficiency is improved.

[0045] Traditional detection methods are based on the two-dimensional image data of the chip to be tested. When a three-dimensional product is projected as a two-dimensional image, the information about the depth and invisible parts will be lost. The detection method is based on the three-dimensional imaging data of the chip to be tested, so the reliability of the detection results is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the structure of the present invention;

[0047] Figure 2 It is a schematic diagram of the work flow structure of the present invention;

[0048] Figure 3 It is a schematic diagram of the network structure of the three-dimensional convolutional neural network in the present invention.

[0049] Figure 1 Laser is He-Ne laser, M is reflector, PBS is polarization beam splitter, BS is beam splitter, BE is collimation and beam expansion system, CCD is CCD camera, and computer is computer. DETAILED DESCRIPTION

[0050] The following is a further explanation of the chip appearance defect detection method based on laser holographic imaging technology of the present invention in conjunction with the accompanying drawings and specific implementation methods: As shown in the figure, the technical solution adopted in this embodiment is: a chip appearance defect detection system based on laser holographic imaging technology, including a He-Ne laser, a holographic optical path module, an image detection and acquisition module, an image data processing module and a defective chip sorting module; the He-Ne laser is used to emit a laser beam; the holographic optical path module is used to split the laser beam into two beams through a beam splitter, one laser beam passes through an object after passing through a beam expansion and collimation system to form an object light; the other laser beam passes through the beam expansion and collimation system as a reference light, and after the object light and the reference light pass through the beam splitter, interference fringes are generated at the target surface of the image detection and acquisition module to form a hologram, and the image detection and acquisition module performs data acquisition.

[0051] The image detection and acquisition module usually uses a CCD device to receive the interference fringe field of the object light and the reference light, and the image acquisition card transmits it to the image processing module to record the digital hologram.

[0052] The image processing module is a computer. The computer's computing unit uses the Fresnel diffraction principle to simulate the optical diffraction process in the CPU to achieve digital reproduction of the hologram. Finally, the hologram reproduced using the basic principles of digital images is further processed to remove digital interference and reconstruct the three-dimensional morphology of the chip to be inspected. The reconstructed three-dimensional data of the chip to be inspected is input into the three-dimensional convolutional neural network to extract features, and the extracted feature vectors are then input into the support vector machine model for classification. Based on the classification results of the support vector machine, the chip appearance detection can be achieved, such as scratches, pits and other defects on the chip surface, chip size defects and pin detection such as missing pins and pin tilt.

[0053] The defective chip sorting module is a parallel-link robot that receives instructions from the image processing module and removes defective chips from the assembly line in a timely manner.

[0054] The system structure of the present invention is further described in detail below in conjunction with the accompanying drawings. The technical solutions provided below are only illustrative and do not constitute a particular limitation to the present invention.

[0055] like Figure 1 As shown, the input He-Ne laser is split into two beams by the polarization beam splitter PBS. One beam of laser is expanded and collimated by the beam expansion and collimation system BE1 and then passes through the object to form object light; the other beam of laser is expanded and collimated by the beam expansion and collimation system BE2 and then serves as reference light. After the object light and the reference light pass through the beam splitter BS, interference fringes are generated on the CCD target surface to form a hologram. The image detection and acquisition module collects data and the image acquisition card transmits it to the image processing module.

[0056] The complex amplitude distribution of the object light wave irradiated to the CCD target surface is:

[0057]

[0058] The complex amplitude distribution of the reference light wave irradiated to the CCD target surface is:

[0059]

[0060] ox,y and rx,y represent the intensity distribution of object light and reference light respectively. and Represent the phase distribution of object light and reference light respectively. The total light field of interference fringes reaching the CCD target surface is:

[0061]

[0062] The CCD target surface size is Lx×Ly, the pixel size is Δx, Δy, the number of pixels is Nx=Lx / Δx, Ny=Ly / Δy, then the discrete hologram recorded on the CCD target surface can be expressed as:

[0063]

[0064] After the hologram is read into the computer, it is digitally reproduced according to the principle of Fresnel diffraction integral, which mainly includes the following three steps.

[0065] Step 2.1, set the wavelength of the reproduced light to E R (x, y), use this light wave to simulate the illumination of the hologram. According to the Rayleigh-Sommerfeld diffraction formula, the complex amplitude at the reconstruction plane with a reconstruction distance of d is:

[0066]

[0067]

[0068] ρ represents the distance from the holographic target surface to the corresponding point on the reconstruction plane, and λ is the wavelength. Sampling Uξ',η' can obtain the discretized expression:

[0069]

[0070]

[0071] Δξ', Δη' represent the sampling interval of the reconstructed image plane, and the intensity and phase distribution of the reconstructed image plane are respectively:

[0072] Im,n=|Um,n| 2

[0073]

[0074] Step 2.2: Substituting into Uξ',η' we get:

[0075]

[0076] , transform the position of the exponential function and according to the meaning of Fourier transform, we can get:

[0077]

[0078] Step 2.3, use fast Fourier transform to calculate the result of the previous step and use the row-column point-by-point method to perform phase unwrapping to obtain the reconstructed image.

[0079] In order to improve the spatial resolution of the holographic reconstruction image, the present invention proposes an image fusion combined with an image filtering algorithm, which mainly includes the following four steps.

[0080] Step 2-1: Extract a certain number (n=4) of sub-holograms of equal size from a hologram.

[0081] Step 2-2: digitally reconstruct these sub-holograms to obtain a certain number of reconstructed images.

[0082] Step 2-3, averaging the grayscale values ​​of these reconstructed images to obtain a reconstructed image with preliminary noise reduction.

[0083] Step 2-4: Perform non-local mean filtering (NLM) on the reconstructed image to obtain the final reconstructed image, that is, the three-dimensional morphology of the chip to be inspected.

[0084] use Figure 3 That is, the defined three-dimensional convolutional neural network structure is trained on the training set to determine a set of optimal parameters, which mainly includes the following three steps.

[0085] Step 2-1) normalize and enhance the training set data, wherein the data enhancement includes flipping and rotation operations.

[0086] Step 2-2), for the input data of the chip 3D morphology, first divide it into 4*4*4 sub-blocks, and extract the chip defect features in each sub-block.

[0087] Step 2-3) Input the training set data into the network, perform forward propagation, calculate the error loss between the defect classification result and the label value, and use the back propagation algorithm to update the network parameters. Repeat multiple iterations until the network converges to the local optimum or stops training when a certain number of training times is reached. This means that a set of optimal network parameters has been found.

[0088] The reconstructed three-dimensional morphology data of the chip to be inspected is input into a three-dimensional convolutional neural network for feature extraction.

[0089] The features extracted by the three-dimensional convolutional neural network are input into the support vector machine model for classification, and chip defect detection can be achieved based on the classification results of the support vector machine.

[0090] The three-dimensional convolutional neural network includes four three-dimensional convolutional layers, two pooling layers, one Dropout layer to prevent overfitting, one Flatten layer to realize one-dimensionalization of multi-dimensional input features and two fully connected layers.

[0091] The defective chip sorting module receives instructions from the image processing module and removes the defective chips from the assembly line in a timely manner.

[0092] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A chip appearance defect detection method based on laser holographic imaging technology, characterized in that: Contains the following steps, Step 1, start the laser to form a hologram, the image acquisition module receives the hologram and sends it to the image processing module; Step 2, numerically transform the hologram to obtain a digital hologram, and reconstruct the three-dimensional morphology through image processing; when improving the spatial resolution of the hologram reconstruction image, image fusion is combined with image filtering algorithm, including the following four steps: Step 2-1, extracting a certain number of sub-holograms of equal size from a hologram; Step 2-2, digitally reproducing these sub-holograms to obtain a certain number of reconstructed images; Step 2-3, averaging the grayscale values ​​of these reconstructed images to obtain a reconstructed image with preliminary noise reduction; Step 2-4, performing non-local mean filtering on the reconstructed image to obtain the final reconstructed image, i.e., the three-dimensional morphology of the chip to be inspected; Image processing and reconstruction of three-dimensional morphology use the defined three-dimensional convolutional neural network structure to train on the training set and determine a set of optimal parameters, including the following three steps: Step 2-1), normalize and enhance the training set data, the data enhancement includes flipping and rotation operations; Step 2-2), for the input data of the chip 3D morphology, first divide it into 4*4*4 sub-blocks, and extract the chip defect features in each sub-block; the defect types detected by the chip appearance are scratches on the chip surface, pits on the chip surface, chip size defects, pin missing or pin tilt; Step 2-3) Input the training set data into the network, perform forward propagation, calculate the error loss between the defect classification result and the label value, and use the back propagation algorithm to update the network parameters. Repeat multiple iterations until the network converges to the local optimum or stops training when a certain number of training times is reached. This means that a set of optimal network parameters has been found. Step 2-4), input the reconstructed three-dimensional morphology data of the chip to be inspected into a three-dimensional convolutional neural network for feature extraction, input the features extracted by the three-dimensional convolutional neural network into a support vector machine model for classification, and implement chip defect detection based on the classification results of the support vector machine; the three-dimensional convolutional neural network includes 4 three-dimensional convolutional layers, 2 pooling layers, 1 Dropout layer to prevent overfitting, 1 Flatten layer to realize one-dimensionalization of multi-dimensional input features and 2 fully connected layers; Step 3: Determine whether the chip is abnormal based on the classification result of the support vector machine; Step 4: If the result is abnormal, the parallel linkage robot removes the defective chip; if the result is normal, the test of the next chip ends.

2. The chip appearance defect detection method based on laser holographic imaging technology according to claim 1 is characterized in that: The laser in step 1 is a He-Ne laser. The input He-Ne laser is divided into two beams by a polarization beam splitter PBS. One beam of laser is expanded and collimated by a beam expansion and collimation system BE1 and then passes through an object to form an object light. The other beam of laser is expanded and collimated by a beam expansion and collimation system BE2 and then serves as a reference light. After the object light and the reference light pass through the beam splitter BS, interference fringes are generated on the CCD target surface to form a hologram. The image detection and acquisition module performs data acquisition and the image acquisition card transmits it to the image processing module. The complex amplitude distribution of the object light wave irradiated to the CCD target surface is: The complex amplitude distribution of the reference light wave irradiated to the CCD target surface is: o(x,y) and r(x,y) represent the intensity distribution of object light and reference light respectively. and Represent the phase distribution of object light and reference light respectively. The total light field of interference fringes reaching the CCD target surface is: , The CCD target surface size is Lx×Ly, the pixel size is Δx, Δy, the number of pixels is Nx=Lx / Δx, Ny=Ly / Δy, then the discrete hologram recorded on the CCD target surface is expressed as:

3. The chip appearance defect detection method based on laser holographic imaging technology according to claim 1 is characterized in that: After the hologram in step 2 is read into the computer, the computer's computing unit simulates the optical diffraction process in the CPU according to the principle of Fresnel diffraction integral to digitally reproduce the hologram, which mainly includes the following three steps: Step 2.1, set the wavelength of the reproduced light to E R (x, y), use this light wave to simulate the illumination of the hologram. According to the Rayleigh-Sommerfeld diffraction formula, the complex amplitude at the reconstruction plane with a reconstruction distance of d is: ρ represents the distance from the holographic target surface to the corresponding point on the reconstruction plane, λ is the wavelength, and the discretized expression obtained by sampling U(ξ',η') is: Δξ', Δη' represent the sampling interval of the reconstructed image plane, and the intensity and phase distribution of the reconstructed image plane are respectively: I(m,n)=|U(m,n)| 2 Step 2.2: Substituting into U(ξ',η') we get: Transforming the position of the exponential function and following the meaning of Fourier transform, we get: Step 2.3, use fast Fourier transform to calculate the result of the previous step and use the row-column point-by-point method to perform phase unwrapping to obtain the reconstructed image.