Semiconductor defect classification method based on polarized light technology and unsupervised neural network
By combining polarization light technology and unsupervised neural networks, the deep features of polarization imaging data are automatically extracted, and the problem of low accuracy of defect classification in industrial detection is solved, and the defect classification with high accuracy and robustness is achieved, which is suitable for industrial and Smart devices.
Patent Information
- Application Number
- CN202411206649.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2025-01-10
AI Technical Summary
In the industrial detection environment, the color and polarization-based defect classification method is not accurate because the diffused component is larger than the mirror component, making it difficult to achieve robust and accurate defect classification.
Combining polarization light technology and unsupervised neural networks, polarization degree images are obtained through polarization imaging systems, and deep features are automatically extracted using a generative adversarial network (GAN) or an autoencoder to perform defect classification.
It improves the accuracy and robustness of defect classification, can adapt to multiple defects under complex lighting conditions, and significantly improves the classification accuracy and the intelligence level of the system.
Smart Images
Figure CN119295378A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image processing technology, and in particular relates to a semiconductor defect classification method based on polarized light technology and unsupervised neural network. Background Art
[0002] In industrial automation and machine vision systems, accurate classification of defects is the key to ensuring production quality and efficiency, and distinguishing defects on the surface of objects from image data is a fundamental problem in many fields, including object recognition. In traditional technologies, there are two types of clues for defect classification: color and polarization. The color-based classification method is based on the dichroic reflection characteristics of the surface of the medium to be tested, that is, the detector converts the output of the object surface into a color distribution histogram of two linear clusters in the RGB space. The directions of the two linear clusters correspond to the diffuse reflection color from the object and the specular reflection color from the illumination light source. However, this distinction is often difficult because the color histogram depends on multiple factors, including surface position, light source, detector posture, and the color of the object itself.
[0003] According to Fresnel reflection theory, the surface of a dielectric causes strong partial polarization of the incident light during specular reflection, while the surface of a metal causes a smaller partial polarization of the reflected light. Since the ratio of the maximum and minimum transmitted radiation detected on the surface of an object, relative to the direction of the polarizer, can be used to estimate the polarization Fresnel ratio, the distinction between dielectrics and metals can be achieved by setting a threshold for this ratio of the transmitted radiation through the polarizer at each pixel. However, in an industrial inspection environment, the diffuse component is often greater than the specular component, making the accuracy of the results of this method unsatisfactory. Therefore, finding a more robust and accurate defect classification method has become an urgent problem to be solved. Summary of the invention
[0004] The purpose of the present invention is to provide a semiconductor defect classification method based on polarized light technology and unsupervised neural network, which not only utilizes the physical properties of polarized light, but also automatically extracts complex features in the data through neural network, thereby improving classification accuracy and robustness.
[0005] A semiconductor defect classification method based on polarized light technology and unsupervised neural network includes the following steps:
[0006] Step 1: Build a polarization imaging system and an unsupervised neural network;
[0007] Step 2: Based on the polarization imaging system, a sample image img1 with surface defects is obtained, where the sample image img1 is a polarization degree image;
[0008] Step 3, input the sample image img0 and the sample image img1 into the unsupervised neural network to obtain the sample image img2; wherein the sample image img0 is the original image of the sample with surface defects;
[0009] Step 4: Compare the sample image img2 obtained in step 3 with the sample image img1, and classify the defects according to the degree of polarization.
[0010] Preferably, step 4 specifically includes:
[0011] The defect area position data index is extracted from the sample image img2, and the corresponding position of the defect area position is found in the sample image img1 to find the polarization degree corresponding to the defect area position.
[0012] Preferably, the unsupervised neural network is a generative adversarial network (GAN) or an autoencoder.
[0013] Preferably, the polarization imaging system comprises:
[0014] A sample, a light source and an imaging component arranged above the sample;
[0015] The sample is arranged horizontally, the angle between the light source and the vertical line in the sample is a first angle, and the angle between the imaging component and the vertical line in the sample is a second angle;
[0016] A polarization mechanism is provided between the imaging component and the sample;
[0017] The polarization direction of the polarization mechanism is perpendicular to the light source incident surface of the light source component, and the mounting surface of the polarization mechanism is perpendicular to the optical axis direction of the imaging component.
[0018] The imaging component is used to acquire the sample image img1.
[0019] Preferably, the polarization mechanism comprises a rotating linear polarizer for adjusting the incident angle of polarized light; the light source is a linear polarized light source with adjustable direction, for irradiating the defective surface to obtain reflected light.
[0020] The physical principles of the present invention are as follows:
[0021] Polarization imaging distinguishes different defects by analyzing the polarization distribution of highlight areas.
[0022] According to the Fresnel equation and Snell's law, the polarization characteristics of the reflected light from the defect can be described as follows:
[0023]
[0024]
[0025] Furthermore, the Stokes parameters of the light field can fully describe the polarization state of light:
[0026]
[0027]
[0028] S2=2E x E y cosδ
[0029] S3=2E x E y sinδ
[0030] Therefore, the degree of polarization can be defined according to the Stokes parameters:
[0031]
[0032] These physical parameters provide a detailed description of the surface characteristics of the defects.
[0033] However, the multidimensionality and complexity of polarization imaging data make it difficult for traditional algorithms to fully utilize this information.
[0034] The present invention automatically extracts and processes these complex features by introducing an unsupervised neural network. The core of the neural network is an unsupervised learning model for automatically extracting deep features in polarization imaging data.
[0035] Specifically, the present invention can use a neural network architecture of an autoencoder or a generative adversarial network (GAN) to achieve this goal through specially designed objective function optimization iterations.
[0036] Compared with the prior art, the advantages of the present invention are:
[0037] 1. The technical innovation of the present invention lies in the combination of polarization imaging and deep learning. By combining the physical polarization characteristics with the deep learning ability of unsupervised neural networks, high-precision and automated defect classification is achieved.
[0038] Among them, the physical polarization characteristics refer to:
[0039] When the sample surface is smooth and defect-free, all reflected light is polarized light with a specific polarization angle. The deflection angle is constrained by the Fresnel equation and can be cut off by the polarization mechanism, so that the imaging component obtains a high-grayscale image.
[0040] When there are scratches or dirt defects on the sample surface, diffuse reflection is formed locally at the defect, and the diffuse reflection light is no longer polarized light and is no longer cut off by the polarization mechanism. The corresponding position of the imaging component is low grayscale, forming a grayscale difference with the background, and the defect is detected.
[0041] 2. Unsupervised learning improves robustness: Using unsupervised neural networks to process polarization imaging data can automatically extract effective features without the need for a large amount of labeled data, and adapt to more diverse defects and complex lighting conditions.
[0042] Optimized neural network structure: Through the deep learning structure of autoencoder or GAN, the deep features of polarization images are extracted, and the classification accuracy is further improved by combining the attention mechanism.
[0043] By introducing unsupervised neural networks, the present invention not only significantly improves the defect classification accuracy and robustness, but also improves the intelligence level of the system, and can be widely used in industry and intelligent equipment. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is an example of the sample image img1, which is a polarization image calculated from the Stokes vector;
[0045] Figure 2 is an example of the sample image img0, which is the original image directly measured by the image sensor;
[0046] Figure 3 is an example of the sample image img2, and is the image processing result of the proposed method;
[0047] Figure 4 To generate the data flow diagram of the adversarial network GAN, the target image x t is the polarization image and x s Original image. DETAILED DESCRIPTION
[0048] The semiconductor defect classification method based on polarized light technology and unsupervised neural network of the present invention will be described in more detail below in conjunction with the schematic diagram, wherein the preferred embodiment of the present invention is shown, and it should be understood that those skilled in the art can modify the present invention described herein, while still achieving the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art, and not as a limitation of the present invention.
[0049] like Figures 1 to 4 , a semiconductor defect classification method based on polarized light technology and unsupervised neural network, comprising the following steps:
[0050] Step 1: Build a polarization imaging system and unsupervised neural network.
[0051] The unsupervised neural network is a generative adversarial network (GAN) or an autoencoder.
[0052] Polarization imaging system includes:
[0053] A sample, a light source and an imaging component arranged above the sample;
[0054] The sample is arranged horizontally, the angle between the light source and the vertical line in the sample is a first angle, and the angle between the imaging component and the vertical line in the sample is a second angle;
[0055] A polarization mechanism is provided between the imaging component and the sample;
[0056] The polarization direction of the polarization mechanism is perpendicular to the light source incident surface of the light source component, and the mounting surface of the polarization mechanism is perpendicular to the optical axis direction of the imaging component.
[0057] The imaging component is used to obtain the sample image img1.
[0058] The polarization degree in the sample image img1 is calculated by the above DOP formula. The above formulas involved in the DOP formula are all integrated in the imaging component.
[0059] Step 2: Based on the polarization imaging system, obtain a sample image img1 with surface defects: the sample image img1 is a polarization image.
[0060] Specifically, the light source irradiates the semiconductor surface and generates reflected light, and the reflected light passes through the polarization mechanism and is transmitted to the imaging component; the imaging component obtains the polarization image img1.
[0061] Step 3: Input the sample image img0 and the sample image img1 into the unsupervised neural network to obtain the sample image img2.
[0062] Among them, the neural network is recommended to use the adversarial network GAN. It should be pointed out that in the setting of the unsupervised neural network architecture, the output should be set to be consistent with the input dimension, and the objective function is the root mean square error of the network output and the Laplace kernel convolution result of the sample images img0 and img1. It is recommended to use the Adam optimizer to optimize the neural network.
[0063] The sample image img0 is an original image of a sample with surface defects, and the means of obtaining the sample image img0 is the existing technology. In addition, it should also be pointed out that since the neural network adopts unsupervised technology, the image img0 is normalized as a weighted regularization input to the neural network to suppress the numerical divergence during the network training process.
[0064] In the training network, the feature value input and output corresponding to each defect sample in the sample image img0 and the sample image img1, as well as the defect classification results, will be used as the output of the network structure layer and the input of the next structure layer. The final output result of the network is a two-dimensional image with a single defect as the connected area, and its expression is similar to the defect contour map, such as Figure 3 shown.
[0065] Step 4: Compare the sample image img2 obtained in step 3 with the sample image img1, and classify the defects according to the degree of polarization.
[0066] Classification can be performed based on the different connected areas (defects) in the sample image img2 and the polarization degrees of the corresponding positions in the sample image img1. In this example, all the connected areas of the sample image img2 have similar corresponding polarization degrees, and therefore belong to the same defect type. Accordingly, if the polarization degrees of different connected areas are very different, different defect types can be classified. The specific classification can be determined according to actual needs.
[0067] Furthermore, the polarization mechanism includes a rotating linear polarizer for adjusting the incident angle of polarized light; the light source is a linear polarized light source with adjustable direction, which is used to irradiate the defective surface to obtain reflected light.
[0068] The specific implementation is as follows:
[0069] 1) System construction
[0070] First, build the polarization imaging system and neural network processing system.
[0071] The polarization imaging system includes a rotating linear polarizer, a high-resolution digital camera, and an adjustable linear polarization light source. By adjusting the angles of the light source and the polarizer, the system can capture images of reflected light from defective surfaces at different polarization angles. At the same time, an unsupervised neural network is built using a deep learning framework (such as TensorFlow or PyTorch).
[0072] Autoencoder and Generative Adversarial Network (GAN) are two optional network structures. Autoencoder is used to automatically extract deep features of images, while GAN learns complex patterns of defect surface features through adversarial training of generator and discriminator.
[0073] 2) Data collection and preprocessing
[0074] a) Image capture: Capture the reflected light image of the object surface at different polarization angles.
[0075] b) Data preprocessing: The captured images are standardized, the image size and grayscale value range are adjusted, and an image dataset suitable for neural network input is generated.
[0076] After the system is set up, data collection begins. First, reflective light images of defective surfaces are taken at different polarization angles. These images are standardized, including adjusting the size and grayscale value range, to generate image datasets suitable for neural network input.
[0077] 3) Polarization degree calculation and feature extraction
[0078] a) Polarization calculation: Calculate the DOP of each pixel in the highlight area and generate a polarization image such as Figure 1 .
[0079] b) Input the processed polarization image into the autoencoder or GAN to automatically extract the deep features of the defect surface through unsupervised learning.
[0080] The polarization degree image is calculated based on Snell's law and Fresnel equation combined with Stokes parameters.
[0081] The polarization distribution of the defect area usually manifests itself as the optical characteristics of different defect surfaces at specific incident angles.
[0082] Low-dimensional feature vectors are extracted from these polarization imaging data through autoencoders or GANs.
[0083] These features can effectively capture the reflection characteristics and differences of the defective surface and provide important information for the input of the classifier.
[0084] The encoder part of the autoencoder passes through multiple convolutional layers and pooling layers to convert the high-dimensional polarization image and the original image into Figure 2 Compress to a low-dimensional feature space.
[0085] Through these unsupervised learning methods, the model is able to automatically learn characteristic patterns from unlabeled data.
[0086] 4) Defect classification and experimental verification
[0087] a) Test different metal and dielectric defects, according to Figure 4 The structure shown records the classification results and compares them with the actual defect properties to verify the accuracy and robustness of the method.
[0088] b) Adjust the structure and parameters of the neural network according to the experimental results to further optimize the performance of defect classification.
[0089] Through these implementation steps, the present invention can achieve efficient and automated defect classification, especially under complex lighting conditions and multiple defect backgrounds, and exhibits excellent classification accuracy and robustness.
[0090] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.
Claims
1. A semiconductor defect classification method based on polarized light technology and unsupervised neural network, characterized in that: The following steps are involved: Step 1: Build a polarization imaging system and an unsupervised neural network; Step 2: Based on the polarization imaging system, a sample image img1 with surface defects is obtained, where the sample image img1 is a polarization degree image; Step 3, input the sample image img0 and the sample image img1 into the unsupervised neural network to obtain the sample image img2; wherein the sample image img0 is the original image of the sample with surface defects; Step 4: Compare the sample image img2 obtained in step 3 with the sample image img1, and classify the defects according to the degree of polarization.
2. The semiconductor defect classification method based on polarized light technology and unsupervised neural network according to claim 1 is characterized in that: Step 4 is as follows: The defect area position data index is extracted from the sample image img2, and the corresponding position of the defect area position is found in the sample image img1 to find the polarization degree corresponding to the defect area position.
3. The semiconductor defect classification method based on polarized light technology and unsupervised neural network according to claim 1, characterized in that: The unsupervised neural network is a generative adversarial network GAN or an autoencoder.
4. The semiconductor defect classification method based on polarized light technology and unsupervised neural network according to claim 1, characterized in that: The polarization imaging system comprises: A sample, a light source and an imaging component arranged above the sample; The sample is arranged horizontally, the angle between the light source and the vertical line in the sample is a first angle, and the angle between the imaging component and the vertical line in the sample is a second angle; A polarization mechanism is provided between the imaging component and the sample; The polarization direction of the polarization mechanism is perpendicular to the light source incident surface of the light source component, and the mounting surface of the polarization mechanism is perpendicular to the optical axis direction of the imaging component. The imaging component is used to acquire the sample image img1.
5. The semiconductor defect classification method based on polarized light technology and unsupervised neural network according to claim 4 is characterized in that: The polarization mechanism comprises a rotating linear polarizer for adjusting the incident angle of polarized light; the light source is a linear polarized light source with adjustable direction, for irradiating the defective surface to obtain reflected light.