Chip surface defect detection method based on self-supervised pre-training
By employing self-supervised pre-training and data augmentation techniques, the high-cost annotation and complex scale issues in chip surface defect detection have been resolved, enabling efficient and accurate defect detection that adapts to changes in the industrial environment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING MICROELECTRONICS TECH INST
- Filing Date
- 2025-12-18
- Publication Date
- 2026-05-01
AI Technical Summary
Existing technologies for chip surface defect detection suffer from high data annotation costs, complex defects, and large scale differences. Self-supervised learning methods are still in the early stages of research in the field of chip defect detection.
A self-supervised pre-training method is adopted, which involves building an online network and a target network, pre-training with unlabeled samples, and combining data augmentation techniques such as mirror flipping, cropping and scaling, color variation and sharpness adjustment to optimize feature extraction and recognition.
It reduces the cost of manual labeling, improves the accuracy and efficiency of chip defect detection, reduces errors in the production process, and adapts to complex industrial environments.
Smart Images
Figure CN121962706A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a chip surface defect detection method based on self-supervised pre-training, belonging to the field of defect detection technology. Background Technology
[0002] Faced with the increasingly complex demands of the information society, microelectronic chips, as the "heart" of information processing and transmission, are playing an increasingly important role in global scientific and technological development. In recent years, chips have played a decisive role in key areas such as communication equipment, consumer electronics, aerospace, and industrial manufacturing, while also playing a core role in national security and defense. After modern chip manufacturing, surface defects must be inspected before the chips can proceed to the packaging process. Therefore, timely detection and elimination of defects such as dirt adhesion and scratches on the chip surface before packaging is crucial for the performance, quality, and lifespan of the finished chip. Exploring effective chip surface defect detection technologies will greatly improve the quality and stability of chip products.
[0003] Current mainstream surface defect detection technologies are primarily based on traditional machine learning methods, which involve image processing and manually set features to detect input samples. However, deep learning methods for defect detection are not limited by manually set features; they can automatically extract features through extensive data training and achieve higher accuracy. In recent years, with the development of deep neural network technology, its application in chip defect detection has become increasingly widespread. Deep learning methods can effectively replace traditional manual inspection processes, achieving automated and efficient product quality monitoring. Some existing studies have improved network structures from the target detection field for chip inspection tasks. However, in the process of implementing chip defect detection technology, some problems still need to be solved and researched.
[0004] In real industrial production, there is an imbalance between positive and negative samples, with qualified samples far outnumbering defective samples, and abnormal defects concentrated in a small number of samples. Deep learning algorithms require massive amounts of data, and manual defect labeling is extremely resource-intensive. Chip defect assessment is strongly coupled with manufacturing processes, requiring data labeling by experts with extensive industry experience. Defects in real industrial scenarios are often very small, making manual identification and labeling difficult. Chip surface defects are often complex, with multiple types and scales of defects appearing on a single sample surface. Compared to supervised learning methods that heavily rely on the quantity and quality of labels, self-supervised learning can fully utilize unlabeled data in datasets, making it widely used in complex real-world scenarios where data labeling is difficult, and achieving performance superior to supervised learning in downstream tasks. Unlike the rapid development of self-supervised learning methods in natural images, research on defect detection methods based on self-supervised learning is still in its early stages and represents a significant technical challenge that requires further resolution. Summary of the Invention
[0005] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a chip surface defect detection method based on self-supervised pre-training, which solves the problems of high data annotation cost, complex defects and large scale differences.
[0006] The technical solution of this invention is: a chip surface defect detection method based on self-supervised pre-training, comprising: Raw data of chip surface defect images without sample labels were selected as training and testing sets, respectively. The raw data were preprocessed and standardized to obtain standard training data and standard testing data. A self-supervised learning network structure is constructed; the self-supervised learning network structure includes an online network and a target network. Both the online network and the target network include an encoding network and a projection network, each containing two fully connected layers, one batch normalization layer, and one ReLU function. The online network also includes a prediction layer with the same structure as the projection network. The training hyperparameters are set, and the standard training data is input into the encoding network of the online network. After passing through the projection network and the prediction layer, the online network is optimized by gradient descent so that its prediction output approximates the feature representation of the target network. The target network performs momentum update based on the parameters of the online network. The improved chip surface defect detection capability of the trained self-supervised learning network structure was verified using the standard test data.
[0007] Furthermore, the preprocessing includes the removal of out-of-focus samples and the standardization of input dimensions.
[0008] Furthermore, before inputting the standard training data into the encoding network of the online network, the standard training data is augmented, and the data augmentation includes: Randomly flip horizontally and vertically; Defect image cropping and scaling: set the cropping scale parameter range to (0.5, 1), and set the cropping aspect ratio parameter range to (0.7, 1.5); Dynamically adjusting the color parameters of the image and reducing the model's sensitivity to color information through grayscale processing; Random image sharpening and slight Gaussian noise perturbation are applied to adjust the sharpness of defective images.
[0009] Furthermore, the momentum update is ;in For parameters of the online network, For the parameters of the target network, This is the momentum parameter.
[0010] Furthermore, during training, the self-supervised learning network structure first randomly selects defect samples from the standard training data. Two different data augmentation methods were selected for defect samples. Two-dimensional representation augmentation: The parameters of the online network are: The output result is The output after passing through the projection network is The output after pre-classification network The parameters of the target network are: The target network output is Output after projection onto the target network .
[0011] Furthermore, the error calculation formula for training the self-supervised learning network structure is as follows: ;in express and The inner product of two vectors.
[0012] Furthermore, Input into the target network, The input is fed into the target network, and the loss is obtained through the loss calculation formula. The final loss of the network is .
[0013] Furthermore, the encoding network includes several Ghost bottleneck structures with a step size of 2, which are used to reduce the feature map size and enter the next stage of the network. Moreover, through the laterally connected feature fusion structure, the extracted features are passed to the decoder part of the network to ensure that the features between different levels are fused together, thereby realizing the utilization of feature map information at different levels.
[0014] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a chip surface defect detection method based on self-supervised pre-training.
[0015] A chip surface defect detection device based on self-supervised pre-training includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: when the processor executes the computer program, it implements the steps of the chip surface defect detection method based on self-supervised pre-training.
[0016] The advantages of this invention compared to the prior art are: This invention addresses the high cost of manually labeled samples and the fact that unlabeled samples contain a large amount of defect information. It learns chip defect features from unlabeled samples and ultimately uses a small number of labeled samples for training to achieve effective detection of chip defect samples. This reduces errors that may arise from manual judgment, improves the efficiency of workers, and reduces production costs. Attached Figure Description
[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram of a dual-branch structure for the online network and the target network. Figure 2 This is a schematic diagram of a self-supervised learning network structure; Figure 3 Comparison of the results of horizontal and vertical flipping of the defect image; Figure 4 Comparison of cropped and scaled images of defective images; Figure 5 A comparison image showing the color jitter results of defective images; Figure 6 This is a comparison image of the grayscale processing results of defective images; Figure 7 Comparison of results for randomly sharpening and adding Gaussian blur to defective images; Figure 8 A graph showing the loss variation during the training process of a self-supervised learning network. Figure 9 This is an example diagram for detecting defects on the chip surface. Detailed Implementation
[0018] To better understand the above technical solutions, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.
[0019] Symbol explanation: To standardize the input image tensor; The output image tensor is standardized; This is the mean vector of the three channels; This is the standard deviation vector of the three channels; Output feature vectors for the online network; Output feature vectors for the target network; It is a norm 2; It is the dot product of vectors; and These are the parameters for the online network and the target network, respectively. The momentum coefficient has a range of values. .
[0020] The following description, in conjunction with the accompanying drawings, provides a more detailed explanation of a chip surface defect detection method based on self-supervised pre-training provided by an embodiment of the present invention. Specific implementation methods may include: Step 1: Data Preprocessing. First, the raw data is preprocessed, including removing out-of-focus samples and standardizing the input size. Second, to ensure consistent features used in training, the data needs to be standardized to obtain a usable dataset.
[0021] Step 2: Divide the obtained available dataset into training and test sets. The available dataset contains a small amount of manually labeled defect data. Select unlabeled chip surface defect images from the dataset as the training set, and select manually labeled chip defect data from the dataset as the test set. Preprocess the training set data using the method in Step 1.
[0022] Step 3: Construct a self-supervised learning network structure, employing a two-branch structure of a real-time updated online network and a slower-updating target network, such as... Figure 1 Both consist of an encoder and a projection network (containing two fully connected layers + a batch normalization layer + ReLU activation), but the online network adds an additional prediction layer, with the same structure as the projection network. During training, data in the online network is input into the encoder network, such as... Figure 2 After passing through a projection network and a prediction layer, the online network is optimized through gradient descent to make its predicted output approximate the feature representation of the target network. The target network does not directly update with gradients, but instead updates with momentum based on the parameters of the online network. This design avoids model degradation through the prediction layer and utilizes the slowly updated target network for stable training, ultimately making the encoded features more discriminative.
[0023] Step 4: Defect Image Data Augmentation Methods. In common self-supervised learning frameworks, data augmentation methods often employ a fixed pipeline, including random cropping, horizontal flipping, color dithering, Gaussian blurring, and grayscale conversion. Since the foreground and background content of surface defect image datasets in real-world industrial scenarios differs significantly from other large datasets containing objects in everyday life, a data augmentation method specifically for defect images needs to be designed. This method should ensure that the generated defect image positive sample pairs have sufficient similarity so that the model can identify them as the same original sample, while also possessing sufficient difference to increase the difficulty of contrastive learning. In contrastive self-supervised learning, this invention employs several data augmentation methods—defect image mirror flipping, defect image cropping and scaling, defect image color variation, and defect image sharpness variation—to augment chip defect images.
[0024] Step 5, Train the network model. Train the self-supervised learning network using the training set. Based on the network model built in Step 3, read in the training set from Step 2, use the data augmentation method from Step 4 to input it into the network model, train the network model, set hyperparameters such as the number of training epochs and the model update frequency, and plot the loss curve of the training process.
[0025] Step 6, Encoding Effect Verification. To verify the feature representation capability of the defect data after encoding by the self-supervised learning network, the encoder structure and parameters in the self-supervised learning network are fixed, and the improvement of the chip surface defect detection capability after encoding is verified on the test set.
[0026] In step 1, the resolution is adjusted to 512×512 using a scaling method and then used as the input to the model. The image is then standardized using the following formula:
[0027] Wherein, input is the scaled input data, output is the standardized output data, mean is the mean of each channel, and std represents the standard deviation of each channel.
[0028] In step 3, during training, the online network is updated based on the network loss, and then the target network is updated using momentum update. The momentum update calculation formula is:
[0029] in These are parameters of the online network. These are the parameters of the target network. It is the momentum parameter, which is usually set to a number very close to 1.
[0030] In step 3, during network training, defect samples are first randomly selected from the training set. Two different augmentation methods were selected to augment the two-dimensional representation of defect data: The parameters of the online network are: The output result is The output after passing through the projection network is The output after pre-classification network The parameters of the target network are: The target network output is Output after projection onto the target network .
[0031] To extract the absolute magnitudes of the two feature vectors output by the online and target networks while preserving their directionality, this method uses L2-normalization to normalize the feature vectors output by both networks. The calculation formula is as follows:
[0032] in Target network output features This is the output feature of the online network.
[0033] This method trains the network by minimizing the error between the feature vectors output by the online network and the target network. The error calculation formula is as follows:
[0034] in express and The inner product of two vectors. In actual training, the generated vectors also need to be swapped. ,Will Input into the target network, The input is fed into the target network, and the loss is obtained through the loss calculation formula. The final loss of the network is the sum of the two losses, as calculated by the following formula:
[0035] In step 4, the augmentation method for chip surface defect data is as follows: (1) Mirroring and flipping the defective image. For example... Figure 3The defect images are all collected from real production environments. These images differ from general object images, possessing unique characteristics in their representation of defect features. When a general object image is vertically flipped, the resulting image shows a significant difference in object shape from the original, making it difficult for the model to classify them as belonging to the same category. However, for defect images, the key semantic information they contain is unaffected by image mirroring. Regardless of whether the image undergoes horizontal or vertical flipping, the essential attributes of the defect remain unchanged, and its crucial discriminative information for the self-supervised learning network is not lost. Based on this characteristic, this invention employs two data augmentation strategies—horizontal and vertical random flipping—to augment the defect images. This satisfies the requirement of constructing positive sample pairs while improving the learning ability and generalization performance of the self-supervised network when faced with defect image input.
[0036] (2) Defective image cropping and scaling. For example... Figure 4 In the defect dataset used in this invention, the relative size of defects in the images is relatively small. This characteristic places special demands on the selection of data augmentation strategies. Conventional random cropping methods easily remove key defect parts, making it difficult for the model to capture and learn these key defect features during training. To address this challenge, this invention selects a smaller cropping size, setting the cropping scale parameter to a range of (0.5, 1) to preserve defect information while introducing appropriate spatial variation, improving the model's ability to perceive different image regions. This approach ensures that the model can cover defect areas with a certain probability while maintaining data diversity. Furthermore, considering the importance of shape features for defect recognition, a more stringent cropping aspect ratio parameter is set, with a value range of (0.7, 1.5). By controlling the aspect ratio of the cropped image, severe distortion of shape features is avoided when the cropped image is subsequently stretched to the model input size, which would negatively impact subsequent defect recognition performance.
[0037] (3) Color changes in defective images. For example... Figure 5 , Figure 6In actual industrial production environments, due to varying shooting times and diverse lighting conditions, the color contrast, brightness, saturation, and hue parameters displayed in defect images exhibit significant variability. Furthermore, defect category determination relies more on features such as shape and texture than color features. This characteristic determines the secondary role of color information in defect determination; therefore, color variation methods are incorporated to process input defect samples. This invention augments defect images with color variations, including color dithering and grayscale processing. Color dithering dynamically adjusts the image's color parameters to simulate effects under different lighting and environmental conditions; grayscale processing reduces the model's sensitivity to color information, emphasizing the importance of shape and texture. The application of these two strategies aims to mitigate or even eliminate the potential impact of different environmental and lighting conditions on defect visibility, improve the generalization ability of the defect feature extraction network and its adaptability to complex real-world environments, and significantly enhance the model's ability to withstand changes in the external environment.
[0038] (4) Changes in the sharpness of defective images. For example... Figure 7 In real-world industrial production environments, acquiring defect images typically relies on industrial cameras, which are either placed under microscopes or positioned alongside production lines. Due to various environmental factors and equipment limitations, image quality is often compromised. To enable the encoder to effectively adapt to and learn the characteristics of defects under both normal and poor imaging conditions, this invention adds random image sharpening operations and slight Gaussian noise perturbations to the images to simulate image quality issues that may be encountered in real industrial acquisition environments. In this way, the encoder is forced to recognize and adapt to various image quality variations during training, incorporating imaging effects from excellent to poor. Using this data augmentation strategy deepens the neural network's understanding of defect features and enables it to process on-site acquired defect images more robustly, improving the algorithm's adaptability and reliability on real-world industrial defect images.
[0039] In step 6, the detection of chip surface defects is achieved based on the target network.
[0040] At each resolution level of the encoder, a feature extraction module is used to extract multi-dimensional feature information from the input. Using a Ghost bottleneck structure with a stride of 2 not only effectively reduces the feature map size, allowing it to smoothly enter the next stage of the network, but also, through a laterally connected feature fusion structure, passes the extracted features to the network's decoder, ensuring that features from different levels can be fused together and fully utilizing the feature map information from different levels.
[0041] The network employs a top-down decoding process to propagate rich semantic information from high-level features to lower-level features, ensuring that even low-level features retain rich semantic information. Four upsampling layers are used to progressively recover feature maps at different resolutions, and a proposed feature fusion structure is used to fuse the high- and low-level features at each layer. Specifically, a 1×1 convolution is used to integrate the output features of each layer and unify the feature map dimension to 256. Upsampling is achieved through 1×1 nearest neighbor interpolation, maximizing the preservation of semantic information in the feature maps during the upsampling process. Each layer uses the `add` feature fusion method to fuse the upsampled result from the previous layer with the features processed by the 1×1 convolution in the current layer.
[0042] After the add operation, a 3×3 convolutional layer is used to obtain the feature output Pn of this layer, with the number of output channels set to 256, further eliminating the aliasing effect caused by the previous upsampling process. In addition, a 1×1 max pooling layer is added on the top-level feature layer to obtain a 7×7×256 feature layer for detecting defects at larger scales.
[0043] Furthermore, in step 1, the resolution is adjusted to 512×512 using a scaling method before being used as the model input, and the image is standardized using the following formula:
[0044] Wherein, input is the scaled input data, output is the standardized output data, mean is the mean of each channel, and std represents the standard deviation of each channel.
[0045] Furthermore, in step 3, during network training, defect samples are first randomly selected from the training set. Two different augmentation methods were selected to augment the two-dimensional representation of defect data: The parameters of the online network are: The output result is The output after passing through the projection network is The output after pre-classification network The parameters of the target network are: The target network output is Output after projection onto the target network .
[0046] To extract the absolute magnitudes of the two feature vectors output by the online and target networks while preserving their directionality, this method uses L2-normalization to normalize the feature vectors output by both networks. The calculation formula is as follows:
[0047] in Target network output features This is the output feature of the online network.
[0048] This method trains the network by minimizing the error between the feature vectors output by the online network and the target network. The error calculation formula is as follows:
[0049] in express and The inner product of two vectors. In actual training, the generated vectors also need to be swapped. ,Will Input into the target network, The input is fed into the target network, and the loss is obtained through the loss calculation formula. The final loss of the network is the sum of the two losses, as calculated by the following formula:
[0050] The solution provided in the embodiments of the present invention includes the following steps: Step 1: Data Preprocessing. First, the raw data is preprocessed, including removing out-of-focus samples and standardizing the input size. Second, to ensure consistent features used in training, the data needs to be standardized.
[0051] Step 2: Training and test set division. A large number of unlabeled chip surface defect images are selected as the training set, and a small number of labeled chip defect data are selected as the test set. The training set data is preprocessed using the method in Step 1.
[0052] Step 3: Construct a self-supervised learning network structure and design a loss function. The supervised learning network structure proposed in this invention is generally divided into an online network and a target network. The online network is trained by constraining the mean squared error of the output features of these two networks, while the parameter updates of the target network depend on the currently updated parameters of the online network and the current target network. Training degradation is avoided by adding a prediction network and not using gradient updates to the target network. The projection network structure consists of two fully connected layers, followed by a batch normalization layer activated using the ReLU function. The input data is encoded by the encoder to obtain a 128-dimensional feature representation, which is the feature extracted by the convolutional neural network. The projection network integrates the feature maps output by the convolutional structure into a feature vector, mapping the feature space to a more explicit representation space for classification information. Both the online network and the target network have encoding and projection layers, which extract the feature representation of the input data in the latent space. This invention adds a prediction layer after the projection layer of the online network. The feature representation obtained by the online network is further mapped using the prediction layer to make the output value as close as possible to the feature representation output by the target network. The goal is to make the features output by the online network as close as possible to the features output by the target network, which means using the prediction layer of the online network to predict the feature representation output by the target network.
[0053] Step 4: Defect Image Data Augmentation Methods. In common self-supervised learning frameworks, data augmentation methods often employ a fixed pipeline, including random cropping, horizontal flipping, color dithering, Gaussian blurring, and grayscale conversion. Since the foreground and background content of surface defect image datasets in real-world industrial scenarios differs significantly from other large datasets containing objects in everyday life, a data augmentation method specifically for defect images needs to be designed. This method should ensure that the generated defect image positive sample pairs have sufficient similarity so that the model can identify them as the same original sample, while also possessing sufficient difference to increase the difficulty of contrastive learning. In contrastive self-supervised learning, this invention employs several data augmentation methods—defect image mirror flipping, defect image cropping and scaling, defect image color variation, and defect image sharpness variation—to augment chip defect images.
[0054] Step 5, Train the network model. Train the self-supervised learning network using the training set. Based on the network model built in Step 3, read in the training set from Step 2, and input it into the network model using the data augmentation method from Step 4. Train the network model, setting hyperparameters such as the number of training epochs and the model update frequency, and plotting the loss curve during the training process, as shown below. Figure 8 .
[0055] Step 6, Encoding Effect Verification. To verify the feature representation ability of the defect data after encoding by the self-supervised learning network, the encoder structure and parameters in the self-supervised learning network are fixed, and the improvement of the chip surface defect detection capability after encoding is verified on the test set, such as... Figure 9 .
[0056] Table 2 shows a comparison of the accuracy of different methods under the same training set conditions: Table 2 Comparison results of different algorithms on the test set
[0057] The results above demonstrate that the method of this invention has the significant advantage of maintaining both good detection performance and fast processing speed, which is crucial for real-time applications and high-efficiency scenarios. Furthermore, the method of this invention significantly reduces the number of network parameters compared to the other two algorithms, decreasing the number of parameters by nearly half compared to RetinaNet and by more than two-thirds compared to Faster R-CNN. This demonstrates its lightweight advantage in model size, greatly reducing computational load and facilitating model deployment in resource-constrained environments.
[0058] This invention provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform... Figure 1 The method described.
[0059] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0060] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0061] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0062] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0063] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0064] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
[0065] The contents not described in detail in this specification are common knowledge to those skilled in the art.
Claims
1. A chip surface defect detection method based on self-supervised pre-training, characterized in that, include: Raw data of chip surface defect images without sample labels were selected as training and testing sets, respectively. The raw data were preprocessed and standardized to obtain standard training data and standard testing data. A self-supervised learning network structure is constructed; the self-supervised learning network structure includes an online network and a target network. Both the online network and the target network include an encoding network and a projection network, each containing two fully connected layers, one batch normalization layer, and one ReLU function. The online network also includes a prediction layer with the same structure as the projection network. The training hyperparameters are set, and the standard training data is input into the encoding network of the online network. After passing through the projection network and the prediction layer, the online network is optimized by gradient descent so that its prediction output approximates the feature representation of the target network. The target network performs momentum update based on the parameters of the online network. The improved chip surface defect detection capability of the trained self-supervised learning network structure was verified using the standard test data.
2. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, The preprocessing includes removing out-of-focus samples and standardizing the input size.
3. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, Before inputting the standard training data into the encoding network of the online network, the standard training data is augmented, and the data augmentation includes: Randomly flip horizontally and vertically; Defect image cropping and scaling: set the cropping scale parameter range to (0.5, 1), and set the cropping aspect ratio parameter range to (0.7, 1.5); Dynamically adjusting the color parameters of the image and reducing the model's sensitivity to color information through grayscale processing; Random image sharpening and slight Gaussian noise perturbation are applied to adjust the sharpness of defective images.
4. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, The momentum update is ;in For parameters of the online network, For the parameters of the target network, This is the momentum parameter.
5. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, During training, the self-supervised learning network structure first randomly selects defect samples from the standard training data. Two different data augmentation methods were selected for defect samples. Two-dimensional representation augmentation: The parameters of the online network are: The output result is The output after passing through the projection network is The output after pre-classification network The parameters of the target network are: The target network output is Output after projection onto the target network .
6. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, The error calculation formula for training the self-supervised learning network structure is as follows: ;in express and The inner product of two vectors.
7. The chip surface defect detection method based on self-supervised pre-training according to claim 6, characterized in that, Will Input into the target network, The input is fed into the target network, and the loss is obtained through the loss calculation formula. The final loss of the network is .
8. The chip surface defect detection method based on self-supervised pre-training according to claim 1, characterized in that, The encoding network includes several Ghost bottleneck structures with a step size of 2, which are used to reduce the feature map size and enter the next stage of the network. Moreover, through the laterally connected feature fusion structure, the extracted features are passed to the decoder part of the network to ensure that the features between different levels are fused together, thereby realizing the utilization of feature map information at different levels.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 8.
10. A chip surface defect detection device based on self-supervised pre-training, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.