Lightweight wafer defect detection method

Abstract

The invention discloses a lightweight wafer defect detection method, and relates to the technical field of deep learning and defect detection. In order to solve the technical problems that in the prior art, an existing wafer defect detection method is too large in calculated amount and memory consumption, insufficient in feature extraction efficiency and difficult to guarantee defect detection precision, the technical scheme provided by the invention comprises the following steps: acquiring an image data set containing wafer defects, and preprocessing; performing feature extraction by using an optimized YOLOv8 model, wherein the model comprises a backbone network, a neck network and a detection module; performing feature extraction on the image set in the backbone network; carrying out feature fusion in the neck network; performing bounding box regression in the detection module by using a loss function; and according to an output result of the model, identifying defect positions and types in the image. The wafer defect detection device is suitable for wafer defect detection.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning and defect detection, and specifically to lightweight wafer defect detection based on YOLOv8. Background Art

[0002] With the rapid development of the semiconductor industry, wafers, as core materials, play a vital role in integrated circuit manufacturing. The detection accuracy of wafer surface defects is directly related to the performance and reliability of the chip. Therefore, wafer defect detection has always been an indispensable part of semiconductor manufacturing. In recent years, with the advancement of industrial automation and the development of image processing technology, traditional manual detection methods have gradually been replaced by automated detection systems.

[0003] At present, the main technical means of wafer defect detection include traditional methods based on optical image processing and automatic detection methods based on machine learning and deep learning. Traditional image processing methods, such as edge detection and morphological processing, usually rely on manually set thresholds and rules. These methods work well when dealing with simple defects, but often perform poorly for complex defects (such as tiny defects, irregular shapes, etc.), and are difficult to adapt to detection needs under different lighting and background conditions.

[0004] With the rise of deep learning technology, automatic defect detection methods based on convolutional neural networks (CNN) and other deep learning architectures have gradually become mainstream. In particular, the YOLO (You Only Look Once) series of models have been widely used in various defect detection tasks due to their efficient real-time detection capabilities and high detection accuracy. The YOLO series has significant advantages by dividing images into multiple regions, quickly predicting whether each region contains defects and their categories, and enabling end-to-end training and detection. The latest YOLOv8 model has further improved detection accuracy, inference speed, and real-time performance, and is particularly suitable for large-scale automated detection tasks.

[0005] However, although the application of YOLOv8 in defect detection has achieved remarkable results, its high computational complexity and large number of parameters are still the main obstacles affecting its application on edge devices. The YOLOv8 model consumes a lot of computation and storage in the process of feature extraction, feature fusion and target detection, which makes it difficult to achieve real-time and efficient deployment in resource-constrained edge devices. Especially when dealing with delicate and complex tasks such as wafer defects, the computational overhead and memory consumption of the existing YOLOv8 model limit its application in environments such as embedded systems and mobile devices.

[0006] Although the wafer defect detection method based on YOLOv8 has high detection accuracy and real-time performance, it has the following technical problems when deployed on resource-constrained devices: Excessive computation and memory consumption: The high computation and large number of parameters of the YOLOv8 model limit its real-time application on edge devices, especially in wafer defect detection tasks that require high accuracy.

[0007] Insufficient feature extraction efficiency: The feature extraction process of the Backbone network in YOLOv8 is relatively lengthy, resulting in low computational efficiency, and the redundant information in the multi-level feature fusion process increases the computational burden.

[0008] Challenges in defect detection accuracy: Although YOLOv8 can detect larger and regular defects well, it still has certain difficulties in detecting small targets, complex shapes and detailed defects. Summary of the invention

[0009] In order to solve the technical problems in the prior art that the existing wafer defect detection method has excessive calculation and memory consumption, insufficient feature extraction efficiency and difficulty in ensuring the accuracy of defect detection, the technical solution provided by the present invention is: A lightweight wafer defect detection method, comprising: A step of acquiring an image data set containing wafer defects as an image set, and preprocessing the image set; A step of performing feature extraction on the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; Based on the output of the YOLOv8 model, identify the location and type of defects in the image; The optimized YOLOv8 model also includes the steps of performing data enhancement by image rotation, scaling, cropping, and color jittering.

[0010] Furthermore, a preferred embodiment is provided, wherein the preprocessing includes image slicing and data enhancement.

[0011] Furthermore, a preferred embodiment is provided, wherein the C2f_IDR module includes an inverted residual module and a dilation reparameterization module.

[0012] Furthermore, a preferred implementation is provided, wherein the HSPAN module fuses low-level and high-level features in the image set through a bidirectional path aggregation mechanism.

[0013] Furthermore, a preferred embodiment is provided, wherein the loss function combines Focaler-IoU and Shape-IoU.

[0014] A lightweight wafer defect detection device, comprising: A module for acquiring an image data set containing wafer defects as an image set and preprocessing the image set; A module for extracting features from the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; A module that identifies the location and type of defects in an image based on the output of the YOLOv8 model; The optimized YOLOv8 model also includes a module for data enhancement through image rotation, scaling, cropping and color jittering.

[0015] A computer storage medium is used to store a computer program. When the computer reads the computer program, the computer executes the method described.

[0016] A computer includes a processor and a storage medium. When the processor reads a computer program stored in the storage medium, the computer executes the method described.

[0017] The computer program product, as a computer program, implements the method described when the computer program is executed.

[0018] Compared with the prior art, the technical solution provided by the present invention is beneficial in that: By making lightweight improvements to the YOLOv8 model, the problems of large computational complexity and high resource consumption in the existing technology are solved. Specifically, the C2f_IDR module is used to optimize the feature extraction process, significantly reducing the amount of computation while maintaining high accuracy. Compared with the complex feature extraction module in the original YOLOv8 model, the C2f_IDR module improves computational efficiency by introducing lightweight inverted residuals and dilated reparameterized structures. Studies have shown that similar improvements have also been used in other deep learning models, such as MobileNetV2 and EfficientNet, which achieve efficient deployment by reducing redundant calculations and parameters. The C2f_IDR module of this solution not only reduces the consumption of computing resources, but also optimizes the ability to extract multi-scale features, making it possible to achieve real-time detection on edge devices.

[0019] In terms of the optimization of the Neck network, the HSPAN module designed by the present invention enhances the expression ability of fine-grained features through a bidirectional path aggregation mechanism and effectively reduces the transmission of redundant information. The HSPAN module screens features through a channel attention mechanism and fuses low-level fine-grained features with high-level semantic information, thereby improving the ability to detect small targets and complex shape defects. Compared with the traditional feature pyramid network (FPN), the HSPAN module can more effectively suppress the interference of redundant information and retain important detail information. The advantage of this method is that it reduces the computational burden while retaining fine-grained information, which is more advantageous than traditional methods when dealing with defect detection in complex scenarios.

[0020] For small targets and complex shape defects in target detection, the present invention enhances the model's ability to focus on difficult-to-detect samples by introducing the Focaler-Shape IoU loss function. This loss function combines the ideas of Focaler-IoU and Shape-IoU, and can more accurately regress the boundaries of complex shape defects, and improves the detection performance of small targets by increasing the loss weight of low IoU samples. In existing research, similar loss functions such as FocalLoss in RetinaNet also adopt a weighted mechanism to deal with small target problems, but Focaler-Shape IoU provides higher accuracy by further introducing shape matching factors, especially for complex and small targets such as wafer defects, the effect is more obvious.

[0021] Suitable for use in wafer defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a wafer defect detection method based on an improved lightweight YOLOv8 model; Figure 2Schematic diagram of the improved YOLOv8 model; Figure 3 This is a schematic diagram of the structure of the C2f_IDR module; Figure 4 Schematic diagram of using an extended small kernel convolutional layer to enhance a non-extended large kernel layer for DRB; Figure 5 This is a schematic diagram of the HSPAN module; Figure 6 Schematic diagram of training visualization results; Figure 7 This is a schematic diagram of wafer defect images. DETAILED DESCRIPTION

[0023] In order to make the advantages and benefits of the technical solution provided by the present invention more clearly reflected, it is now further described in detail with reference to the accompanying drawings, specifically: Embodiment 1: This embodiment provides a lightweight wafer defect detection method, including: A step of acquiring an image data set containing wafer defects as an image set, and preprocessing the image set; A step of performing feature extraction on the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; Based on the output of the YOLOv8 model, identify the location and type of defects in the image; The optimized YOLOv8 model also includes the steps of performing data enhancement by image rotation, scaling, cropping, and color jittering.

[0024] Specifically, the technical solution provided in this embodiment includes: Step 1: Obtain wafer defect image dataset and perform preprocessing First, collect and acquire image datasets containing different wafer defect types. To ensure the diversity of the dataset, the image dataset should include samples of various defect types, different scales, shapes, locations, and other features. The preprocessing stage includes the following key steps: Image slicing: To avoid losing detail information during image scaling, each defect image is segmented into four regions (upper left, upper right, lower left, and lower right). This step can effectively reduce the amount of image calculation while maintaining detail information and providing more local features for subsequent processing.

[0025] Data enhancement: Use a variety of data enhancement techniques, such as rotation, flipping, scaling, and cropping, to increase the number of small sample defects, balance the distribution of various samples in the data set, and enhance the generalization ability of the model.

[0026] Dataset division: The preprocessed dataset is divided into training set, validation set, and test set in a ratio of 7:1.5:1.5 to ensure the accuracy of the training process and the reliability of the test results.

[0027] The output of this step is a preprocessed and enhanced training set, validation set, and test set for subsequent model training.

[0028] Step 2: Build an improved lightweight YOLOv8 model Based on the YOLOv8 model, lightweight improvements are made to adapt to the deployment requirements of edge devices. The specific construction steps are as follows: Backbone network optimization: The improved C2f_IDR module proposed in this invention is used to replace the original C2f module of YOLOv8 to optimize the feature extraction process. The C2f_IDR module reduces redundant calculations and parameters by combining the inverted residual moving block (iRMB) and the dilated reparameterization module (DRB), while improving the feature extraction efficiency.

[0029] Feature extraction: The improved C2f_IDR module includes multiple Conv modules, IDR Block and SPPF modules, which realize efficient extraction of image features through layer-by-layer connection.

[0030] Output: The output of this step is an optimized feature map with lower computational complexity and higher feature extraction capability, which serves as the input of the subsequent network.

[0031] Neck network optimization: In order to reduce redundant information and enhance the ability to extract fine-grained features, the present invention designs the HSPAN module to replace the traditional FPN module in YOLOv8. HSPAN uses a bidirectional path aggregation mechanism to effectively filter low-level features using high-level semantic information and inject low-level fine-grained features into high-level features, thereby enhancing the expressiveness of high-level features.

[0032] Feature fusion: The HSPAN module includes a channel attention mechanism (CA module), a convolution module, and a selective feature fusion module (SFF). These modules work together to improve the network's detail processing capabilities without increasing the computational burden.

[0033] Output: The output of this module is an optimized and fused feature map with efficient feature expression capabilities for use by the next stage detection module.

[0034] Head detection module optimization: In order to improve the detection accuracy of complex shape and small target defects, this paper introduces the Focaler-Shape IoU loss function to replace the traditional loss function in YOLOv8. This loss function combines the advantages of Focaler-IoU and Shape-IoU, and is particularly suitable for the regression tasks of small targets and complex shape defects.

[0035] Bounding Box Regression: The Focaler-Shape IoU loss function improves the accuracy of small object detection through a weighted mechanism, ensuring that the model can fit the bounding box more accurately.

[0036] Output: The detection box optimized by the loss function is output as the final defect detection result.

[0037] The output of this step is an optimized YOLOv8 model with low computational overhead and high-precision defect detection capabilities.

[0038] Step 3: Model training and validation The improved lightweight YOLOv8 model is trained on the training set and verified in real time using the validation set to ensure the effectiveness and robustness of the model. The training steps include: Training settings: Set the training rounds to 300 epochs and the batch size to 128. During the training process, the conventional stochastic gradient descent (SGD) or Adam optimizer is used to optimize the loss function to minimize the model prediction error.

[0039] Validation set evaluation: After each epoch, the validation set is used to evaluate the performance of the model, and indicators such as F1 Score and mAP are calculated to ensure the detection accuracy of the model on small objects and complex shape defects.

[0040] Parameter tuning: Based on the verification results, adjust hyperparameters such as learning rate and weight decay to further improve model performance.

[0041] The output of this step is a trained lightweight YOLOv8 model with high detection accuracy and low computational complexity.

[0042] Step 4: Model testing and defect detection The trained model is applied to the test set to perform wafer defect detection. The test steps are as follows: Input test set images: Input the wafer defect images in the test set into the trained lightweight YOLOv8 model. The model predicts each image and outputs the detection box and defect category information.

[0043] Result evaluation: Use evaluation indicators such as F1 Score and mAP50 to comprehensively evaluate the performance of the model on the test set to ensure the accuracy and real-time performance of the model in practical applications.

[0044] Performance comparison: The improved YOLOv8 model is compared with the original unoptimized YOLOv8 model to evaluate the improvement in detection accuracy, number of parameters, and amount of computation.

[0045] The output of this step is the inspection results and their evaluation report, which shows the actual effect of the model in wafer defect detection.

[0046] Embodiment 2: This embodiment further limits the lightweight wafer defect detection method provided in Embodiment 1, and the preprocessing includes image slicing and data enhancement.

[0047] Implementation method three: This implementation method further limits the lightweight wafer defect detection method provided in implementation method one, and the C2f_IDR module includes an inverted residual module and an expansion reparameterization module.

[0048] Embodiment 4: This embodiment further limits the lightweight wafer defect detection method provided in embodiment 1. The HSPAN module fuses the low-level and high-level features in the image set through a bidirectional path aggregation mechanism.

[0049] Implementation method five: This implementation method further limits the lightweight wafer defect detection method provided in implementation method one, and the loss function combines Focaler-IoU and Shape-IoU.

[0050] Embodiment 6: This embodiment provides a lightweight wafer defect detection device, including: A module for acquiring an image data set containing wafer defects as an image set and preprocessing the image set; A module for extracting features from the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; A module that identifies the location and type of defects in an image based on the output of the YOLOv8 model; The optimized YOLOv8 model also includes a module for data enhancement through image rotation, scaling, cropping and color jittering.

[0051] Embodiment 7: This embodiment provides a computer storage medium for storing a computer program. When the computer reads the computer program, the computer executes the method provided in embodiment 1.

[0052] Embodiment 8: This embodiment provides a computer, including a processor and a storage medium. When the processor reads a computer program stored in the storage medium, the computer executes the method provided in embodiment 1.

[0053] Embodiment 9: This embodiment provides a computer program product, which is a computer program. When the computer program is executed, the method provided in embodiment 1 is implemented.

[0054] Implementation Method 10: Combination Figure 1-7 This embodiment is described in detail by using specific examples to further describe the technical solution provided above. Specifically: The YOLOv8 model consists of three parts: Backbone network, Neck network and Head network. This implementation method is based on the YOLOv8 model, and comprehensively reconstructs and improves the Backbone network, Neck network and Head network. Figure 2 This is a diagram of the improved model structure.

[0055] In order to reduce the computational burden and parameter scale problems caused by redundant operations in the Backbone network during feature extraction, this implementation uses an improved C2f_IDR module to optimize the original C2f module. The C2f_IDR has a built-in innovative lightweight IDR block structure, which not only improves processing efficiency and reduces resource consumption, but also significantly enhances the ability to identify and extract multi-scale defect features. After optimization, the Backbone backbone network includes: the first Conv module, the second Conv module, the first C2f_IDR module, the third Conv module, the second C2f_IDR module, the fourth Conv module, the third C2f_IDR module, the fifth Conv module, the fourth C2f_IDR module and the first SPPF module are connected in sequence from top to bottom.

[0056] Combination Figure 3 In order to solve the problem that the Neck network in the feature fusion step uses the layer-by-layer transmission and point-by-point summation strategy, which increases the computational pressure and may weaken the sensitivity to subtle features in the wafer defect detection task, this implementation method reconstructs the original neck network through the HSPAN (High-level Screening Path Aggregation Network) bidirectional high-level screening path aggregation module, uses the channel attention mechanism to accurately screen key features, reduces the transmission of redundant information, and uses high-level semantic information to effectively filter low-level features. At the same time, the fused low-level features containing rich fine-grained position and edge detail information are integrated into the high-level features to enhance the expression ability of the high-level features, thereby significantly reducing the computational complexity and parameter quantity of the network.

[0057] In the Head part, due to the high proportion of small targets and complex-shaped defects in wafer defects and the difficulty of detection, this implementation uses the Focaler-Shape-IoU loss function instead of the original bounding box regression loss function. This loss function combines the ideas of Focaler-IoU and Shape-IoU, enhances the model's adaptive attention to small defects that are difficult to detect, and improves the regression accuracy of complex-shaped defect boundaries, so that the model is lightweight while still having high-precision defect detection performance. The Head network includes: a first Detect module, a second Detect module, and a third Detect module; the first C2f is connected to the first Detect module, the third C2f module is connected to the second detect module, and the fourth C2f module is connected to the third Detect module.

[0058] The original C2f module in YOLOv8n can significantly improve the feature extraction capability by increasing the number of Bottleneck structures. However, Bottleneck achieves dimensionality reduction and dimensionality increase through a combination of 1x1, 3x3 and 1x1 convolutions. When 3x3 convolution processes features in the middle layer, each output channel needs to be convolved with all input channels, resulting in a sharp increase in the amount of calculation as the number of channels increases. At the same time, the dimensionality increase process will cause channel information redundancy, making it difficult to deploy on resource-constrained devices.

[0059] This implementation innovatively combines the lightweight inverted residual move (iRMB) module and the dilated reparameterized DRB module, which not only reduces the resource consumption of the model, but also further improves the feature extraction capability. The core idea of ​​iRMB is the fusion of the inverted residual module (IRB) and the efficient multi-head attention (Effificient Window-MHSA, EWMHSA) module. Since the multi-head attention mechanism has a large amount of computation, this implementation optimizes the attention part in iRMB, and uses the dilated reparam block (DRB) to replace the original multi-head attention mechanism, designs a more lightweight IDR Block module, and uses it to replace the Bottleneck module in the C2f module to form a new structure C2f-IDR. The C2f_IDR module is as follows Figure 3 The specific technical solution is as follows: The C2f_IDR module includes: a sixth Conv module, a Split module, an n-layer IDR Block module and a seventh Conv module connected in sequence; the output ends of the sixth Conv module, the Split module and the n-layer IDR Block module are spliced ​​and connected to the seventh Conv module.

[0060] The IDR Block module includes: the input end is respectively connected to the first 1*1 Conv module and the first DRB module; the first 1*1 Conv module, the first DW_Conv module, and the second 1*1Conv module are connected in sequence; the output end of the first 1*1 Conv module is multiplied by the output end of the first DRB module, and then connected to the first DW_Conv module and added to the output end of the first DW_Conv module; the input end is added and outputted with the second 1*1 Conv module.

[0061] The DRB module contains a 9*9 large kernel convolution and 5 parallel small kernel convolution layers with different expansion rates, such as Figure 4 As shown in the figure, the input defect image features are processed by parallel small-kernel convolution layers with kernel sizes of 9, 5, 5, 3, and 3 and dilation rates of 1, 1, 2, 3, and 4, respectively, and then reparameterized with batch normalization layers to obtain a 9*9 non-dilated large-kernel convolution.

[0062] The IDR-Block module can effectively combine the advantages of the two structures by combining DW-Conv and DRB in parallel: DW-Conv focuses on capturing local features, extracting rich feature information while keeping the amount of computation low. DRB expands the receptive field through dilated convolution, allowing the model to capture a wider range of contextual and multi-scale information. Using C2f-IDR for feature extraction and processing in wafer defect detection tasks can reduce the number of parameters and computation while maintaining good robustness and accuracy for features of different scales and details.

[0063] After the IDR-Block module is improved, the specific processing flow of defect detection image data is as follows: First, the detection feature map is processed in two ways: one is mapped to the identity map, retaining the original features for subsequent residual connection; the other enters the dilation reparameterization module (DRB) for multi-scale feature extraction and global context modeling. Convolution with a dilation rate of 1 is used to extract local detail features, while convolution with gradually increasing dilation rates is used to obtain a larger receptive field to adapt to defect patterns of different scales.

[0064] Then, the outputs of these parallel dilated convolutional layers are fused (by element-by-element addition or cascading) and optimized by reparameterization combined with the parameters of the Batch Normalization layer during the training phase. During the inference phase, the fused convolution kernel weights are simplified to a large kernel convolution W', thereby reducing the amount of computation and improving inference efficiency. The original feature map is multiplied and fused with the output of the DRB by increasing the number of channels through a 1*1 convolution.

[0065] Secondly, a 3×3 depthwise convolution (DW-Conv) is applied to extract spatial features. In order to maintain the integrity of the information, a skip connection is used to add the feature maps before and after the depthwise convolution to achieve a residual connection.

[0066] Finally, channel shrinkage is performed through 1*1 convolution to ensure that the number of input and output channels is consistent. The output of the feature extraction module is added in parallel with the output of the identity mapping branch to obtain the final output of the IDR Block module.

[0067] The original neck feature fusion module FPN in YOLOv8n realizes the transfer of deep semantics to shallow layers, while PANet strengthens the interaction between shallow and deep layers. However, the layer-by-layer transfer and point-by-point summation methods increase the computational burden and affect the recognition of subtle features in wafer defect detection.

[0068] In view of the above problems of the Neck network, this implementation is optimized into a self-created HSPAN module. Through advanced screening and bidirectional path aggregation mechanism, high-level semantic information is used to filter low-level redundant information, and the filtered low-level features rich in fine-grained position information and edge details are integrated into high-level features to enhance the subtle expression ability, thereby effectively reducing the computational complexity of the network.

[0069] The HSPAN module includes: from left to right, the first CA module, the first Conv2d module, the second CA module, the second Conv2d module, the first SFF_2 module, and the first C2f module are connected in sequence; from left to right, the third CA module, the third Conv2d module, the first SFF_1 module, the second C2f module, the fourth CA module, the fourth Conv2d module, the second SFF_2 module, and the third C2f module are connected in sequence; from left to right, the fifth CA module, the fifth Conv2d module, the second SFF_1 module, and the fourth C2f module are connected in sequence; the output end of the first SPPF is connected to the first CA module, the output end of the third C2f_IDR module is connected to the third CA module, the output end of the second C2f_IDR module is connected to the fifth CA module, the output end of the first Conv2d is connected to the first SFF_1 and the second SFF_1 modules respectively, and the fourth C2f module is connected to the second SFF_2 module and the first SFF_2 module respectively.

[0070] The HSPAN module consists of two parts: feature selection and feature fusion. The detailed structure is as follows: Figure 5 As shown in Figure 1. First, the feature selection module is used to filter feature maps of different scales, and the low-level important features are filtered through high-level semantic information to complete the initial fusion. Then, the fused features are filtered again, and the low-level fine-grained information is injected into the high-level features to reduce the transmission and fusion of redundant information.

[0071] Feature selection module: In the feature selection process, channel attention (CA) and 1*1 convolution are used to screen and unify the dimensions of feature maps of each scale. The CA module uses global average pooling and global maximum pooling to extract representative information of each channel. The combination of the two pooling methods can minimize information loss and focus on key features. After the pooling operation calculates the representative weight of the channel, the weight of each channel is generated by the sigmoid activation function and multiplied with the original feature map to obtain the enhanced feature map. Finally, the number of channels of feature maps of different scales is unified through 1×1 convolution to ensure the consistency of feature fusion.

[0072] In the feature fusion stage, (HSPAN) uses the upstream selective feature fusion (SFF_1) mechanism to use high-level features as weights to filter important semantic information embedded in low-level features. Next, HSPAN first screens and processes the fused feature maps to complete the second feature selection, and then uses the downstream selective feature fusion (SFF_2) mechanism to further enrich the expression of high-level features using the fine-grained position information and edge details in the low-level features.

[0073] The principles of the uplink selective feature fusion mechanism SFF_1 and the downlink selective feature fusion mechanism SFF_2 are as follows: The SFF_1 module includes high-level feature and low-level feature input terminals. The high-level features are connected to the first ConvTansposed module and the seventh CA module in sequence. The low-level features are multiplied by the seventh CA module and then added to the output of the first ConvTansposed module to obtain the final output features.

[0074] First, a transposed convolution (T-Conv) with a stride of 2 and a convolution kernel of 3×3 is used to upsample the deep features to unify the dimensions of the deep features and the shallow features. Then, the CA module converts the deep features into corresponding attention weights, which are multiplied by the shallow features to be fused to filter out important semantic information in the shallow features. Finally, the filtered shallow features are added to the deep features to obtain a preliminary fusion of the shallow and deep features.

[0075] The feature fusion process is as follows: First Transpose, then convert it into corresponding weights through the CA module, and then Multiply, filter the semantic information of shallow features, and finally add it to the transposed Add point by point, As shown in formula (1): (1) It is a shallow feature. .

[0076] The SFF_2 module includes high-level feature and low-level feature input terminals. The low-level features are connected to the sixth Conv2d module and the sixth CA module in sequence. The high-level features are multiplied by the sixth CA module and then added to the output of the sixth Conv2d module to obtain the final output features.

[0077] First, the shallow features are downsampled through the standard 3×3 strided convolution, and then the CA module injects the shallow features rich in fine-grained information into the high-level features to further enhance the expressiveness of the deep features. Finally, the fused result is added to the downsampled shallow features. The feature fusion process is as follows: Replace the transposed convolution with the standard strided convolution Conv, As shown in formula (2): (2) Conv is a standard strided convolution.

[0078] By designing the bidirectional feature screening aggregation path mechanism of HSPAN, the number of parameters and the amount of calculation of the model are effectively reduced. HSPAN uses channel attention (CA) to weightedly screen channels, which can effectively remove redundant or irrelevant channels, thereby reducing the computational complexity. The selective feature fusion mechanism (SFF_1 and SFF_2) more efficiently uses high-level feature information to filter low-level redundant features, and injects low-level fine-grained information into high-level features, further enhancing the expressive power of high-level features, avoiding the introduction of irrelevant features, and reducing unnecessary calculations and redundant parameters. In addition, HSPAN aligns feature maps of different scales through transposed convolution and bilinear interpolation instead of directly using a large number of convolution operations to adjust the size of feature maps. This approach effectively reduces the consumption of computing resources while retaining the effective information between high-level and low-level features. Overall, HSPAN significantly improves computing efficiency while meeting the accuracy requirements of wafer defect detection, and is particularly suitable for deployment and application in resource-constrained environments.

[0079] YOLOv8's built-in bounding box loss function CIOU only considers the overlapping area, center point distance and aspect ratio, which is difficult to effectively deal with the common problems of complex shapes and small sizes in wafer defect detection. It also ignores the weighted processing of difficult-to-detect samples and the accuracy optimization of bounding box shape regression.

[0080] This implementation combines Focaler-IoU and Shape-IoU, using Focaler-Shape IoU as the bounding box loss function of the model. Focaler-IoU can increase the loss weight for samples with low IoU, allowing the model to better handle small targets and complex defects that are difficult to detect. At the same time, Shape-IoU introduces considerations for shape matching to ensure that the predicted box is not only close to the real box in terms of area and center point, but also better fits the target in shape. Focaler-Shape-IoU can take into account both the positioning accuracy and shape matching of the bounding box, and can also make up for the accuracy loss caused by lightweight improvements.

[0081] The Focaler-Shape IOU formula is as follows:

[0082] In the formula, IOU is the traditional intersection-over-union ratio. The higher its value, the greater the overlap between the predicted box and the true box, that is, the closer the predicted result is to the true value; [d,u]∈[0,1] is used to adjust the behavior of Focaler-IoU. By adjusting the values ​​of d and u, we can focus on different regression samples. The two parameters d and u are introduced to increase the loss weight for low IoU samples, thereby improving the model's ability to detect small targets and complex defects; B represents the predicted box, represents the real box; ww and hh represent the weight coefficients in the horizontal and vertical directions respectively, and their values ​​are related to the shape of the GT box. and is the width and height of the real box. The introduction of weight coefficient helps to better balance the importance of width and height in the loss function. scale is a scaling factor used to adjust the size differences of targets in different data sets. represents the distance term that takes the shape factor of the box into account, and is the coordinate of the center of the prediction box, and is the coordinate of the center of the real box, c is the diagonal length of the minimum bounding rectangle containing the predicted box and the real box, Adjust the distance metrics in the horizontal and vertical directions by weights hh and ww; Represents the shape loss term, which is used to measure the difference in width and height between the predicted box and the true box. The introduction of the exponential term can impose a greater loss on the predicted box with a large shape difference, so as to better match the shape of the true box. is a fixed scale factor (usually 4w); ω and h represent the width and height of the prediction box. and are the shape loss factors for width and height, respectively. The relative error between the predicted box and the real box in width and height is calculated and adjusted by weights hh and ww to comprehensively consider the shape ratio of the real box. Finally, It is a comprehensive loss function that takes into account both the positioning accuracy and shape matching of the bounding box.

[0083] By introducing Focaler-IoU, distance term and shape loss term, this loss function can improve the performance of the model in small target and complex defect detection, especially after the lightweight improvement of the YOLOv8 backbone and neck network, which makes up for the accuracy loss that may be caused by model simplification.

[0084] In this embodiment, the improved lightweight YOLOv8 model is trained using the training set, and during the training process, the effectiveness of the improved lightweight YOLOv8 model is verified using the verification set.

[0085] Training process: Initialize various parameters of the improved lightweight YOLOv8 model; input the wafer defect image in the preprocessed wafer defect image dataset into the improved lightweight YOLOv8 model for recognition, and obtain category information and detection frame; input the above information together with the annotation information of the defect image into the optimized loss function to calculate the loss value.

[0086] The training rounds were set to 300 epochs and the batch size was 128, and the trained improved lightweight YOLOv8 model was obtained. The accuracy of the improved detection network was evaluated by F1 Score (F1 score) and mAP (average precision), and the complexity and size of the model were measured by the number of parameters (Para) and the amount of calculation (GFLOPs). The training results of the improved YOLOv8 detection network were compared with the original network without improvement. The results show that the lightweight YOLOv8 model shows better accuracy and real-time performance in wafer defect detection, as shown in the table below.

[0087]

[0088] From the table data and visualization results Figure 6It can be seen that the introduction of the C2f-IDR module significantly improves the detection performance, with F1 Score and mAP increased by 1.4% and 1.1% respectively. At the same time, the number of parameters (Params) is reduced by 11.6%, and the amount of calculation (GFLOPs) is reduced by 9.9%, achieving the effect of model lightweighting while improving accuracy. After replacing the Neck part of YOLOv8 with the HSPAN module, although the F1 Score and mAP decreased by 0.9% and 0.8% respectively, the number of parameters was reduced by 31.1% and the amount of calculation was reduced by 12.3%, which performed well in lightweighting. After combining C2f-IDR and HSPAN, compared with using only the C2f-IDR module, the number of parameters and the amount of calculation were reduced by 37% and 13.7% respectively, while the accuracy (F1 Score and mAP) was improved by 0.5% compared with using HSPAN alone. This shows that the combined application of the two modules can achieve a significant lightweight effect while maintaining high accuracy. The experimental results of Group E show that the introduction of the Focaler-Shape-IoU loss function can significantly compensate for the loss of accuracy caused by HSPAN lightweighting. After the final comprehensive application of the three improvement methods of C2f-IDR, HSPAN and Focaler-Shape-IoU, the model achieved the best balance between lightweighting and accuracy, with the number of parameters and the amount of calculation reduced by 43.3% and 22.2% respectively, and the F1 Score and mAP50 also increased by 0.6% and 0.7% respectively. This combination successfully reduced the complexity of the model while improving the detection accuracy, showing excellent performance optimization effects.

[0089] To further verify the effect of the model, this embodiment selects multiple types of wafer defect images from the test set, covering the diversity of defects in scale, shape and location. Next, this embodiment uses the baseline model and the optimization model proposed in this embodiment to generate corresponding heat maps, so as to more intuitively compare the difference in detection performance between the two.

[0090] The model proposed in this embodiment shows significant advantages in dealing with various types of defects. For small defect samples that are difficult to identify, the activation of the improved lightweight YOLOv8 model is concentrated on the defect location, with higher brightness and clarity, reflecting its accuracy in detail processing. When dealing with irregularly shaped defects, the model effectively reduces the activation response of irrelevant areas, thereby more accurately aligning the actual defect area. In summary, the improved lightweight YOLOv8 model improves the defect positioning accuracy and edge recognition capabilities through structural optimization, and also has significant advantages in suppressing background noise and enhancing anti-interference capabilities. These improvements lay the foundation for its high-precision defect detection in complex scenarios and demonstrate its application potential in actual embedded deployments.

[0091] Process such as Figure 1 As shown, a wafer surface defect detection method based on an improvement of YOLOv8 includes the following steps: Step 1: Obtain a wafer defect image dataset and preprocess the wafer image dataset. Slice the defect image into four regions in the upper left, upper right, lower left, and lower right order to avoid losing details when directly input into the YOLOv8 network due to image scaling, and to reduce the amount of calculation and improve the model inference speed. Use a variety of data enhancement techniques to enrich the number of small sample defects and balance the distribution of various samples in the dataset. Divide the preprocessed dataset into training set, validation set, and test set in a ratio of 7:1.5:1.5.

[0092] Step 2: Build an improved lightweight YOLOv8 model; based on the YOLOv8 model, introduce the lightweight IDR Block module designed to improve the multi-scale defect feature extraction capability, the bidirectional advanced screening path aggregation module HSPAN designed to reduce redundant transmission in feature fusion, and the Focaler-Shape IoU loss function added to improve the accuracy of complex boundary regression and adaptive weight adjustment for difficult-to-detect samples; improve the lightweight YOLOv8 model such as Figure 2 shown.

[0093] Step 3: Use the training set to train the improved lightweight YOLOv8 model, and use the validation set to verify the effectiveness of the improved lightweight YOLOv8 model during the training process to obtain a trained improved lightweight YOLOv8 model; Step 4: Input the test set into the trained improved lightweight YOLOv8 model to obtain the defect detection results. Figure 7 This is a schematic diagram of wafer defect images.

[0094] The technical solution provided by the present invention is further described in detail above through several specific implementation modes in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the several specific implementation modes described above are not intended to be used as limitations on the present invention. Any reasonable modification and improvement of the present invention, combination of implementation modes and equivalent substitution within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A lightweight wafer defect detection method, characterized in that: include: A step of acquiring an image data set containing wafer defects as an image set, and preprocessing the image set; A step of performing feature extraction on the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; Based on the output of the YOLOv8 model, identify the location and type of defects in the image; The optimized YOLOv8 model also includes the steps of performing data enhancement by image rotation, scaling, cropping, and color jittering.

2. A lightweight wafer defect detection method according to claim 1, characterized in that: The preprocessing includes image slicing and data enhancement.

3. The lightweight wafer defect detection method according to claim 1, characterized in that: The C2f_IDR module includes an inverted residual module and a dilation reparameterization module.

4. The lightweight wafer defect detection method according to claim 1, characterized in that: The HSPAN module fuses low-level and high-level features in the image set through a bidirectional path aggregation mechanism.

5. The lightweight wafer defect detection method according to claim 1, characterized in that: The loss function combines Focaler-IoU and Shape-IoU.

6. A lightweight wafer defect detection device, characterized in that: include: A module for acquiring an image data set containing wafer defects as an image set and preprocessing the image set; A module for extracting features from the preprocessed image set using an optimized YOLOv8 model, wherein the optimized YOLOv8 model includes a preset Backbone network, a Neck network, and a Head detection module; Using the C2f_IDR module in the preset Backbone network to extract features from the image set; Using HSPAN module to perform feature fusion in the Neck network; Using the Focaler-Shape IoU loss function for bounding box regression in the Head detection module; A module that identifies the location and type of defects in an image based on the output of the YOLOv8 model; The optimized YOLOv8 model also includes a module for data enhancement through image rotation, scaling, cropping and color jittering.

7. A computer storage medium for storing a computer program, characterized in that: When the computer reads the computer program, the computer executes the method of claim 1.

8. A computer, comprising a processor and a storage medium, characterized in that: When the processor reads the computer program stored in the storage medium, the computer executes the method of claim 1 .

9. A computer program product, being a computer program, characterized in that When the computer program is executed, the method of claim 1 is implemented.

Images