Multi-stage automatic detection method, system and equipment for smudginess and / or metal defects of laser chip and medium

Abstract

The invention relates to a multi-stage automatic detection method, system and device for smudginess and / or metal defects of a laser chip and a medium, and the method comprises the steps: carrying out the preprocessing of chip image data; performing coarse positioning on the preprocessed chip image by using an improved Yolov7 network, and determining and adaptively adjusting a size threshold in the coarse positioning process; if the size of the detection target is greater than the size threshold value, directly outputting a detection result of the coarse positioning stage; if the size of the detection target is not greater than the size threshold value, entering a second stage of processing, and performing fine classification by selecting a MobileNet classification network to obtain a detection result of a fine classification stage; and outputting a final judgment result according to detection results in the coarse positioning stage and the fine classification stage. According to the method, the processing advantages of all stages are fully fused, the problems of detection and classification of two types of tiny targets including dirt and metal defects are solved, and powerful support is provided for quality control of laser chips.

Description

Technical Field

[0001] The present invention relates to the field of chip detection technology, and in particular to an automatic detection method for dirty metal defects of laser chips based on convolutional neural networks. Background Art

[0002] In the field of laser chip defect detection, the diversity of chip types and the complexity of chip appearance have brought significant challenges to the detection work. In particular, the size of laser chip defects is small, and the contrast difference with the background area is also small, which makes the accurate detection of defects particularly difficult. At the same time, due to the instability of chip processes, there are many types of defects, and the characteristic differences between different defects are subtle. Even the same type of defects may show a variety of different characteristic manifestations. These problems are particularly prominent when detecting dirt and metal defects with extremely small sizes and small characteristic differences, which significantly increases the difficulty of detection and classification.

[0003] At present, defect detection in the field of laser chips mainly relies on manual work. However, manual detection methods have many shortcomings, such as low detection efficiency, poor accuracy, low reliability, and high cost. In addition, the results of manual detection are often affected by the operator's work experience and working status, which is highly subjective and will greatly reduce work efficiency and accuracy after long-term labor, which not only increases labor costs, but also may lead to a large number of missed detections and false detections.

[0004] In recent years, with the rapid development of machine learning and deep learning, image processing technology has been widely used in the automatic identification of laser chip defects. Compared with traditional machine learning methods that require manual feature extraction, deep learning, especially convolutional neural networks, can automatically extract effective and advanced semantic information, significantly improving the detection rate of laser chip defects and reducing the false detection rate. However, even methods based on deep learning still face certain challenges when faced with a wide variety of chip defects with subtle feature differences.

[0005] In particular, the detection and differentiation of dirt and metal defects has always been a difficult problem in the industry. Dirt usually presents a black feature, while the characteristics of metal defects are more complex and changeable. For extremely small defects (such as less than 20um), the difference in their characteristics may only be reflected in a few pixel values, which brings great difficulties to manual or machine judgment. Therefore, although deep learning has made significant progress in the field of image recognition, there are still many challenges and problems to be solved in laser chip defect detection, especially the detection and classification of small targets. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a multi-stage automatic detection method, system, equipment and medium for laser chip dirt and / or metal defects, which solves the technical problems existing in the existing laser chip defect detection, namely, the small size of chip defects, small contrast difference with the background, many types and subtle feature differences, and the difficulty in detecting and distinguishing dirt and metal defects.

[0008] (II) Technical solution

[0009] In order to achieve the above object, the main technical solutions adopted by the present invention include:

[0010] In a first aspect, an embodiment of the present invention provides a multi-stage automatic detection method for laser chip dirt and / or metal defects, comprising:

[0011] Preprocessing the acquired chip image data;

[0012] Using the improved Yolov7 network to perform coarse localization of dirt and / or metal defects on the preprocessed chip image, and determining and adaptively adjusting the size threshold for distinguishing the size of the detection target during the coarse localization process;

[0013] If the size of the detection target is larger than the size threshold, the detection result of the rough positioning stage is directly output as the final judgment result of dirt and metal defects on the chip;

[0014] If the size of the detected target is not greater than the size threshold, the second stage of processing is entered, and a lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage;

[0015] Based on the detection results in the two stages of coarse positioning and fine classification, the final judgment result of dirt and / or metal defects on the chip is output.

[0016] Optionally, preprocessing the acquired chip image data includes:

[0017] Acquire chip image data, and identify the aspect ratio of the acquired chip image;

[0018] If the chip image aspect ratio is greater than a preset threshold, the chip image is cropped into multiple segments so that the size of each segment approaches a 1:1 ratio;

[0019] Perform color space conversion on the cropped chip image, converting the image from RGB space to HSV color space;

[0020] Extract the V channel image in the HSV color space and calculate the mean pixel value of the V channel image;

[0021] Based on the preset step length, the number of times of image amplification is determined based on the amount of data;

[0022] Determine the number of times of image amplification according to the amount of data to generate images with different brightness values;

[0023] Identify and separate metal defect areas in images with different brightness values;

[0024] For the metal defect area, the data volume expansion operation including rotation and flipping is performed in the image dimension, and the local interest map of the metal defect is selected in the feature dimension, and adaptive size transformation and feature fusion processing are performed to obtain the processed metal defect feature sub-map;

[0025] The processed metal defect feature sub-image is merged onto the background sub-image by pasting the image, and the number of the pastings is determined according to the dirtiness ratio of the background image;

[0026] The metal defects pasted on the background image are processed by filling the defect edges with pixels at corresponding positions of the background image, and first performing image blurring processing on the defect edges and then performing image enhancement.

[0027] Optionally, using an improved Yolov7 network to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image, and determining and adaptively adjusting a size threshold for distinguishing the size of the detection target during the coarse positioning process includes:

[0028] YOLOv7 is selected as the basic model, and the improved Yolov7 network is constructed and trained by replacing some convolution modules with Ghost convolution and introducing FocalLoss loss function;

[0029] Input the preprocessed chip image data into the trained improved Yolov7 network;

[0030] The improved Yolov7 network backbone feature extraction part is used to extract features and obtain feature layers of multiple scales.

[0031] The feature fusion network of the improved Yolov7 network is used to fuse the feature layers of multiple scales from bottom to top and from top to bottom to obtain an enhanced effective feature layer.

[0032] Grid each effective feature layer and use the prior box to predict whether there is a target in the grid, so as to complete the rough positioning of the target;

[0033] In the coarse positioning process, a size threshold for distinguishing detected targets from non-detected targets is determined based on the size distribution of the detected targets;

[0034] The detected targets are screened and classified using the size threshold, and the targets that meet the size requirements are retained and output as the result of coarse positioning;

[0035] By statistically analyzing the size distribution of the detection results in the coarse positioning stage, or automatically adjusting the threshold according to preset rules and conditions, it is ensured that the size threshold can always meet the needs of the current detection task.

[0036] Optionally, the Focal Loss loss function is:

[0037] Focal Loss(p t )=-α t (1-p t )γlog(p t );

[0038] In the formula, p t is the predicted probability value of the model, α t is the weight of the balanced sample, γ is the weight of adjusting the difficult and easy samples, and t is the tth category.

[0039] Optionally, if the size of the detected target is not greater than the size threshold, the second stage of processing is entered, and a lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage including:

[0040] If the size of the detected target is not greater than the threshold, the second stage processing flow is automatically triggered;

[0041] In the second stage of processing, a lightweight MobileNet classification network is selected for fine classification tasks, and the lightweight MobileNet classification network is configured and optimized by adopting an inverted residual structure, a linear bottleneck structure, and a global average pooling instead of a fully connected layer;

[0042] Constructing a weighted cross entropy loss function, where the weight is determined according to the proportion of dirt and / or metal defects in the training data;

[0043] Use the weighted cross entropy loss function to train the MobileNet classification network. Through iterative optimization, the network parameters are continuously adjusted to minimize the loss function value on the training set.

[0044] After the training is completed, the preprocessed chip image data is input into the trained MobileNet classification network, and the deep separable convolution, inverted residual structure, linear bottleneck structure and global average pooling of the MobileNet classification network are used for feature extraction and classification to obtain the detection results of the detailed classification stage.

[0045] Optionally, the weighted cross entropy loss function is:

[0046]

[0047] In the formula, ω c is the weight of each category, N is the total number of targets, N c is the number of targets in each category, y c is the true probability, p c is the predicted probability, c is the index used to identify the category currently being processed, and M is the total number of categories.

[0048] Optionally, according to the detection results in the two stages of coarse positioning and fine classification, the final judgment result of dirt and / or metal defects on the chip is output, including:

[0049] Obtain and parse the target detection results of the coarse positioning stage to obtain the bounding box coordinates, size information and first category labels of the detected dirt and metal defects;

[0050] For detected targets smaller than the size threshold, obtain and parse the target detection results of the fine classification stage to obtain the confidence score and the second category label;

[0051] Align the target detection in the coarse positioning stage and the fine classification stage;

[0052] For each dirt and / or metal defect detected in the coarse localization stage, find a matching class label in the results of the fine classification stage based on the bounding box coordinates and size information;

[0053] If the match is successful, the first category label in the coarse positioning stage is compared with the second category label in the fine classification stage, and the label with higher confidence is selected as the final category label;

[0054] If the match is not successful, the target detection result of the coarse positioning stage is retained;

[0055] Integrate the bounding box coordinates, size information, final class label and confidence score of each defect, sort them in descending order of confidence, and generate a defect list containing all detected dirt and / or metal defects;

[0056] Filter the generated defect list to remove duplicate or redundant detection boxes to ensure that each defect is detected and marked only once;

[0057] Based on the screened defect list, the final judgment results of dirt and metal defects on the chip are output.

[0058] In a second aspect, an embodiment of the present invention provides a multi-stage automatic detection system for laser chip dirt and / or metal defects, comprising:

[0059] A preprocessing module, used for preprocessing the acquired chip image data;

[0060] A coarse positioning module is used to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image using an improved Yolov7 network, and to determine and adaptively adjust a size threshold for distinguishing the size of the detection target during the coarse positioning process; if the size of the detection target is greater than the size threshold, the detection result of the coarse positioning stage is directly output as the final judgment result of dirt and metal defects on the chip;

[0061] The fine classification module is used to enter the second stage of processing if the size of the detection target is not greater than the size threshold. The lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage; based on the detection results in the coarse positioning and fine classification stages, the final judgment results of dirt and / or metal defects on the chip are output.

[0062] In a third aspect, an embodiment of the present invention provides a multi-stage automatic detection device for laser chip dirt and / or metal defects, comprising:

[0063] An image acquisition device, used for acquiring chip image data;

[0064] The controller is connected to the image acquisition device and is configured to execute the multi-stage automatic detection method for laser chip contamination and / or metal defects as described above.

[0065] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, which, when executed by a processor, implements the multi-stage automatic detection method for laser chip contamination and / or metal defects as described above.

[0066] (III) Beneficial effects

[0067] The beneficial effect of the present invention is that the multi-stage automatic detection method for laser chip dirt and / or metal defects provided by the present invention significantly improves the accuracy and efficiency of detection and classification through a series of refined processing processes.

[0068] Specifically, the acquired chip image data is first preprocessed to effectively eliminate noise and interference factors in the image, laying a solid foundation for subsequent processing. Then, the improved Yolov7 network is used to roughly locate dirt and / or metal defects in the preprocessed chip image, and the size threshold is intelligently determined and adjusted in the process, thereby achieving flexible response to targets of different sizes. This coarse positioning mechanism not only improves the detection speed, but also directly outputs accurate results when the target size is large, avoiding unnecessary subsequent processing.

[0069] Furthermore, the present invention adds fine classification processing after the coarse positioning stage. For detection targets whose size is larger than the threshold, the coarse positioning result is directly output, which simplifies the processing flow. When the size of the detection target is not larger than the set size threshold, the present invention will further enter the second stage of processing. In this stage, by selecting a lightweight MobileNet classification network for fine classification, more accurate analysis and judgment can be performed on small targets. This staged processing strategy not only ensures the comprehensiveness of detection and classification, but also effectively balances computing resources and processing efficiency.

[0070] Finally, based on the detection results in the two stages of coarse positioning and fine classification, the present invention can comprehensively output the final judgment result of dirt and / or metal defects on the chip. This comprehensive judgment result fully integrates the processing advantages of each stage, ensuring accuracy while taking into account real-time performance. It not only solves the detection and classification problems of two types of tiny targets, dirt and metal defects, but also provides strong support for the quality control of laser chips, and has significant practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A schematic flow chart of a multi-stage automatic detection method for laser chip contamination and / or metal defects provided by an embodiment of the present invention;

[0072] Figure 2 A specific flow chart of step S1 of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0073] Figure 3 A Yolov7 network structure diagram of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0074] Figure 4 A specific flow chart of step S2 of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0075] Figure 5 A Ghost module diagram of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0076] Figure 6 A specific flow chart of step S3b of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0077] Figure 7 A specific flow chart of step S4 of a multi-stage automatic detection method for laser chip dirt and / or metal defects provided by an embodiment of the present invention;

[0078] Figure 8 A schematic diagram of the effect of locating dirt and metal defects of a multi-stage automatic detection method for dirt and / or metal defects of a laser chip provided by an embodiment of the present invention;

[0079] Fig. 9 A schematic diagram of a contamination area of ​​a multi-stage automatic detection method for contamination and / or metal defects of a laser chip provided by an embodiment of the present invention;

[0080] Fig.10 A schematic diagram of a metal defect area of ​​a multi-stage automatic detection method for laser chip contamination and / or metal defects provided by an embodiment of the present invention;

[0081] Fig.11 A schematic diagram of the overall process of a multi-stage automatic detection method for laser chip contamination and / or metal defects provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0083] like Figure 1 As shown, a multi-stage automatic detection method for laser chip dirt and / or metal defects proposed in an embodiment of the present invention includes: preprocessing the acquired chip image data; using an improved Yolov7 network to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image, and determining and adaptively adjusting the size threshold for distinguishing the size of the detection target during the coarse positioning process; if the size of the detection target is greater than the size threshold, directly outputting the detection result of the coarse positioning stage as the final judgment result of the dirt and metal defects on the chip; if the size of the detection target is not greater than the size threshold, entering the second stage of processing, and performing fine classification by selecting a lightweight MobileNet classification network to obtain the detection result of the fine classification stage; based on the detection results in the two stages of coarse positioning and fine classification, outputting the final judgment result of the dirt and / or metal defects on the chip.

[0084] The multi-stage automatic detection method for laser chip dirt and / or metal defects provided by the present invention significantly improves the accuracy and efficiency of detection and classification through a series of refined processing flows.

[0085] Specifically, the acquired chip image data is first preprocessed to effectively eliminate noise and interference factors in the image, laying a solid foundation for subsequent processing. Then, the improved Yolov7 network is used to roughly locate dirt and / or metal defects in the preprocessed chip image, and the size threshold is intelligently determined and adjusted in the process, thereby achieving flexible response to targets of different sizes. This coarse positioning mechanism not only improves the detection speed, but also directly outputs accurate results when the target size is large, avoiding unnecessary subsequent processing.

[0086] Furthermore, the present invention adds fine classification processing after the coarse positioning stage. For detection targets whose size is larger than the threshold, the coarse positioning result is directly output, which simplifies the processing flow. When the size of the detection target is not larger than the set size threshold, the present invention will further enter the second stage of processing. In this stage, by selecting a lightweight MobileNet classification network for fine classification, more accurate analysis and judgment can be performed on small targets. This staged processing strategy not only ensures the comprehensiveness of detection and classification, but also effectively balances computing resources and processing efficiency.

[0087] Finally, based on the detection results in the two stages of coarse positioning and fine classification, the present invention can comprehensively output the final judgment result of dirt and / or metal defects on the chip. This comprehensive judgment result fully integrates the processing advantages of each stage, ensuring accuracy while taking into account real-time performance. It not only solves the detection and classification problems of two types of tiny targets, dirt and metal defects, but also provides strong support for the quality control of laser chips, and has significant practical application value.

[0088] In order to better understand the above technical solution, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a clearer and more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0089] Specifically, an embodiment of the present invention provides a multi-stage automatic detection method for laser chip dirt and / or metal defects, which includes:

[0090] S1. Preprocessing the acquired chip image data.

[0091] In an embodiment of the present invention, a unique image preprocessing method is designed to address the problems of large differences in aspect ratios of chip images, wide variations in brightness of chip background imaging, small targets of dirt and metal defects, small differences in features between defects, and a small amount of defect data. The method includes a series of image preprocessing operations such as size transformation, brightness transformation, and adaptive enhancement of defect features, so as to expand the brightness range of the image, improve the quality of the image, retain the integrity of the defect information, and increase the amount of defect data.

[0092] Furthermore, if Figure 2 As shown, step S1 includes:

[0093] S11, acquiring chip image data, and identifying the aspect ratio of the acquired chip image.

[0094] S12: If the chip image aspect ratio is greater than a preset threshold, the chip image is cut into multiple segments so that the size of each image segment is close to a 1:1 ratio.

[0095] In practical applications, the aspect ratio of the chip image size is 3:1. During the detection process of the algorithm, the length and width of the image will be scaled proportionally, which will lead to the loss of defect features, especially for targets less than 10 pixels. After the image is compressed, the difference between the dirt and metal defect features is almost lost, and the two types of targets cannot be distinguished. In order to ensure that the aspect ratio of the defect is not distorted, it is better to cut the image into three segments so that the image size is approximately 1:1; the image scaling ratio is reduced, and the defect features of the original image are better preserved.

[0096] S13, performing color space conversion on the cropped chip image, converting the image from the RGB color space to the HSV color space.

[0097] S14, extracting the V channel image in the HSV color space, and calculating the pixel value mean of the V channel image.

[0098] S15. Taking a preset step length as a reference, determining the number of times of image amplification based on the amount of data, and generating images with different brightness values ​​according to the number of times of image amplification determined according to the amount of data.

[0099] Taking into account the wide variety of chips and the fact that the chip process is unstable during production, the thickness and surface roughness of the chips vary, resulting in changes in color and brightness in the imaging. Even for chips from the same batch, there are certain differences in imaging. Therefore, an adaptive brightness transformation technology will be used to enhance the image. The specific implementation includes the following steps: convert the image from RGB space to HSV color space, extract the V channel image, calculate the mean pixel value of the V channel, use 20 pixels as the step size, determine the number of image amplifications based on the specific data volume, and generate images with different brightness values, so as to expand the image brightness range and ensure the color brightness diversity of background imaging, laying a solid foundation for the robustness of subsequent models.

[0100] S16, identifying and separating metal defect areas in images with different brightness values.

[0101] S17. For the metal defect area, a data volume amplification operation including rotation and flipping is performed in the image dimension, and a local interest map of the metal defect is selected in the feature dimension, and adaptive size transformation and feature fusion processing are performed to obtain a processed metal defect feature sub-map.

[0102] S18, merging the processed metal defect feature sub-image onto the background sub-image by pasting the image, and determining the number of the pastings according to the dirtiness ratio of the background image.

[0103] S19, performing edge processing on the metal defect pasted on the background image, including filling the defect edge with pixels at corresponding positions of the background image, and first performing image blurring processing on the defect edge and then performing image enhancement.

[0104] Since dirt is a common defect in the chip production process, its data volume is relatively sufficient and can be seen in almost every chip, while metal defects are caused by special process problems and their frequency of occurrence is relatively low, often due to batches of process problems. During the data collection process, the data of dirt and metal defects are extremely unbalanced, showing a ratio of several hundred to one. Directly modeling the data, the model overwhelmingly identifies metal defects as dirt, resulting in an extremely low detection rate of metal defects.

[0105] In an embodiment of the present invention, the data volume of metal defects is amplified from the image dimension and the feature dimension. In the image dimension, the data containing metal defects is rotated, flipped, and other operations are performed to amplify the metal defect image data while removing the interference caused by the defect position. In the defect feature dimension, all local interest images of metal defects are first found, and then image processing operations such as adaptive size transformation and feature fusion are performed on them. Finally, the obtained defect feature sub-image is pasted on the background sub-image in the form of image pasting. The number of images is determined according to the dirt ratio of the background image to ensure that the data volume ratio of dirt and metal defects is similar; for metal defects pasted on the background image, false defect features will appear. Finally, the edges of the defects will be processed, including filling the defect edges with pixels at the corresponding position of the background image, and the defect edges are first blurred and then enhanced.

[0106] S2. Use the improved Yolov7 network to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image, and determine and adaptively adjust the size threshold used to distinguish the size of the detection target during the coarse positioning process.

[0107] In a specific embodiment, YOLOv7 is used as a baseline model to detect and locate chip dirt and metal defects. YOLOv7 consists of three components: a backbone feature extraction network Backbone, a feature fusion network FPN, and a YOLO classification and regression YoloHead. The principle is that the backbone feature extraction network performs feature extraction, and the obtained features are used as feature layers. The three feature layers are obtained as inputs to the next feature fusion network; the feature fusion network performs bottom-up upsampling and top-down downsampling feature fusion on the three feature layers of different sizes; through the first two networks, three effective feature layers enhanced by feature fusion are obtained, and a grid is drawn for each feature layer. Each grid is used as a feature point, and each feature point has the number of features corresponding to the channel of its feature layer. Three prior frames are used for prediction to determine whether the grid has a target, thereby completing the detection and classification of the target.

[0108] The main optimization points of Yolov7 include the following aspects: model structure reparameterization, dynamic label allocation strategy, ELAN efficient network architecture and training method with auxiliary head. The above improvement strategies achieve a balance between model accuracy and speed while reducing the number of parameters and the amount of calculation. Model structure reparameterization improves the performance of the model by optimizing the structure of different layers in the network; the dynamic label allocation strategy effectively improves the training effect of the model; the ELAN network architecture takes efficiency as the main design goal, and its network structure is designed with more branches and fewer convolution kernels. By optimizing the network structure and the calculation process, the inference speed and detection accuracy of the model are improved; the training method with auxiliary head further improves the performance of the model. Preferably, the present invention will use YOLOv7-tiny as the improved basic model. The network structure of YOLOv7-tiny is as follows Figure 3 shown.

[0109] Furthermore, if Figure 4 As shown, step S2 includes:

[0110] S21. Select YOLOv7 as the basic model, and build and train the improved Yolov7 network by replacing some convolution modules with Ghost convolution and introducing Focal Loss loss function.

[0111] S22, input the preprocessed chip image data into the trained improved Yolov7 network.

[0112] S23. Use the backbone feature extraction part of the improved Yolov7 network to extract features and obtain feature layers of multiple scales.

[0113] S24. The feature fusion network of the improved Yolov7 network is used to fuse the feature layers of multiple scales from bottom to top and from top to bottom to obtain an enhanced effective feature layer.

[0114] S25. Divide each effective feature layer into grids, and use the prior frame to make predictions to determine whether there is a target in the grid, thereby completing the rough positioning of the target.

[0115] S26. During the coarse positioning process, a size threshold for distinguishing detected targets from non-detected targets is determined according to the detected target size distribution.

[0116] S27. Use the size threshold to screen and classify the detected targets, retain the targets that meet the size requirements, and output them as the result of coarse positioning.

[0117] S28. Statistically analyzing the size distribution of the detection results in the coarse positioning stage, or automatically adjusting the threshold according to preset rules and conditions, to ensure that the size threshold can always meet the needs of the current detection task.

[0118] Specifically, the size threshold is adaptively adjusted according to actual detection results, such as false detection rate and missed detection rate. A dynamic adjustment strategy can be adopted during the adjustment process, such as statistical analysis based on the detection results of multiple consecutive frames of images to gradually optimize the size threshold. Finally, a suitable size threshold is determined to distinguish detection targets of different sizes.

[0119] Since a two-stage detection scheme of coarse inspection and fine classification has been designed, the detection of small targets only needs to complete the detection and positioning tasks. In order to better balance the detection efficiency and accuracy of the model, a more lightweight Ghost network is selected as the backbone network for feature extraction. The YOLOv7 network model contains more Conv traditional convolution modules, so as to perform feature extraction, feature fusion and other operations on the input image. However, since there are a large number of redundant feature maps in the traditional convolution, and the feature map is crucial to the accuracy of the model, a large number of network parameters are included in the convolution process, which consumes a lot of computing resources. In the present invention, it is considered to use a lightweight backbone network to replace the original backbone network to further improve the accuracy and efficiency of the model.

[0120] In the present invention, Ghost convolution is used to replace part of the ordinary convolution of YOLOv7. Ghost convolution comes from GhostNet, which is an improved convolutional neural network module that generates virtual mappings to reduce calculation and memory consumption, thereby accelerating the operation of CNN. The Ghost module includes three steps: conventional convolution, Ghost generation and feature map splicing. Its main principle is: divide the ordinary convolution layer in the convolutional neural network into two parts, one part uses ordinary convolution to generate a small number of feature maps, and the other part generates more feature maps through simple transformation. The specific process is that first, a small number of convolution kernels are used to perform convolution operations on the input feature map to generate some feature maps, and the number of feature maps is less than the number of feature maps generated by traditional convolution; then, linear transformations such as matrix multiplication are performed on the generated feature map to obtain more feature maps; finally, the feature map generated by ordinary convolution and the feature map generated after linear transformation are concat operated to obtain the final feature map. The network structure of the Ghost module is as follows Figure 5 shown.

[0121] In a specific embodiment, the loss function of Yolov7 is mainly composed of three parts: the target regression box position loss, the target confidence loss, and the target category classification loss. Since there is a problem of data unevenness in the two categories of dirt and metal defects, in the present invention, the classification loss will be mainly improved. The calculation formula of the overall loss is as follows:

[0122] L all =L conf +L class +L object ;

[0123] Among them, L conf is the target confidence loss, L class is the category classification loss, L object is the target regression box loss.

[0124] Next, the target regression box loss uses the CIOU (complete intersection over union) loss algorithm to calculate the positioning loss of the target regression box. It consists of three parts: the intersection over union ratio of the real box and the predicted box, the aspect ratio of the two boxes, and the distance between the center points of the two boxes. The calculation formula is as follows:

[0125]

[0126] Among them, d is the distance between the center points of the two boxes, c is the diagonal distance of the minimum circumscribed rectangle, ν is the correction factor, considering the direction and shape of the box, α is the influencing factor of ν, and IOU is the intersection-union ratio between the two boxes:

[0127]

[0128] Among them, A and B are the real box and the predicted box respectively.

[0129]

[0130]

[0131] Among them, w G , w p ,h G ,h p They are the width of the real box, the width of the predicted box, the height of the real box, and the height of the predicted box.

[0132] The target confidence loss and classification loss are calculated by the cross entropy loss function, and the calculation formula is as follows:

[0133] L(y,p c )=-(y*log(p c )+(1-y)*log(1-p c ));

[0134] where p c , y are the predicted probability and the true probability respectively.

[0135] In order to improve the problem of unbalanced data volume between the two categories, the present invention improves the classification loss part, uses the Focal Loss loss function, introduces adjustment factors alpha and gamma to adjust the weights of different categories, and makes the model pay more attention to the identification of metal defects in a small number of samples. The calculation formula is as follows:

[0136] FocalLoss(p t )=-α t (1-p t ) γ log(p t );

[0137] In the formula, p t is the predicted probability value of the model, α t is the weight of the balanced sample, γ is the weight of adjusting the difficult and easy samples, and t is the tth category.

[0138] S3a. If the size of the detection target is larger than the size threshold, the detection result of the rough positioning stage is directly output as the final judgment result of dirt and metal defects on the chip.

[0139] S3b. If the size of the detected target is not greater than the size threshold, the second stage of processing is entered, and a lightweight MobileNet classification network is selected for fine classification to obtain the detection result of the fine classification stage.

[0140] In an embodiment of the present invention, MobileNetV2 is used as a classification model to achieve detailed classification of chip dirt and metal defects. Based on V1, MobileNetV2 uses an inverted residual structure and a linear bottleneck structure. The inverted residual structure achieves more efficient feature transfer by swapping the order of dimensionality reduction and dimensionality increase, and replacing the 3x3x3 convolution with a depth-separable convolution. The linear bottleneck reduces the model size and computational complexity by reducing the number of channels. In addition, MobileNetV2 also uses global average pooling instead of the fully connected layer, further reducing the size and computational complexity of the model.

[0141] Furthermore, if Figure 6 As shown, step S3b includes:

[0142] S3b1. If the size of the detected target is not greater than the threshold, the second stage processing flow is automatically triggered.

[0143] S3b2. In the second stage of processing, a lightweight MobileNet classification network is selected for fine classification tasks, and the lightweight MobileNet classification network is configured and optimized by adopting an inverted residual structure, a linear bottleneck structure, and a global average pooling instead of a fully connected layer.

[0144] S3b3. Construct a weighted cross entropy loss function, where the weight is determined according to the proportion of dirt and / or metal defects in the training data.

[0145] S3b4. Use the weighted cross entropy loss function to train the MobileNet classification network. Through iterative optimization, the network parameters are continuously adjusted to minimize the loss function value on the training set.

[0146] S3b5. After the training is completed, the preprocessed chip image data is input into the trained MobileNet classification network, and the deep separable convolution, inverted residual structure, linear bottleneck structure and global average pooling of the MobileNet classification network are used for feature extraction and classification to obtain the detection results of the detailed classification stage.

[0147] In a specific embodiment, for a binary classification task, the loss function used by MobileNet is a cross entropy loss function, and its calculation formula is as follows:

[0148] L(y,p c )=-(y*log(p c )+(1-y)*log(1-p c ));

[0149] where p c , y are the predicted probability and the true probability respectively.

[0150] Based on the characteristics of the image data in the present invention, the data volume of dirt and metal defects is unbalanced. The use of the cross entropy loss function will cause the model's recognition to tend to the category with a larger amount of data. Therefore, the loss function is improved. The weighted cross entropy loss function is used to calculate the data volume ratio of dirt and metal defects through the training set, and a higher weight parameter is given to metal defects. The calculation formula is as follows:

[0151]

[0152] Among them, ω c is the weight of each category, N is the total number of targets, N c is the number of targets in each category, y c is the true probability, p c is the predicted probability, c is the index used to identify the category currently being processed, and M is the total number of categories.

[0153] S4. Output the final judgment result of dirt and / or metal defects on the chip based on the detection results in the two stages of coarse positioning and fine classification.

[0154] Furthermore, if Figure 7 As shown, step S4 includes:

[0155] S41, acquiring and parsing the target detection results of the rough positioning stage, and obtaining the bounding box coordinates, size information and first category labels of the detected dirt and metal defects.

[0156] The purpose of this step is to obtain detailed information about these preliminary inspection results, including bounding box coordinates, size information, and first category labels of the detected dirt and metal defects. The bounding box coordinates define the location of the defect in the image, the size information reflects the size of the defect, and the first category label is the system's preliminary judgment of the defect type.

[0157] S42: For detected targets smaller than the size threshold, obtain and parse the target detection results of the detailed classification stage to obtain a confidence score and a second category label.

[0158] The purpose of this step is to take the results of the fine classification stage and obtain a confidence score and a second class label. This information provides a more accurate analysis and classification of small defects.

[0159] S43, aligning the target detection in the coarse positioning stage and the target detection in the fine classification stage.

[0160] Align the results of the coarse positioning stage and the fine classification stage to ensure that they are performed in the same coordinate system and scale. This requires coordinate transformation or scale normalization.

[0161] S44. For each dirt and / or metal defect detected in the coarse positioning stage, a matching category label is searched in the results of the fine classification stage according to the bounding box coordinates and size information.

[0162] S45. If the match is successful, the first category label of the coarse positioning stage is compared with the second category label of the fine classification stage, and the label with higher confidence is selected as the final category label; if the match is not successful, the target detection result of the coarse positioning stage is retained.

[0163] S46, integrating the bounding box coordinates, size information, final category label and confidence score of each defect, arranging them in descending order of confidence, and generating a defect list containing all detected dirt and / or metal defects.

[0164] S47. Screen the generated defect list to remove duplicate or redundant detection boxes, and ensure that each defect is detected and marked only once.

[0165] S48. Output the final judgment result of dirt and metal defects on the chip according to the screened defect list.

[0166] In one embodiment, during the coarse positioning stage, a target suspected to be a metal defect is detected, whose bounding box coordinates are (100, 100, 200, 200) (coordinates of the upper left corner and the lower right corner), the size information is 100x100 pixels, the first category label is "metal defect", and the confidence score is 0.8.

[0167] In the detailed classification stage, the target suspected of being a metal defect was further analyzed, and the second category label was still “metal defect”, but the confidence score was increased to 0.95.

[0168] According to the above implementation steps:

[0169] First, the results of the coarse positioning stage and the fine classification stage were obtained respectively.

[0170] Secondly, the results of the two stages were aligned.

[0171] Then, the results of the two stages are successfully matched, and the labels and fine-tuning information of the detailed classification stage are selected because of their higher confidence.

[0172] Then, the defect information is consolidated and added to the defect list.

[0173] Furthermore, it was confirmed that there were no duplicate detection boxes in the list.

[0174] Finally, the final judgment result is output, that is, the target is a metal defect, the position is (100, 100, 200, 200), the size is 100x100 pixels, and the confidence score is 0.95.

[0175] In another embodiment, Figure 8 The detection and positioning effect diagram of dirt and metal defects and the classification effect diagram of dirt and metal defects are shown. Please refer to Table 1:

[0176] Table 1 Coarse positioning stage effect table

[0177]

[0178] Among them, large targets are targets with more than 100 pixels, medium targets are targets with more than 50 pixels, and small targets are targets with less than 20 pixels. It can be seen from the above table that the model has a good detection effect for large and medium targets, but for small targets, the distinction between the two types of defects needs to be improved.

[0179] Therefore, it can be found that the improved model provided by the present invention has a high positioning effect for defects of different sizes, which meets the actual detection and positioning effect. For larger dirt and metal defects, it has a strong positioning and classification effect, and for small targets, it has a high detection rate.

[0180] Next, for the abnormal area located in the first stage, the sub-image area is cut out and sent to the classification network to classify the sub-area into dirt and metal defects. The final classification data and effect diagram are shown below. Fig. 9 , Fig.10 As shown in Table 2,

[0181] Table 2 Effect table of subdivision stage

[0182]

[0183] In addition, an embodiment of the present invention provides a multi-stage automatic detection system for laser chip dirt and / or metal defects, comprising:

[0184] The preprocessing module is used to preprocess the acquired chip image data.

[0185] The coarse positioning module is used to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image using the improved Yolov7 network, and to determine and adaptively adjust the size threshold used to distinguish the size of the detection target during the coarse positioning process; if the size of the detection target is larger than the size threshold, the detection result of the coarse positioning stage is directly output as the final judgment result of the dirt and metal defects on the chip.

[0186] The fine classification module is used to enter the second stage of processing if the size of the detection target is not greater than the size threshold. The lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage; based on the detection results in the coarse positioning and fine classification stages, the final judgment results of dirt and / or metal defects on the chip are output.

[0187] At the same time, an embodiment of the present invention provides a multi-stage automatic detection device for laser chip contamination and / or metal defects, including: an image acquisition device for acquiring chip image data; a controller connected to the image acquisition device and configured to execute the multi-stage automatic detection method for laser chip contamination and / or metal defects as described above.

[0188] The present invention introduces an efficient and multifunctional automatic inspection device, which is specially designed for multi-stage inspection of dirt and / or metal defects on laser chips. This innovative device integrates two core components, an image acquisition device and a controller, which work together to achieve an accurate and fast inspection process.

[0189] First, the image acquisition device of the equipment is designed to be able to capture the image data of the laser chip in detail. This link is crucial because it provides the basic and necessary visual information for subsequent inspection and analysis. Through high-resolution image acquisition technology, the equipment can capture every detail of the chip surface, including possible tiny dirt and difficult-to-detect metal defects.

[0190] Secondly, the controller, as the "brain" of the device, is closely connected to the image acquisition device and is responsible for receiving and processing the image data. The controller is configured to execute a specially designed multi-stage automatic inspection method. This method combines advanced image processing algorithms and machine learning technology to intelligently analyze image data and accurately identify dirt and metal defects on the chip.

[0191] The implementation of this multi-stage detection method not only improves the accuracy and efficiency of detection, but also greatly reduces the need for human intervention, thereby achieving truly automated detection. Through the equipment provided by the embodiments of the present invention, laser chip manufacturers can more effectively monitor product quality, promptly discover and solve potential problems, and ultimately improve overall production efficiency and product reliability.

[0192] Furthermore, an embodiment of the present invention provides a computer-readable medium having computer executable instructions stored thereon, wherein when the executable instructions are executed by a processor, the multi-stage automatic detection method for laser chip contamination and / or metal defects as described above is implemented.

[0193] The embodiment of the present invention also innovatively provides a computer-readable medium, which has a unique storage function and stores carefully designed computer-executable instructions. The special feature of these instructions is that when they are executed by a processor, they can activate and implement the multi-stage automatic detection method for laser chip dirt and / or metal defects mentioned above.

[0194] Through this computer-readable medium, users can easily deploy efficient inspection algorithms into various computing environments, whether they are stand-alone inspection systems or automated equipment integrated into production lines. When these instructions are read and executed by the processor, they will guide the system to perform detailed dirt and metal defect inspections on laser chips according to a preset multi-stage process.

[0195] This design not only greatly simplifies the implementation process of the detection method, but also significantly improves the flexibility and portability of the detection method. Whether it is batch detection or real-time monitoring, the computer-readable medium provided by the present invention can provide strong support for the quality assurance of laser chips.

[0196] In summary, the present invention provides a multi-stage automatic detection method, system, device and medium for laser chip dirt and / or metal defects, targeting the differences in size and characteristics of dirt and metal defects, such as Fig.11 As shown, the present invention innovatively proposes a two-stage detection scheme.

[0197] First, considering the inconsistency of the target sizes of dirt and metal defects, the present invention designs the first stage based on the improved Yolov7 detection. This stage mainly performs rough inspection on large-sized dirt and metal defects. By setting and adaptively adjusting the defect size threshold, the system can accurately identify and output defect detection results larger than the threshold.

[0198] For small objects smaller than the size threshold, the present invention introduces a second-stage fine classification mechanism. In this stage, a lightweight MobileNet classification network is used to ensure that small object dirt and metal defects can be effectively distinguished and the final detection results can be obtained.

[0199] In addition, in response to the challenges of a wide variety of laser chips, large size variations, and changing imaging conditions, the present invention also specially designs image size transformation, brightness transformation, and adaptive data enhancement methods. These methods significantly improve the accuracy and robustness of the model under various conditions.

[0200] In the rough inspection target positioning stage, the present invention optimizes the algorithm backbone network and loss function. By adopting the lightweight Ghost network, not only the detection accuracy is guaranteed, but also the detection efficiency is greatly improved. At the same time, in view of the uneven distribution of dirt and metal defect categories, the introduction of the Focal Loss loss function further improves the detection effect of the two categories.

[0201] In general, the multi-stage automatic inspection method of the present invention combines the advantages of rough inspection and fine classification, which not only ensures the comprehensiveness of the inspection, but also realizes efficient and accurate defect identification. This innovative solution provides strong technical support for the quality control of laser chips.

[0202] Since the system / device described in the above embodiments of the present invention is a system / device used to implement the method of the above embodiments of the present invention, a person skilled in the art can understand the specific structure and deformation of the system / device based on the method described in the above embodiments of the present invention, and thus will not be described in detail here. All systems / devices used in the method of the above embodiments of the present invention belong to the scope of protection of the present invention.

[0203] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0204] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions.

[0205] It should be noted that in the claims, any reference numerals placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention may be implemented by means of hardware comprising several different components and by means of a suitably programmed computer. In the claims enumerating several means, several of these means may be embodied by the same hardware. The use of the words first, second, third, etc., is for convenience of expression only and does not indicate any order. These words may be understood as part of the component name.

[0206] In addition, it should be noted that, in the description of this specification, the description of the terms "one embodiment", "some embodiments", "embodiment", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0207] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments after knowing the basic creative concept. Therefore, the claims should be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0208] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention should also include these modifications and variations.

Claims

1. A multi-stage automatic detection method for laser chip dirt and / or metal defects, characterized in that: include: Preprocessing the acquired chip image data; Using the improved Yolov7 network to perform coarse localization of dirt and / or metal defects on the preprocessed chip image, and determining and adaptively adjusting the size threshold for distinguishing the size of the detection target during the coarse localization process; If the size of the detection target is larger than the size threshold, the detection result of the rough positioning stage is directly output as the final judgment result of dirt and metal defects on the chip; If the size of the detected target is not greater than the size threshold, the second stage of processing is entered, and a lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage; Based on the detection results in the two stages of coarse positioning and fine classification, the final judgment result of dirt and / or metal defects on the chip is output.

2. The multi-stage automatic detection method for laser chip dirt and / or metal defects according to claim 1, characterized in that: The preprocessing of the acquired chip image data includes: Acquire chip image data, and identify the aspect ratio of the acquired chip image; If the chip image aspect ratio is greater than a preset threshold, the chip image is cropped into multiple segments so that the size of each segment approaches a 1:1 ratio; Perform color space conversion on the cropped chip image, converting the image from RGB space to HSV color space; Extract the V channel image in the HSV color space and calculate the mean pixel value of the V channel image; Based on the preset step length, the number of times of image amplification is determined based on the amount of data; Determine the number of times of image amplification according to the amount of data to generate images with different brightness values; Identify and separate metal defect areas in images with different brightness values; For the metal defect area, the data volume expansion operation including rotation and flipping is performed in the image dimension, and the local interest map of the metal defect is selected in the feature dimension, and adaptive size transformation and feature fusion processing are performed to obtain the processed metal defect feature sub-map; The processed metal defect feature sub-image is merged onto the background sub-image by pasting the image, and the number of the pastings is determined according to the dirtiness ratio of the background image; The metal defects pasted on the background image are processed by filling the defect edges with pixels at corresponding positions of the background image, and first performing image blurring processing on the defect edges and then performing image enhancement.

3. The multi-stage automatic detection method for laser chip dirt and / or metal defects according to claim 1, characterized in that: The improved Yolov7 network is used to perform coarse positioning of dirt and / or metal defects on the preprocessed chip image, and the size threshold for distinguishing the size of the detection target is determined and adaptively adjusted during the coarse positioning process, including: YOLOv7 is selected as the basic model, and the improved Yolov7 network is constructed and trained by replacing some convolution modules with Ghost convolution and introducing FocalLoss loss function; Input the preprocessed chip image data into the trained improved Yolov7 network; The improved Yolov7 network backbone feature extraction part is used to extract features and obtain feature layers of multiple scales. The feature fusion network of the improved Yolov7 network is used to fuse the feature layers of multiple scales from bottom to top and from top to bottom to obtain an enhanced effective feature layer. Grid each effective feature layer and use the prior box to predict whether there is a target in the grid, so as to complete the rough positioning of the target; In the coarse positioning process, a size threshold for distinguishing detected targets from non-detected targets is determined based on the size distribution of the detected targets; The detected targets are screened and classified using the size threshold, and the targets that meet the size requirements are retained and output as the result of coarse positioning; By statistically analyzing the size distribution of the detection results in the coarse positioning stage, or automatically adjusting the threshold according to preset rules and conditions, it is ensured that the size threshold can always meet the needs of the current detection task.

4. The multi-stage automatic detection method for laser chip contamination and / or metal defects according to claim 3, characterized in that: The FocalLoss loss function is: Focal Loss(p t )−α t (1-p t ) γ log(p t ); In the formula, p t is the predicted probability value of the model, α t is the weight of the balanced sample, γ is the weight of adjusting the difficult and easy samples, and t is the tth category.

5. The multi-stage automatic detection method for laser chip dirt and / or metal defects according to claim 1, characterized in that: If the size of the detected target is not greater than the size threshold, the second stage of processing is entered, and a lightweight MobileNet classification network is selected for fine classification to obtain the detection results of the fine classification stage, including: If the size of the detected target is not greater than the threshold, the second stage processing flow is automatically triggered; In the second stage of processing, a lightweight MobileNet classification network is selected for fine classification tasks, and the lightweight MobileNet classification network is configured and optimized by adopting an inverted residual structure, a linear bottleneck structure, and a global average pooling instead of a fully connected layer; Constructing a weighted cross entropy loss function, where the weight is determined according to the proportion of dirt and / or metal defects in the training data; Use the weighted cross entropy loss function to train the MobileNet classification network. Through iterative optimization, the network parameters are continuously adjusted to minimize the loss function value on the training set. After the training is completed, the preprocessed chip image data is input into the trained MobileNet classification network, and the deep separable convolution, inverted residual structure, linear bottleneck structure and global average pooling of the MobileNet classification network are used for feature extraction and classification to obtain the detection results of the detailed classification stage.

6. The multi-stage automatic detection method for laser chip contamination and / or metal defects according to claim 5, characterized in that: The weighted cross entropy loss function is: In the formula, ω c is the weight of each category, N is the total number of targets, N c is the number of targets in each category, y c is the true probability, p c is the predicted probability, c is the index used to identify the category currently being processed, and M is the total number of categories.

7. The multi-stage automatic detection method for laser chip dirt and / or metal defects according to any one of claims 1 to 6, characterized in that: Based on the detection results in the two stages of coarse positioning and fine classification, the final judgment results of dirt and / or metal defects on the output chip include: Obtain and parse the target detection results of the coarse positioning stage to obtain the bounding box coordinates, size information and first category labels of the detected dirt and metal defects; For detected targets smaller than the size threshold, obtain and parse the target detection results of the fine classification stage to obtain the confidence score and the second category label; Align the target detection in the coarse positioning stage and the fine classification stage; For each dirt and / or metal defect detected in the coarse localization stage, find a matching class label in the results of the fine classification stage based on the bounding box coordinates and size information; If the match is successful, the first category label in the coarse positioning stage is compared with the second category label in the fine classification stage, and the label with higher confidence is selected as the final category label; If the match is not successful, the target detection result of the coarse positioning stage is retained; Integrate the bounding box coordinates, size information, final class label and confidence score of each defect, sort them in descending order of confidence, and generate a defect list containing all detected dirt and / or metal defects; Filter the generated defect list to remove duplicate or redundant detection boxes to ensure that each defect is detected and marked only once; Based on the screened defect list, the final judgment results of dirt and metal defects on the chip are output.

8. A multi-stage automatic detection system for laser chip dirt and / or metal defects, characterized in that: include: A preprocessing module, used for preprocessing the acquired chip image data; A coarse positioning module, for performing coarse positioning of dirt and / or metal defects on the preprocessed chip image using an improved Yolov7 network, and determining and adaptively adjusting a size threshold for distinguishing the size of a detection target during the coarse positioning process; If the size of the detection target is larger than the size threshold, the detection result of the rough positioning stage is directly output as the final judgment result of dirt and metal defects on the chip; The fine classification module is used to enter the second stage of processing if the size of the detected target is not greater than the size threshold. The lightweight MobileNet classification network is used for fine classification to obtain the detection results of the fine classification stage. Based on the detection results in the two stages of coarse positioning and fine classification, the final judgment result of dirt and / or metal defects on the chip is output.

9. A multi-stage automatic detection device for laser chip dirt and / or metal defects, characterized in that: include: An image acquisition device, used for acquiring chip image data; The controller is connected to the image acquisition device and is configured to execute the multi-stage automatic detection method for laser chip contamination and / or metal defects as described in any one of claims 1 to 7.

10. A computer-readable medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, the multi-stage automatic detection method for laser chip contamination and / or metal defects as described in any one of claims 1 to 7 is implemented.

Images