A small sample chip appearance defect detection method and detection system based on improved YOLOv5

Abstract

The application discloses a kind of based on improved YOLOv5 Small Sample Chip Appearance Defect Detection Method and detection system, the method includes obtaining defective image as original dataset;Using original dataset training generative adversarial network, to obtain augmented dataset;Introduce CBAM attention mechanism in YOLOv5 neural network model, obtain improved YOLOv5:Use augmented dataset to train improved YOLOv5 neural network model, obtain defect detection model;The picture of chip to be measured is obtained, and after being preprocessed, input into defect detection model and detect, the picture with defect detected is input to display and is shown, according to the picture shown, find out corresponding chip;The system includes model training module, picture acquisition module, display and marking module.The application solves the problem that the deep learning network model used for defect detection based on YOLOv4 network construction in the prior art has too little defect data, which can easily lead to network model overfitting.

Description

Technical Field

[0001] This invention relates to the field of chip defect detection technology, and in particular to a method and system for detecting appearance defects in small sample chips based on an improved YOLOv5. Background Technology

[0002] After chip packaging, it is necessary to inspect for appearance defects such as bubbles, which is an important part of quality inspection. The main methods for defect detection are manual inspection and chip appearance defect detection based on traditional machine vision. Manual inspection is inefficient, has poor real-time performance, high inspection cost, low inspection accuracy, and is easily affected by human subjective factors. Chip defect detection methods based on traditional machine vision mainly use image algorithms to extract chip defect features, and then distinguish and identify defects by the values ​​of various features. Although this method does not require the acquisition of a large number of images, human analysis still plays a dominant role in the entire inspection process, and it is not suitable for complex defect detection.

[0003] Existing technology proposes a deep learning-based method that inputs a large amount of manually labeled image data and utilizes the powerful image feature extraction capabilities of deep convolutional neural networks to learn about surface defects of various chips. This enables the deep convolutional neural network to remember the characteristics of surface defects of different types of chips and to recognize defect images under complex conditions.

[0004] Chinese patent CN112967243B proposes a YOLO-based deep learning method for detecting chip packaging crack defects. This method utilizes a pre-built YOLOv4 network trained on a dataset. Standardized images are input into the network for prediction. Confidence thresholds and crack boundary thresholds are used to further filter the results to obtain the final outcome. While this method solves the problems of low accuracy and slow detection speed in chip defect detection, the deep learning network model used for defect detection based on the YOLOv4 network suffers from the problem of insufficient defect data, which can easily lead to overfitting of the network model. Summary of the Invention

[0005] Purpose of the invention: The purpose of this invention is to provide a small-sample chip appearance defect detection method and system based on the improved YOLOv5 that is suitable for small-sample training and has high accuracy.

[0006] Technical Solution: To achieve the above objectives, the present invention provides a method for detecting appearance defects in small-sample chips based on improved YOLOv5, comprising the following steps:

[0007] Step S1: Acquire multiple sets of images, each containing chips arranged in a two-dimensional array. Mark defective chips with laser to obtain the original dataset.

[0008] Step S2: Train a generative adversarial network using the original dataset, and obtain an augmented dataset from the generative adversarial network;

[0009] Step S3: Introduce the CBAM attention mechanism into the YOLOv5 neural network model to obtain an improved YOLOv5;

[0010] Step S4: Use the augmented dataset to train the improved YOLOv5 neural network model that incorporates the CBAM attention mechanism module to obtain the defect detection model;

[0011] Step S5: Obtain an image of the chip under test and perform preprocessing;

[0012] Step S6: Input the preprocessed image into the defect detection model for detection, input the detected defective image into the display for display, and laser mark the corresponding chip;

[0013] Step S7: Find the corresponding chip based on the displayed image.

[0014] Step 2, which involves training a generative adversarial network using the original dataset and obtaining an augmented dataset from the generative adversarial network, includes the following sub-steps:

[0015] Step S201: The Generative Adversarial Network includes a generative model and a discriminative model. The generative model uses a generator to generate realistic images based on the input random noise. The discriminative model uses a discriminator to determine whether the input image is real data or an image generated by the generator.

[0016] The generator G generates an image based on random noise z, denoted as G(z); G(z) and real images in the original dataset are used as samples and input into the discriminator D; the discriminator D takes any image x in the sample and denoted as D(x);

[0017] Step S202: Using the ability to correctly distinguish between generated and real images as the discriminator's loss, and the ability to generate near-realistic images that the discriminator classifies as real photos as the generator's loss, the target loss function of the generative adversarial network is:

[0018] V(D,G)=E x～μ [logD(x)]+E z～γ [log(1-D(G(z)))];

[0019] Where E represents the expected value of the distribution corresponding to the subscript, z is random noise, x is the real image, G(z) is the image generated by the generator based on the random noise, μ is the distribution of the real image x, γ is the distribution of the generated image G(z), and D(*) is the probability that the discriminator judges whether the image is a real image.

[0020] Keeping the parameters of generator G unchanged, the discriminator D is optimized based on its loss function. The loss function of discriminator D is:

[0021]

[0022] When D(x) is close to 1 and D(G(x)) is close to 0, the discriminator D can distinguish the generated image.

[0023] Keeping the parameters of the discriminator D unchanged, the generator G is optimized based on its loss function. The loss function of the generator G is:

[0024]

[0025] When D(G(z)) is close to 1, the discriminator D cannot distinguish the generated image;

[0026] When μ and γ are equal or very close, the discriminator will determine the generated image G(z) as a real photograph.

[0027] Step S203: Set the number of iterations. In each iteration, input a random noise into the generator G and execute steps S201 and S202 until the distribution γ of the generated image approaches the distribution μ of the real image x, that is, the generative adversarial network is trained. Otherwise, continue iterating.

[0028] Step S204: After the adversarial network training is completed, the generator G generates multiple images that are indistinguishable from real images. The images are then input into the original dataset to obtain the augmented dataset.

[0029] Specifically, step S3, which involves introducing the CBAM attention mechanism module into the YOLOv5 neural network model to obtain an improved YOLOv5, involves the following: the CBAM attention mechanism is a dual attention mechanism combining channel attention and spatial attention. The intermediate feature map of the YOLOv5 neural network model with the CBAM attention mechanism module is used as the input feature map, and the CBAM attention mechanism module sequentially infers a channel attention feature map M. C (F) and a spatial attention feature map M S (F), M C (F) and the input feature map F are multiplied element-wise to generate the transition feature map F' required by the spatial attention module, and M is then multiplied. S Element-wise multiplication of (F`) and the transition feature map F` yields the final feature map F`` of the CBAM attention mechanism:

[0030]

[0031]

[0032] Where F is the input feature map with height H, width W, and number of channels C. For element-wise multiplication, M C For channel attention operations, M S This is a spatial attention operation.

[0033] The CBAM attention mechanism module infers a channel attention feature map M. C (F), specifically: the input feature map F is processed by global average pooling and global max pooling based on height H and width W respectively to obtain two 1×1×C feature maps. The two 1×1×C feature maps are fed into the same two-layer neural network. The number of neurons in the first layer is C / r, where r is the reduction rate and the activation function is the ReLU function. The number of neurons in the second layer is C, and a 1×1 convolution operation is performed to achieve cross-channel information interaction.

[0034] The features output by the two-layer neural network are subjected to element-wise summation, followed by sigmoid activation to generate a channel attention feature map M. C (F):

[0035]

[0036] Where σ is the sigmoid function, AvgPool(*) is average pooling, MaxPool(*) is max pooling, MLP is a two-layer neural network, and MLP(AvgPool(F)) is the result obtained by inputting AvgPool(F) into the two-layer neural network. The result of average pooling for feature map F with C channels. The feature map F with C channels is the result of max pooling. W0 is the first layer of the two-layer neural network, and W1 is the second layer of the two-layer neural network.

[0037] The CBAM attention mechanism module infers a spatial attention feature map M. S (F), specifically: take the intermediate feature map F` as the input feature map of this module, and perform global max pooling and global average pooling based on channel C to obtain two H×W×1 feature maps. Perform channel concatenation operation on the two H×W×1 feature maps based on channel C, and then perform a 7×7 convolution operation to reduce the dimension to 1 channel, i.e. H×W×1.

[0038] Then, after BN normalization and Sigmoid activation, a spatial attention feature map M is generated. S (F`):

[0039]

[0040] Where σ is the sigmoid function, AvgPool(*) is average pooling, MaxPool(*) is max pooling, and f is the convolution operation. The result of average pooling for feature map F with S channels. The result is the feature map F with S channels after max pooling.

[0041] Specifically, step S4, which involves training the improved YOLOv5 neural network model with the CBAM attention mechanism module using the augmented dataset, refers to training the improved YOLOv5 neural network model with the CBAM attention mechanism module using the early stopping method based on the augmented dataset.

[0042] The augmented dataset is divided into a training set and a validation set. The training set is input into the improved YOLOv5 neural network model for training. The number of iterations is set to t. After every n iterations, the training effect is verified using the validation set. Training is complete when the generalization loss GL on the validation set is greater than a set value α. The parameters from the previous iteration are used as the final parameters of the model to obtain the defect detection model. The generalization loss is:

[0043]

[0044] Where t is the iteration number, E va (t) represents the validation set error at the t-th iteration, E opt (t) represents the minimum validation set error in the first t iterations.

[0045] In step S5, acquiring images of the chip under test refers to using a camera to capture images of all the chips, inputting them into a computer, marking the location information of each chip in the image, and then preprocessing the image.

[0046] The preprocessing includes: converting the image to grayscale, median filtering for noise reduction, edge detection, and tilt correction.

[0047] In step S6, the process of inputting the detected defective image to the display and laser marking the corresponding chip means that when the defect detection model detects a defect in the chip in the image, the computer inputs the image to the display and controls the marking machine to laser mark the defective chip according to the position information of the image.

[0048] A small sample chip appearance defect detection system based on improved YOLOv5 includes a model training module, an image acquisition module, and a display and labeling module;

[0049] The model training module includes a generative adversarial network training module and a neural network model training module. The generative adversarial network training module is used to obtain the augmented dataset, and the neural network model training module is used to train the improved YOLOv5 neural network model that takes the augmented dataset as input into the CBAM attention mechanism module to obtain the defect detection model.

[0050] The image acquisition module is used to acquire chip images, which are then input into the defect detection model for detection; the display and marking module is used to display the detection results and mark defective chips.

[0051] Beneficial effects: The present invention has the following advantages: 1. The present invention expands the dataset by using generative adversarial networks to generate training data, thereby achieving small sample training;

[0052] 2. The YOLOv5 model structure used in this invention has the characteristics of high detection accuracy, fast inference speed and good target detection effect. It can identify the types of defects in the image and output the coordinates of the target bounding box. By expanding the dataset with GAN, the improved YOLOv5 neural network model with CBAM introduced by training with small samples is realized, thereby realizing the identification of chip appearance defects with small samples.

[0053] 3. By training the YOLOv5 neural network with the CBAM attention mechanism using the early stopping method, the network can pay more attention to the target to be detected, thereby further improving the detection effect. At the same time, it solves the problem that the network model is prone to overfitting due to insufficient defect data. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of the defect detection method of the present invention;

[0055] Figure 2 This is a schematic diagram of the generative adversarial network process of the present invention;

[0056] Figure 3 This is a schematic diagram of the CBAM attention mechanism of the present invention;

[0057] Figure 4 This invention provides an improved YOLOv5 network structure.

[0058] Figure 5 A comparison of the SPP and SPPF structures of this invention;

[0059] Figure 6 This is the original image of the chip under test for this invention;

[0060] Figure 7 This is an image showing the edge detection results of the Canny operator in this invention;

[0061] Figure 8(a) is an image before tilt correction in an embodiment of the present invention, and (b) is an image after tilt correction;

[0062] Figure 9 This is the output image of the original chip under test after testing, as per the present invention. Detailed Implementation

[0063] The technical solution of the present invention will be described in detail below with reference to the embodiments and accompanying drawings.

[0064] like Figure 1 As shown, the small sample chip appearance defect detection method based on improved YOLOv5 described in this invention includes the following steps:

[0065] Step S1: Acquire multiple sets of images, each containing chips arranged in a two-dimensional array. Mark defective chips with laser to obtain the original dataset.

[0066] Step S2: Train a generative adversarial network using the original dataset, and obtain an augmented dataset from the generative adversarial network, such as... Figure 2 As shown, it includes the following sub-steps:

[0067] Step S201: The Generative Adversarial Network includes a generative model and a discriminative model. The generative model uses a generator to generate realistic images based on the input random noise. The discriminative model uses a discriminator to determine whether the input image is real data or an image generated by the generator.

[0068] The generator G generates an image based on random noise z, denoted as G(z); G(z) and real images in the original dataset are used as samples and input into the discriminator D; the discriminator D takes any image x in the sample and denoted as D(x);

[0069] Step S202: Using the ability to correctly distinguish between generated and real images as the discriminator's loss, and the ability to generate near-realistic images that the discriminator classifies as real photos as the generator's loss, the target loss function of the generative adversarial network is:

[0070] V(D,G)=E x～μ [logD(x)]+E z～γ [log(1-D(G(z)))];

[0071] Where E represents the expected value of the distribution corresponding to the subscript, z is random noise, x is the real image, G(z) is the image generated by the generator based on the random noise, μ is the distribution of the real image x, γ is the distribution of the generated image G(z), and D(*) is the probability that the discriminator judges whether the image is a real image.

[0072] Keeping the parameters of generator G unchanged, the discriminator D is optimized based on its loss function. The loss function of discriminator D is:

[0073]

[0074] When D(x) is close to 1 and D(G(x)) is close to 0, the discriminator D can distinguish the generated image.

[0075] Keeping the parameters of the discriminator D unchanged, the generator G is optimized based on its loss function. The loss function of the generator G is:

[0076]

[0077] When D(G(z)) is close to 1, the discriminator D cannot distinguish the generated image;

[0078] When μ and γ are equal or very close, the discriminator will determine the generated image G(z) as a real photograph.

[0079] Step S203: Set the number of iterations. In each iteration, input a random noise into the generator G and execute steps S201 and S202 until the distribution γ of the generated image approaches the distribution μ of the real image x, that is, the generative adversarial network is trained. Otherwise, continue iterating.

[0080] Step S204: After the adversarial network training is completed, the generator G generates multiple images that are indistinguishable from real images. The images are then input into the original dataset to obtain the augmented dataset.

[0081] Step S3: Introduce the CBAM attention mechanism into the YOLOv5 neural network model to obtain an improved YOLOv5, specifically as follows: Figure 3 As shown, the CBAM attention mechanism is a dual attention mechanism of channel attention and spatial attention. It uses the intermediate feature map of the YOLOv5 neural network model with the CBAM attention mechanism module as the input feature map, and the CBAM attention mechanism module sequentially infers a channel attention feature map M. C (F) and a spatial attention feature map M S (F), M C (F) and the input feature map F are multiplied element-wise to generate the transition feature map F' required by the spatial attention module, and M is then multiplied. S Element-wise multiplication of (F`) and the transition feature map F` yields the final feature map F`` of the CBAM attention mechanism:

[0082]

[0083]

[0084] Where F is the input feature map with height H, width W, and number of channels C. For element-wise multiplication, M C For channel attention operations, M S This is a spatial attention operation.

[0085] The CBAM attention mechanism module infers a channel attention feature map M. C (F), specifically: the input feature map F is processed by global average pooling and global max pooling based on height H and width W respectively to obtain two 1×1×C feature maps. The two 1×1×C feature maps are fed into the same two-layer neural network. The number of neurons in the first layer is C / r, where r is the reduction rate and the activation function is the ReLU function. The number of neurons in the second layer is C, and a 1×1 convolution operation is performed to achieve cross-channel information interaction.

[0086] The features output by the two-layer neural network are subjected to element-wise summation, followed by sigmoid activation to generate a channel attention feature map M. C (F):

[0087]

[0088] Where σ is the sigmoid function, AvgPool(*) is average pooling, MaxPool(*) is max pooling, MLP is a two-layer neural network, and MLP(AvgPool(F)) is the result obtained by inputting AvgPool(F) into the two-layer neural network. The result of average pooling for feature map F with C channels. The feature map F with C channels is the result of max pooling. W0 is the first layer of the two-layer neural network, and W1 is the second layer of the two-layer neural network.

[0089] The CBAM attention mechanism module infers a spatial attention feature map M. S (F), specifically: take the intermediate feature map F` as the input feature map of this module, and perform global max pooling and global average pooling based on channel C to obtain two H×W×1 feature maps. Perform channel concatenation operation on the two H×W×1 feature maps based on channel C, and then perform a 7×7 convolution operation to reduce the dimension to 1 channel, i.e. H×W×1.

[0090] Then, after BN normalization and Sigmoid activation, a spatial attention feature map M is generated. S (F`):

[0091]

[0092] Where σ is the sigmoid function, AvgPool(*) is average pooling, MaxPool(*) is max pooling, and f is the convolution operation. The result of average pooling for feature map F with S channels. The result is the feature map F with S channels after max pooling.

[0093] This invention inserts a CBAM attention mechanism after the activation function of the Conv module in the YOLOv5 neural network model. The Conv module consists of convolution operations, BN normalization, and activation functions. By improving the Conv module into a Conv-CBAM module, the network can pay more attention to the target to be detected, thereby improving the detection effect.

[0094] Improved YOLOv5 neural network model structure, such as Figure 4 As shown, YOLOv5 achieves a better balance between accuracy and speed compared to YOLOv4. Specifically: YOLOv5's data augmentation methods are simpler than YOLOv4's, using only three methods: scaling, color space adjustment, and Mosaic data augmentation; YOLOv4 uses fixed anchor boxes, while YOLOv5's anchor boxes are adaptively learned based on the training set; YOLOv5 uses leakyReLU and sigmoid as activation functions; and in the network detection neck, unlike YOLOv4's SPP, YOLOv5 uses SPPF, which effectively improves runtime efficiency. A comparison of SPP and SPPF structures is shown below. Figure 5 As shown.

[0095] YOLOv5's backbone network is CSP-Darknet-53. The Darknet-53 feature extraction network contains 53 convolutional layers and integrates the idea of ​​residual networks, which can effectively avoid problems such as gradient vanishing and gradient exploding. CSP-Darknet-53 adds a CSP structure to the Darknet-53, splitting the originally stacked residual blocks into two parts. One part is convolutionally processed, and the other part is fused with the feature map after the convolution operation of the first part, which can improve the detection speed while ensuring detection accuracy.

[0096] Step S4: Train the improved YOLOv5 neural network model with the CBAM attention mechanism module using the augmented dataset to obtain the defect detection model. This refers to training the improved YOLOv5 neural network model with the CBAM attention mechanism module using the early stopping method based on the augmented dataset. Specifically:

[0097] The augmented dataset is divided into training and validation sets. The training set is input into the improved YOLOv5 neural network model for training. The number of iterations, t, is set. Every 15 iterations, the validation set is used to verify the training effect. Training is complete when the generalization loss GL on the validation set is greater than a set value α. The parameters from the previous iteration are used as the final parameters of the model to obtain the defect detection model. The generalization loss is:

[0098]

[0099] Where t is the iteration number, E va (t) represents the validation set error at the t-th iteration, E opt (t) represents the minimum validation set error in the first t iterations.

[0100] The value of α needs to be selected based on the training objective. If α is larger, it is more difficult to stop training and there is a greater possibility of errors, but there is a greater chance of finding the global optimum. If α is smaller, the requirements for stopping training are lower and there is a smaller possibility of errors, but the optimum found may only be a local optimum.

[0101] Early stopping is a technique used when training deep neural networks. Once the test error on the data increases, training is stopped, and the weights after stopping are used as the final parameters of the network in order to obtain the best generalization performance. When training a deep neural network, better generalization performance is often required, hoping that the model can fit the data better. However, if the number of iterations is too large, the deep neural network may overfit. That is, the network performs better and better when testing the training set, but its performance on the test set becomes worse and worse. Early stopping can effectively solve the overfitting problem.

[0102] Step S5: Acquire images of the chip under test and perform preprocessing; acquiring images of the chip under test means using a camera to capture images of all chips and inputting them into a computer. The computer marks the position information of each chip in the image and then performs preprocessing on the image; preprocessing includes: converting the image to grayscale, median filtering for noise reduction, edge detection, and tilt image correction.

[0103] like Figure 6 As shown, the original image of the chip under test is processed by converting the image to grayscale to avoid stripe distortion and improve the processing speed. Then, median filtering is used to reduce noise, which suppresses or eliminates image noise to a certain extent, thereby improving the image quality and indirectly improving the accuracy of chip appearance defect detection and identification.

[0104] like Figure 7As shown, based on the Canny edge detection operator, the set of pixels with abrupt changes in brightness in the image is identified to achieve edge detection of the chip, which can significantly reduce the amount of data and thus improve processing speed. The tilted chip image is corrected based on the Hough transform, and the images before and after correction are shown below. Figure 8 As shown in (a) and (b).

[0105] Step S6: Input the preprocessed image into the defect detection model for detection. Images showing defects are displayed on the monitor, and the corresponding chips are laser-marked. This means that when the defect detection model detects a defect in a chip in the image, the computer displays the image on the monitor and, based on the image's location information, controls the marking machine to laser-mark the defective chip. The final output result after detection is as follows: Figure 9 As shown, broken pins were observed in the upper left, middle, and lower middle parts of the image.

[0106] Chinese patent CN107755879A provides a laser marking machine, a method for adjusting the distance between the scanning head and the marking object, and an automatic focusing method for the marking machine. This invention uses this method to laser mark defective chips.

[0107] Step S7: Find the corresponding chip based on the displayed image.

[0108] This invention also proposes a small-sample chip appearance defect detection system based on an improved YOLOv5, including a model training module, an image acquisition module, and a display and labeling module. The model training module includes a generative adversarial network (GAN) training module and a neural network model training module. The GAN training module is used to obtain an augmented dataset, and the neural network model training module is used to train an improved YOLOv5 neural network model by inputting the augmented dataset into the CBAM attention mechanism module to obtain a defect detection model. The image acquisition module is used to acquire chip images and then input them into the defect detection model for detection. The display and labeling module is used to display the detection results and label defective chips.

[0109] The deep convolutional neural network used in this invention is the YOLO (You Only Look Once) target detection model, which can identify the types of defects in images and output the coordinates of the target bounding box. Using a small number of existing chip defect samples, the dataset is expanded by GAN to realize the improved YOLOv5 neural network model with CBAM introduced by training with small samples, thereby realizing the identification of chip appearance defects with small samples. The introduction of CBAM can improve the attention of the YOLOv5 neural network to the detection target, thereby improving the detection effect. It can effectively solve the problems of low efficiency, poor real-time performance, high detection cost and low detection accuracy in manual inspection and chip appearance defect detection based on traditional machine vision.

[0110] This invention expands the dataset significantly by using generative adversarial networks to generate training data, enabling small-sample training. Compared to YOLOv4, the YOLOv5 model structure used has improvements in detection accuracy, inference speed, and object detection performance.

[0111] By training the YOLOv5 neural network with the CBAM attention mechanism using the early stopping method, the network can pay more attention to the target to be detected, thereby further improving the detection effect. At the same time, it solves the problem that the network model is prone to overfitting due to insufficient defect data.

Claims

1. A method for detecting surface defects in small-sample chips based on an improved YOLOv5, characterized in that, Includes the following steps: Step S1: Acquire multiple sets of images, each containing chips arranged in a two-dimensional array. Mark defective chips with laser to obtain the original dataset. Step S2: Train a generative adversarial network using the original dataset, and obtain an augmented dataset from the generative adversarial network; Step S3: Introduce the CBAM attention mechanism into the YOLOv5 neural network model to obtain an improved YOLOv5. Specifically, the CBAM attention mechanism is a dual attention mechanism of channel attention and spatial attention. The intermediate feature map of the YOLOv5 neural network model with the CBAM attention mechanism module is used as the input feature map, and the CBAM attention mechanism module infers a channel attention feature map in sequence. A spatial attention feature map ,Will Perform element-wise multiplication with the input feature map F to generate the transition feature map F' required by the spatial attention module. Element-wise multiplication with the transition feature map F' yields the final feature map F'' of the CBAM attention mechanism: ， `； Where F is the input feature map with height H, width W, and number of channels C, and ⓧ represents element-wise multiplication. For channel attention operations, For spatial attention operations; The CBAM attention mechanism module infers a channel attention feature map. Specifically, the input feature map F is processed by global average pooling and global max pooling based on height H and width W respectively to obtain two 1×1×C feature maps. The two 1×1×C feature maps are then fed into the same two-layer neural network. The first layer has C / r neurons, where r is the reduction rate and the activation function is the ReLU function. The second layer has C neurons and performs a 1×1 convolution operation to achieve cross-channel information interaction. The features output by the two-layer neural network are subjected to element-wise summation, followed by sigmoid activation to generate channel attention feature maps. : ； in, For the sigmoid function, For average pooling, For max pooling, MLP is a two-layer neural network. To be The result obtained by inputting into a two-layer neural network The result of average pooling for feature map F with C channels. The result of max pooling is the feature map F with C channels. This is the first layer of a two-layer neural network. This is the second layer of a two-layer neural network; The CBAM attention mechanism module infers a spatial attention feature map. Specifically, the intermediate feature map F` is used as the input feature map of this module. F` is subjected to global max pooling and global average pooling based on channel C to obtain two H×W×1 feature maps. The two H×W×1 feature maps are concatenated based on channel C, and then subjected to a 7×7 convolution operation to reduce the dimension to 1 channel, i.e. H×W×1. Spatial attention feature maps are then generated through BN normalization and Sigmoid activation. : ； in, For the sigmoid function, For average pooling, For max pooling, For convolution operations, The result of average pooling for feature map F with S channels. The result of max pooling the feature map F with S channels; Step S4: Use the augmented dataset to train the improved YOLOv5 neural network model that incorporates the CBAM attention mechanism module to obtain the defect detection model; Step S5: Obtain an image of the chip under test and perform preprocessing; Step S6: Input the preprocessed image into the defect detection model for detection, input the detected defective image into the display for display, and laser mark the corresponding chip; Step S7: Find the corresponding chip based on the displayed image.

2. The method for detecting small-sample chip appearance defects based on improved YOLOv5 according to claim 1, characterized in that: Step 2, which involves training a generative adversarial network using the original dataset and obtaining an augmented dataset from the generative adversarial network, includes the following sub-steps: Step S201: The Generative Adversarial Network includes a generative model and a discriminative model. The generative model uses a generator to generate realistic images based on the input random noise. The discriminative model uses a discriminator to determine whether the input image is real data or an image generated by the generator. The generator G generates an image based on random noise z, denoted as G(z); G(z) and real images from the original dataset are input as samples into the discriminator D; the discriminator D assigns any image x from the samples to the discriminator, denoted as D(x). Step S202: Using the ability to correctly distinguish between generated and real images as the discriminator's loss, and the ability to generate near-realistic images that the discriminator classifies as real photos as the generator's loss, the target loss function of the generative adversarial network is: ； Where E represents the expected value of the distribution corresponding to the subscript, and z is random noise. G(z) represents the real image, and G(z) represents the image generated by the generator based on random noise. These are real pictures. The distribution, G(z) is the distribution of the generated image, and D(*) is the probability that the discriminator determines whether the image is a real image. Keeping the parameters of generator G unchanged, the discriminator D is optimized based on its loss function. The loss function of discriminator D is: ； When D(x) is close to 1 and D(G(x)) is close to 0, the discriminator D can distinguish the generated image. Keeping the parameters of the discriminator D unchanged, the generator G is optimized based on its loss function. The loss function of the generator G is: ； When D(G(z)) is close to 1, the discriminator D cannot distinguish the generated image; when and When the values ​​are equal or very close, the discriminator will determine the generated image G(z) as a real photograph. Step S203: Set the number of iterations. In each iteration, input a random noise into the generator G and execute steps S201 and S202 until the distribution γ of the generated image approaches the distribution μ of the real image x, that is, the generative adversarial network is trained. Otherwise, continue iterating. Step S204: After the adversarial network training is completed, the generator G generates multiple images that are indistinguishable from real images. The images are then input into the original dataset to obtain the augmented dataset.

3. The method for detecting small-sample chip appearance defects based on improved YOLOv5 according to claim 1, characterized in that: Step S4, which describes training the improved YOLOv5 neural network model with the CBAM attention mechanism module using the augmented dataset, refers to training the improved YOLOv5 neural network model with the CBAM attention mechanism module using the early stopping method based on the augmented dataset. Specifically: The augmented dataset is divided into a training set and a validation set. The training set is input into the improved YOLOv5 neural network model for training. The number of iterations is set to t. After every n iterations, the training effect is verified using the validation set. Training is complete when the generalization loss GL on the validation set is greater than a set value α. The parameters from the previous iteration are used as the final parameters of the model to obtain the defect detection model. The generalization loss is: ； Where t is the number of iterations. Let be the validation set error at the t-th iteration. The minimum validation set error in the first t iterations. .

4. The method for detecting small-sample chip appearance defects based on improved YOLOv5 according to claim 1, characterized in that: The step S5, which involves acquiring images of the chip under test, refers to using a camera to capture images of all the chips and inputting them into a computer. The computer then marks the location information of each chip in the image and performs preprocessing on the image.

5. The method for detecting small-sample chip appearance defects based on improved YOLOv5 according to claim 4, characterized in that: The preprocessing includes: converting the image to grayscale, median filtering for noise reduction, edge detection, and tilt correction.

6. The method for detecting small-sample chip appearance defects based on improved YOLOv5 according to claim 1, characterized in that: Step S6, which involves inputting the detected defective image to the display and laser marking the corresponding chip, means that when the defect detection model detects a defect in the chip in the image, the computer inputs the image to the display and controls the marking machine to laser mark the defective chip based on the position information of the image.

7. A small-sample chip appearance defect detection system based on an improved YOLOv5, characterized in that: It includes a model training module, an image acquisition module, and a display and labeling module; The model training module includes a generative adversarial network training module and a neural network model training module. The generative adversarial network training module is used to obtain the augmented dataset, and the neural network model training module is used to train the improved YOLOv5 neural network model that takes the augmented dataset as input into the CBAM attention mechanism module to obtain the defect detection model. The CBAM attention mechanism is a dual attention mechanism combining channel attention and spatial attention. It takes the intermediate feature maps of the YOLOv5 neural network model with the CBAM attention mechanism module as input feature maps, and the CBAM attention mechanism module sequentially infers a channel attention feature map. A spatial attention feature map ,Will Perform element-wise multiplication with the input feature map F to generate the transition feature map F' required by the spatial attention module. Element-wise multiplication with the transition feature map F' yields the final feature map F'' of the CBAM attention mechanism: ， `； Where F is the input feature map with height H, width W, and number of channels C, and ⓧ represents element-wise multiplication. For channel attention operations, For spatial attention operations; The CBAM attention mechanism module infers a channel attention feature map. Specifically, the input feature map F is processed by global average pooling and global max pooling based on height H and width W respectively to obtain two 1×1×C feature maps. The two 1×1×C feature maps are then fed into the same two-layer neural network. The first layer has C / r neurons, where r is the reduction rate and the activation function is the ReLU function. The second layer has C neurons and performs a 1×1 convolution operation to achieve cross-channel information interaction. The features output by the two-layer neural network are subjected to element-wise summation, followed by sigmoid activation to generate channel attention feature maps. : ； in, For the sigmoid function, For average pooling, For max pooling, MLP is a two-layer neural network. To be The result obtained by inputting into a two-layer neural network The result of average pooling for feature map F with C channels. The result of max pooling is the feature map F with C channels. This is the first layer of a two-layer neural network. This is the second layer of a two-layer neural network; The CBAM attention mechanism module infers a spatial attention feature map. Specifically, the intermediate feature map F` is used as the input feature map of this module. F` is subjected to global max pooling and global average pooling based on channel C to obtain two H×W×1 feature maps. The two H×W×1 feature maps are concatenated based on channel C, and then subjected to a 7×7 convolution operation to reduce the dimension to 1 channel, i.e. H×W×1. Spatial attention feature maps are then generated through BN normalization and Sigmoid activation. : ； in, For the sigmoid function, For average pooling, For max pooling, For convolution operations, The result of average pooling for feature map F with S channels. The result of max pooling the feature map F with S channels; The image acquisition module is used to acquire chip images, which are then input into the defect detection model for detection; the display and marking module is used to display the detection results and mark defective chips.

Images