Improved YOLOv8 chip surface defect lightweight target detection model and training method and application thereof

Abstract

The invention discloses an improved YOLOv8 chip surface defect lightweight target detection model and a training method and application thereof, the improved YOLOv8 chip surface defect lightweight target detection model is built, the improved YOLOv8 chip surface defect lightweight target detection model comprises a Backbone part, a Neck part and a Head part, the Backbone part is formed by stacking a plurality of AKConv modules and C2f modules, an input picture is subjected to multiple times of convolution, and a plurality of AKConv modules and C2f modules are obtained; a plurality of effective feature layers are output to the Neck module; the Neck part comprises a multi-scale feature enhancement module and a cross-scale channel fusion module CCFM, and the multi-scale feature enhancement module adopts a RepGFPN network to perform effective fusion on multiple effective feature layers output by the Backbone part; the cross-scale channel fusion module CCFM is used for automatically adjusting a feature fusion weight; the Head part outputs a chip surface defect detection result; the trained improved YOLOv8 target detection model is used for detecting the surface defects of the chip, the defect that the detection effect is poor when a small target and a complex background are detected can be overcome, and the defect detection speed and precision are improved.

Description

Technical Field

[0001] The present invention belongs to the field of machine vision and deep learning technology, and specifically relates to an improved YOLOv8 chip surface defect lightweight target detection model and a training method and application thereof. Background Art

[0002] With the rapid development of the semiconductor industry, chip surface defect detection has become a key link in industrial production. Traditional detection methods mainly rely on manual detection or simple machine vision systems. These methods are inefficient, have a high rate of missed detection, and cannot meet modern high-precision requirements. Therefore, automated defect detection methods have emerged, especially target detection models based on deep learning, which show great application potential in chip defect detection.

[0003] As a model in the field of target detection, YOLOv8 has efficient and accurate target detection capabilities. However, in practical applications, the traditional YOLOv8 model has poor detection effect on small targets and its accuracy decreases when dealing with complex backgrounds. Summary of the invention

[0004] In order to address the deficiencies in the prior art, this application proposes an improved YOLOv8 chip surface defect lightweight target detection model and its training method and application, and uses an optimized YOLOv8 network to perform target detection on chip surface images. It can overcome the shortcomings of existing detection methods that have poor detection effects on small targets and complex backgrounds, significantly improve the speed and accuracy of defect detection, and achieve efficient and accurate chip surface defect detection.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A training method for an improved YOLOv8 chip surface defect lightweight target detection model includes the following steps:

[0007] Step 1, build an improved YOLOv8 chip surface defect lightweight target detection model, including Backbone part, Neck part and Head part. The Backbone part is composed of multiple AKConv modules and C2f modules stacked together, and the input image is convolved multiple times, and multiple layers of effective feature layers are output to the Neck module; the Neck part includes a multi-scale feature enhancement module and a cross-scale channel fusion module CCFM. The multi-scale feature enhancement module adopts the RepGFPN network to effectively fuse the multiple layers of effective feature layers output by the Backbone part; the cross-scale channel fusion module CCFM is used to automatically adjust the feature fusion weight; the Head part outputs the chip surface defect detection result;

[0008] Step 2: After completing the construction of the improved YOLOv8 target detection model structure, use the chip surface defect image to train and verify the built improved YOLOv8 target detection model structure.

[0009] Furthermore, the network structure of the Backbone part is: from input to output, the first AKConv module, the second AKConv module, the first C2f module, the third AKConv module, the second C2f module, the fourth AKConv module, the third C2f module, the fifth AKConv module, the fourth C2f module, and the SPPF module are connected in sequence, wherein the second C2f module outputs the first effective feature layer, the third C2f module outputs the second effective feature layer, and the SPPF module outputs the third effective feature layer.

[0010] Furthermore, the AKConv module is composed of two Conv2d, normalization and SELU activation functions, uses Conv2d to perform a convolution operation on the input image, and then outputs the final result through the activation function SiLU.

[0011] Furthermore, the structure of the network of the Neck part is: the first Upsample, the first RepGFPN network, the fifth C2f module, the second Upsample, the second RepGFPN network, the sixth C2f module, the first CCFM module, the first Conv module, the concat, the seventh C2f module, the second CCFM module, the second Conv module, the third RepGFPN network, the eighth C2f module, and the third CCFM module are connected in sequence, and are directly connected to the fifth C2f module concat. The inputs of the first Upsample and the third RepGFPN network are connected to the SPPF module to receive the third effective feature layer output by the SPPF module; the input of the first RepGFPN network is connected to the third C2f module to receive the second effective feature layer output by the SPPF module; the input of the second RepGFPN network is connected to the second C2f module to receive the first effective feature layer output by the C2f module; the first CCFM module, the second CCFM module, and the third CCFM module respectively output features for prediction.

[0012] Furthermore, the RepGFPN network includes 5 CSPStage layers, the first CSPStage layer, the second CSPStage layer, the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are connected in sequence, and the output of the first CSPStage layer is also directly connected to the fifth CSPStage layer, and the output of the second CSPStage layer is also directly connected to the fourth CSPStage layer and the fifth CSPStage layer, respectively; wherein the outputs corresponding to the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are respectively used as the output results of the RepGFPN network.

[0013] Furthermore, the CSPStage layer is divided into three processing paths. First, the input features are spliced, and then they are processed by 1*1 Conv convolution in the first path and output. After splicing, they are processed by 1*1 Conv convolution, 3*3Rep, and 3*3Conv in the second path and output. After splicing, they are processed by 1*1 Conv convolution in the second path and repeated 3*3Rep and 3*3Conv. The outputs of the three processing paths are spliced ​​by Concat and then processed by 1*1 Conv convolution and output.

[0014] Furthermore, the process of training the improved YOLOv8 target detection model structure in step 2 includes:

[0015] Step 2.1, collect chip surface defect images and mark the defect categories to form a data set; divide the data set into a training set and a test set;

[0016] Step 2.2: Train the model based on the training set and the test set.

[0017] Furthermore, a variety of image enhancement operations are performed on the dataset in step 2.1, including random translation, flipping, rotation, and brightness adjustment.

[0018] An improved YOLOv8 chip surface defect lightweight target detection model is trained using the above method.

[0019] A chip surface defect target detection method comprises the following steps:

[0020] Step 1: Use an industrial camera to obtain an image of the chip to be tested;

[0021] Step 2: Input the chip image to be detected into the improved YOLOv8 chip surface defect lightweight target detection model and output the detection result.

[0022] Beneficial effects of the present invention:

[0023] In order to solve the problem that the traditional YOLOv8 model has poor detection effect on small targets, the present invention replaces part of the traditional convolution in Backbone with the lightweight convolution AKConv to reduce model parameters and improve detection efficiency. Secondly, the activation function of AKConv is replaced by the SELU function from the SiLU function to enhance the feature extraction ability of the convolution and expand the effective receptive field. Then, the feature pyramid network (FPN) is replaced by the RepGFPN in the Neck part to enhance the multi-scale feature fusion, thereby improving the detection ability of subtle defects on the chip surface. Finally, the cross-scale channel fusion module (CCFM) is introduced to further improve the detection accuracy of small defects by strengthening the weights of key features. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the construction method of the improved YOLOv8 target detection model in the present invention, showing the logical relationship between each step and the model improvement process.

[0025] Figure 2 It is a schematic diagram of the overall structure of the improved YOLOv8 target detection model of the present invention, which shows in detail the composition and functions of the Backbone, Neck, Head and other modules of the model.

[0026] Figure 3 It is a structural diagram of the AKConv module, showing the connection mode and data flow of each submodule in the AKConv module.

[0027] Figure 4 It is a characteristic curve diagram of the SELU activation function, which shows the mathematical characteristics of the SELU activation function and its processing effect on the input data.

[0028] Figure 5 It is a structural diagram of the RepGFPN multi-scale feature fusion module, which shows how the RepGFPN module fuses multi-scale feature maps to improve detection accuracy.

[0029] Figure 6 It is a structural diagram of the CCFM cross-scale channel fusion module, showing how the CCFM module optimizes the feature map through the channel attention mechanism.

[0030] Figure 7 This is a rendering of the improved YOLOv8 target detection model of the present invention in chip surface defect detection, showing the visualization effect of the model detection results. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0032] Example 1

[0033] Combined with Figure 1 , this application proposes a training method for an improved YOLOv8 chip surface defect lightweight target detection model, comprising the following steps:

[0034] Step 1: Build an improved YOLOv8 chip surface defect lightweight target detection model. The network structure is as follows: Figure 2 As shown in the figure, the improved YOLOv8 target detection model structure consists of three parts: Backbone, Neck and Head:

[0035] 1. Backbone

[0036] The Backbone part is composed of 5 stacked AKConv modules and 4 C2f modules. It performs multiple convolutions on the input image and outputs the effective feature layer to the Neck module. The specific connection relationship is: from input to output, the first AKConv module, the second AKConv module, the first C2f module, the third AKConv module, the second C2f module, the fourth AKConv module, the third C2f module, the fifth AKConv module, the fourth C2f module, and the SPPF module are connected in sequence, among which the second C2f module outputs the first effective feature layer, the third C2f module outputs the second effective feature layer, and the SPPF module outputs the third effective feature layer.

[0037] More specifically, the AKConv module structure is as follows Figure 3 As shown, each AKConv module consists of two Conv2d, normalization and SELU activation functions; the input image has dimensions (C, H, W), where C is the number of channels, H and W are the height and width of the image, respectively, and an initial sampling shape is performed to give the initial sampling shape of the convolution kernel.

[0038] Use Conv2d to perform convolution operations on the input image, adjust the initial sampling shape through the learned offset, allow the convolution kernel shape to dynamically adjust to the characteristics of the image, resample the feature map according to the adjusted sampling shape, reshape the resampled feature map, reconvolute, and standardize it, and finally output the final result through the activation function SiLU. Furthermore, the SELU activation function is used in the AKConv module to replace the traditional SiLU activation function. The self-normalization property of the SELU function makes the output of the network layer stable and reduces the gradient disappearance phenomenon. Figure 4 The expression of the SELU activation function shown is as follows:

[0039]

[0040] 2. Neck

[0041] The Neck part uses a multi-scale feature enhancement module to effectively fuse high, medium and low resolution feature maps, thereby enhancing the detection capability of small targets. The multi-scale feature enhancement module uses the RepGFPN network to implement an adaptive feature fusion strategy, and improves the overall feature expression capability through information interaction between multi-scale feature maps.

[0042] The structure of the network of the Neck part is: the first Upsample, the first RepGFPN network, the fifth C2f module, the second Upsample, the second RepGFPN network, the sixth C2f module, the first CCFM module, the first Conv module, the concat, the seventh C2f module, the second CCFM module, the second Conv module, the third RepGFPN network, the eighth C2f module, and the third CCFM module are connected in sequence, and are directly connected to the fifth C2f module concat. The inputs of the first Upsample and the third RepGFPN network are connected to the SPPF module to receive the third effective feature layer output by the SPPF module; the input of the first RepGFPN network is connected to the third C2f module to receive the second effective feature layer output by the SPPF module; the input of the second RepGFPN network is connected to the second C2f module to receive the first effective feature layer output by the C2f module. The first CCFM module, the second CCFM module, and the third CCFM module output features for prediction respectively.

[0043] More specifically, the structure of the RepGFPN network is as follows Figure 5 As shown, it includes 5 CSPStage layers, the first CSPStage layer, the second CSPStage layer, the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are connected in sequence, and the output of the first CSPStage layer is also directly connected to the fifth CSPStage layer, and the output of the second CSPStage layer is also directly connected to the fourth CSPStage layer and the fifth CSPStage layer respectively; wherein, the outputs corresponding to the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are respectively used as the output results of the RepGFPN network.

[0044] More specifically, there are three processing paths in the CSPStage layer. First, the input features are concatenated, and then they are processed by 1*1 Conv convolution in the first path and output. After concatenation, they are processed by 1*1 Conv convolution, 3*3Rep, and 3*3Conv in the second path and output. After concatenation, they are processed by 1*1 Conv convolution and repeated 3*3Rep and 3*3Conv in the second path and output. The outputs of the three processing paths are concatenated by Concat and then processed by 1*1 Conv convolution before output.

[0045] More specifically, this application also introduces a cross-scale channel fusion module CCFM in the Neck part, and its network structure is as follows: Figure 6 As shown in the figure, after the feature maps of different scales are integrated, the features are extracted by convolution and residual, and finally the features are fused. CCFM learns the channel importance of feature maps of different scales, automatically adjusts the feature fusion weight, highlights significant features, and effectively reduces detection errors.

[0046] 3. Head part

[0047] The head part includes components such as convolutional layer, bounding box regression, category prediction, anchor box generation, loss function calculation and non-maximum suppression (NMS). Through these modules, the model can output the precise location of chip surface defects, category confidence and probability distribution of target categories.

[0048] More specifically, the Head part uses the SELU Loss loss function to dynamically adjust the loss weight for easy-to-classify and difficult-to-classify samples, reducing the negative impact of class imbalance on model training. The calculation formula of SELU Loss is:

[0049]

[0050] Among them, μ is the average value of the IoU value of all bounding boxes. The loss contribution of the sample is used to reduce the weight of the easy-to-classify samples and increase the weight of the difficult-to-classify samples.

[0051] Step 2: After completing the construction of the improved YOLOv8 target detection model structure, it is necessary to train and verify the built improved YOLOv8 target detection model structure.

[0052] Step 2.1, dataset preparation:

[0053] First, an industrial camera was used to collect 600 images of chip surface defects, and the Labelimg tool was used to annotate defect categories such as "DIE_BROKEN" in the PascalVOC format.

[0054] Perform various image enhancement operations on the dataset in step 2.1, including random translation, flipping, rotation, brightness adjustment, etc., to increase data diversity and model robustness. All images are uniformly adjusted to 640×640 resolution. Then, Mosaic data enhancement technology is used to expand batch_size by splicing multiple images, which effectively improves training efficiency. The enhanced dataset is divided into training set and test set at a ratio of 9:1, and the training set is further divided into training subset and validation subset at a ratio of 9:1 to ensure the adequacy of training and validation.

[0055] Step 2.2, model training:

[0056] First, freeze the AKConv layer of the Backbone module and train it for 150 epochs with an initial learning rate of 0.01; then unfreeze the Backbone module and adjust the learning rate to 0.0001, and continue training for 300 epochs.

[0057] The SELU Loss function is used to increase the model's attention to difficult-to-classify samples and reduce the probability of misclassification, especially in the detection of small defects.

[0058] Example 2

[0059] An improved YOLOv8 chip surface defect lightweight target detection model is trained using the method of the above embodiment 1.

[0060] Example 3

[0061] Based on the above-mentioned improved YOLOv8 chip surface defect lightweight target detection model, the present invention also proposes a chip surface defect target detection method, comprising the following steps:

[0062] Step 1: Use an industrial camera to obtain the image of the chip to be tested.

[0063] Step 2: Input the chip image to be detected into the improved YOLOv8 chip surface defect lightweight target detection model, run the defect detection algorithm, and output detailed detection results including bounding box, size, category confidence, etc. Figure 7 shown.

[0064] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

[0065] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A training method for an improved YOLOv8 chip surface defect lightweight target detection model, characterized in that: The steps include: Step 1, build an improved YOLOv8 chip surface defect lightweight target detection model, including Backbone part, Neck part and Head part. The Backbone part is composed of multiple AKConv modules and C2f modules stacked together, and the input image is convolved multiple times, and multiple layers of effective feature layers are output to the Neck module; the Neck part includes a multi-scale feature enhancement module and a cross-scale channel fusion module CCFM. The multi-scale feature enhancement module adopts the RepGFPN network to effectively fuse the multiple layers of effective feature layers output by the Backbone part; the cross-scale channel fusion module CCFM is used to automatically adjust the feature fusion weight; the Head part outputs the chip surface defect detection result; Step 2: After completing the construction of the improved YOLOv8 target detection model structure, use the chip surface defect image to train and verify the built improved YOLOv8 target detection model structure.

2. According to claim 1, a training method for an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: The network structure of the Backbone part is: from input to output, it is connected in sequence by the first AKConv module, the second AKConv module, the first C2f module, the third AKConv module, the second C2f module, the fourth AKConv module, the third C2f module, the fifth AKConv module, the fourth C2f module, and the SPPF module, wherein the second C2f module outputs the first effective feature layer, the third C2f module outputs the second effective feature layer, and the SPPF module outputs the third effective feature layer.

3. According to claim 2, a training method for an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: The AKConv module consists of two Conv2d, normalization and SELU activation functions. Conv2d is used to perform a convolution operation on the input image, and then the final result is output through the activation function SiLU.

4. According to claim 1, a training method for an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: The structure of the network of the Neck part is: the first Upsample, the first RepGFPN network, the fifth C2f module, the second Upsample, the second RepGFPN network, the sixth C2f module, the first CCFM module, the first Conv module, the seventh C2f module, the second CCFM module, the second Conv module, the third RepGFPN network, the eighth C2f module, and the third CCFM module are connected in sequence, and are directly connected to the fifth C2f module concat. The inputs of the first Upsample and the third RepGFPN network are connected to the SPPF module to receive the third effective feature layer output by the SPPF module; the input of the first RepGFPN network is connected to the third C2f module to receive the second effective feature layer output by the SPPF module; the input of the second RepGFPN network is connected to the second C2f module to receive the first effective feature layer output by the C2f module; the first CCFM module, the second CCFM module, and the third CCFM module respectively output features for prediction.

5. According to claim 4, the training method of an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: The RepGFPN network includes 5 CSPStage layers, the first CSPStage layer, the second CSPStage layer, the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are connected in sequence, and the output of the first CSPStage layer is also directly connected to the fifth CSPStage layer, and the output of the second CSPStage layer is also directly connected to the fourth CSPStage layer and the fifth CSPStage layer respectively; wherein the outputs corresponding to the third CSPStage layer, the fourth CSPStage layer, and the fifth CSPStage layer are respectively used as the output results of the RepGFPN network.

6. According to claim 5, the training method of an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: The CSPStage layer is divided into three processing paths. First, the input features are spliced, and then they are processed by 1*1 Conv convolution in the first path and output. After splicing, they are processed by 1*1 Conv convolution, 3*3Rep, and 3*3Conv in the second path and output. After splicing, they are processed by 1*1 Conv convolution in the second path and repeated 3*3Rep and 3*3Conv. The outputs of the three processing paths are spliced ​​by Concat and then processed by 1*1 Conv convolution and output.

7. The training method of an improved YOLOv8 chip surface defect lightweight target detection model according to claim 1 is characterized in that: Step 2: The process of training the improved YOLOv8 target detection model structure includes: Step 2.1, collect chip surface defect images and mark the defect categories to form a data set; divide the data set into a training set and a test set; Step 2.2: Train the model based on the training set and the test set.

8. According to claim 7, the training method of an improved YOLOv8 chip surface defect lightweight target detection model is characterized in that: Perform various image enhancement operations on the dataset in step 2.1, including random translation, flipping, rotation, and brightness adjustment.

9. An improved YOLOv8 chip surface defect lightweight target detection model, characterized in that: The improved YOLOv8 chip surface defect lightweight target detection model is trained using the training method of claim 1.

10. A chip surface defect target detection method, characterized in that: The steps include: Step 1: Use an industrial camera to obtain an image of the chip to be tested; Step 2: Input the chip image to be detected into the improved YOLOv8 chip surface defect lightweight target detection model and output the detection result.

Images