Surface defect segmentation model based on improved YOLOv8-seg chip and training method and application thereof

Abstract

The invention discloses an improved YOLOv8-seg-based chip surface defect segmentation model and a training method and application thereof, a network of an improved YOLOv8-seg segmentation detection model is built, a Backbone module comprises a multi-layer ShuffleNetV2 network structure, a picture is subjected to multiple convolution processing, and three effective feature layers are extracted; the Neck module adopts a ConcatSDI network structure and an EMA attention mechanism network structure, the ConcatSDI network structure is used for carrying out multi-level feature fusion on an effective feature layer output by the Backbone module to obtain a feature map, and the EMA attention mechanism network structure is used for carrying out feature weighting on the feature map; the head module is a segmentation head and outputs final feature information based on an output result of the Neck module; and after the network structure of the improved YOLOv8-seg segmentation detection model is constructed, training, verification and testing are carried out. According to the method, the improved yov8-seg segmentation detection model is used for accurately segmenting and detecting the surface defects of the chip, and particularly, the detection precision can still be ensured when the method faces complex backgrounds and low-contrast defects.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision, and in particular relates to a chip surface defect segmentation model based on an improved YOLOv8-seg, and a training method and application thereof. Background Art

[0002] In the semiconductor manufacturing process, the detection of chip surface defects is a key link to ensure product quality and reliability. Traditional defect detection methods often rely on manual inspection or rule-based algorithms. Manual inspection is not only labor-intensive, but also easily affected by subjective factors, while traditional image processing algorithms have limitations in dealing with complex backgrounds and diverse defects. Therefore, it is necessary to develop an efficient and accurate segmentation model detection algorithm that can be optimized in terms of real-time performance and accuracy to meet the improvement of defect detection accuracy and efficiency.

[0003] With the advancement of deep learning technology, especially the development of convolutional neural networks (CNN), deep learning-based defect detection methods have gradually been widely used. Deep learning models can automatically learn features and achieve efficient defect detection in complex backgrounds. The combination of target detection (such as the YOLO series) and image segmentation (such as Mask R-CNN) models makes defect positioning and boundary recognition more accurate. YOLOv8-seg is an improved version of the YOLO series model that combines the capabilities of target detection and image segmentation. This model can not only detect the presence of defects, but also accurately segment defect areas, with significant improvements in speed and accuracy, and is suitable for real-time processing and high-resolution image detection tasks.

[0004] However, the existing methods have relatively low detection accuracy when facing complex backgrounds and low-contrast defects, and further improvement and optimization are needed. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, this application proposes a chip surface defect segmentation model based on an improved YOLOv8-seg and its training method and application. The present invention uses the improved yolov8-seg segmentation detection model to accurately segment and detect chip surface defects, especially when facing complex backgrounds and low-contrast defects, the detection accuracy can still be guaranteed.

[0006] The technical solution adopted by the present invention is as follows:

[0007] A chip surface defect segmentation model training method based on improved YOLOv8-seg includes the following steps:

[0008] Step 1, build the network structure of the improved YOLOv8-seg segmentation detection model, the improved YOLOv8-seg segmentation detection model includes Backbone module, Neck module and head module; the Backbone module includes a multi-layer ShuffleNetV2 network structure, the input image is subjected to multiple convolution processing by the ShuffleNetV2 network structure, and 3 effective feature layers are extracted; the Neck module adopts the Concat_SDI network structure and the EMA attention mechanism network structure, and uses the Concat_SDI network structure to perform multi-level feature fusion on the effective feature layer output by the Backbone module to obtain a feature map, and uses the EMA attention mechanism network structure to perform feature weighting on the feature map; the head module is a segmentation head, which outputs the final feature information based on the output result of the Neck module;

[0009] Step 2: After completing the network structure of the improved YOLOv8-seg segmentation detection model, use pictures with chip surface defects to train, verify and test the built improved YOLOv8-seg segmentation detection model structure.

[0010] Further, the Backbone module is composed of a Conv_maxpool network structure, a first ShuffleNetV2 network structure, a second ShuffleNetV2 network structure, a third ShuffleNetV2 network structure, a fourth ShuffleNetV2 network structure, a fifth ShuffleNetV2 network structure, a sixth ShuffleNetV2 network structure, an SPPF network structure and an EMA, which are connected in sequence; wherein the fourth ShuffleNetV2 network structure, the sixth ShuffleNetV2 network structure and the SPPF network structure output the first effective feature layer, the second effective feature layer and the third effective feature layer respectively.

[0011] Furthermore, different ShuffleNetV2 network structures are selected according to different input channel step sizes. When the input channel step size is 1, the input is output in two ways after passing through the Channel Split layer. One is directly connected to the input of the Concat layer, and the other is connected to the Conv layer, the DWConv layer, and the Conv layer in sequence, and then connected to the input of the Concat layer. After the Concat layer concatenates the input tensors, they are input into the Channel Shuffle layer, and the Channel Shuffle layer outputs the rearranged feature map channels.

[0012] When the input channel stride is 2, the input is processed through two paths. One path includes the DWConv layer and the Conv layer connected in sequence, and the other path includes the Conv layer, the DWConv layer, and the Conv layer connected in sequence. The output ends of the two paths are connected to the input of the Concat layer. The input tensors are concatenated by the Concat layer and then input into the Channel Shuffle layer, which outputs the rearranged feature map channels.

[0013] Furthermore, the Neck module is composed of 2 Conv network structures, 4 C2f network structures, 2 upsampling, 4 Concat_SDI network structures, and 3 EMA attention mechanism network structures; among them, the first upsampling is connected to the first Concat_SDI network structure, the first C2f network structure, the second upsampling, the second Concat_SDI network structure, the second C2f network structure, the first EMA attention mechanism network structure, the first Conv network structure, the third Concat_SDI network structure, the third C2f network structure, the second EMA attention mechanism network structure, the second Conv network structure, the fourth Concat_SDI network structure, the fourth C2f network structure, and the third EMA attention mechanism network structure in sequence, and the first C2f network structure is also directly connected to the third Concat_SDI network structure.

[0014] Furthermore, the process of multi-level feature fusion in the Neck module is as follows: the Neck module receives the three valid feature layers output by Backbone, wherein the first valid feature layer P1 output by the fourth ShuffleNetV2 network structure is connected to the second Concat_SDI network structure and fused with the feature map N1 to be detected at the same scale;

[0015] The second effective feature layer P2 output by the sixth ShuffleNetV2 network structure is connected to the first Concat_SDI network structure, and is fused with the feature map N2 to be detected at the same scale;

[0016] The third effective feature layer P3 output by the SPPF module is connected to the fourth Concat_SDI network structure and fused with the feature map N3 to be detected at the same scale.

[0017] Furthermore, the EMA attention mechanism network structure includes a groups-style branch and a Cross-spatial learning branch; in the groups-style branch, the attention weight descriptor of the grouped feature map is extracted through three parallel routes, and the groups-style branch not only encodes the inter-channel information to adjust the importance of different channels, but also retains the precise spatial structure information into the channel.

[0018] Furthermore, in the groups-style branch, three parallel routes are used to extract the attention weight descriptor of the grouped feature map. The three parallel routes include two parallel paths on the 1x1 branch and one path on the 3x3 branch. For any given input feature map, the X equivalent of the feature map is divided into G sub-features along the 1x1 branch to learn different semantics, and the Y equivalent of the feature map is learned along another 1x1 branch. The outputs of the two 1x1 branches are decomposed into two vectors after Concat and conv processing, and two nonlinear Sigmoid functions are used to fit the two-dimensional binomial distribution after linear convolution to achieve different cross-channel interaction features. Finally, the outputs of the two parallel paths of the 1x1 branch and the grouped output are input to Re-weight to re-weight the features. The feature map is also input into the Cross-spatial learning branch after feature learning along the 3x3 branch.

[0019] Furthermore, in the Cross-spatial learning branch, the output of the 1x1 branch and the output of the 3x3 branch are used as tensors; the output of the 1x1 branch is sequentially connected to GroupNorm, Avg Pool, Softmax, and Matmul to output a vector of category probabilities; the output of the 3x3 branch is sequentially connected to Avg Pool, Softmax, and Matmul to output the probability of each category, and GroupNorm needs to be connected to Matmul in the 3x3 branch;

[0020] Finally, the nonlinear Sigmoid function is used to fuse the outputs of the two branches to obtain a probability value between 0 and 1, and then input it together with the grouped output into Re-weight to re-weight the features, finally obtaining the smoothed model parameters and features.

[0021] A chip surface defect segmentation model based on the improved YOLOv8-seg is trained using the above method.

[0022] A chip surface defect segmentation detection method based on improved YOLOv8-seg includes the following steps:

[0023] Step 1: Use an industrial camera to collect images of surface defects of the chip to be identified;

[0024] Step 2: Input the image to be identified obtained in step 1 into a chip surface defect segmentation model based on the improved YOLOv8-seg, perform segmentation prediction on the chip surface defects of the image to be identified, and output information on chip surface defects in the industrial production process.

[0025] Beneficial effects of the present invention:

[0026] 1. In view of the problem that the existing methods have low detection accuracy when facing complex backgrounds and low-contrast defects, the present invention improves the network structure of the feature extraction part and the feature fusion part of the YOLOv8 network, and introduces a lighter feature extraction module to better capture the details of low-contrast defects. In the feature fusion process, a more sophisticated fusion strategy is adopted to enable features from different levels to complement each other more effectively. The optimization of the network structure not only improves the detection capability of low-contrast defects, but also enhances the overall detection accuracy and reliability.

[0027] 2. By introducing the IoU (Intersection over Union) loss function to quantify the prediction effect of the mask, the IoU value is included in the total loss function as an additional loss term, which directly optimizes the prediction performance of the model in the boundary and overlapping areas, effectively improving the overall segmentation performance of chip surface defects, so that the model can more accurately identify and locate various types of defects in practical applications, and improve higher quality detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the method for constructing the improved YOLOv8-seg segmentation detection model of the present invention.

[0029] Figure 2 It is a schematic diagram of the network structure of the improved YOLOv8-seg segmentation detection model of the present invention.

[0030] Figure 3 It is a schematic diagram of the ShuffleNetV2 network structure; (a) is the network structure when the input step size is 1, and (b) is the network structure when the input step size is 2.

[0031] Figure 4 It is a structural diagram of the EMA attention mechanism module.

[0032] Figure 5 It is a schematic diagram of the Concat_SDI structure; among them, (c) is a schematic diagram of the Concat_SDI network structure and (d) is a schematic diagram of the Concat_SDI feature fusion.

[0033] Figure 6 This is a diagram showing the effect of segmenting chip surface defects based on the improved YOLOv8-seg segmentation detection model. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] Example 1

[0036] Combined with Figure 1-6 , the present application discloses a chip surface defect segmentation model training method based on an improved YOLOv8-seg, comprising the following steps:

[0037] Step 1: Build the network structure of the improved YOLOv8-seg segmentation detection model, as shown in the attached Figure 2 As shown in FIG. 1 , the network structure of the improved YOLOv8-seg segmentation detection model specifically includes: Backbone module, Neck module and head module.

[0038] 1. The Backbone module is mainly composed of 1 Conv_maxpool network structure, 6 ShuffleNetV2 network structures, 1 SPPF network structure and 1 EMA attention mechanism structure; it is composed of the Conv_maxpool network structure, the first ShuffleNetV2 network structure, the second ShuffleNetV2 network structure, the third ShuffleNetV2 network structure, the fourth ShuffleNetV2 network structure, the fifth ShuffleNetV2 network structure, the sixth ShuffleNetV2 network structure, the SPPF network structure and the EMA connected in sequence; among them, the fourth ShuffleNetV2 network structure, the sixth ShuffleNetV2 network structure and the SPPF network structure output the first effective feature layer, the second effective feature layer and the third effective feature layer respectively. In the Backbone module, the input image is subjected to multiple convolutions (ShuffleNetV2 network structure) to extract 3 effective feature layers.

[0039] More specifically, the Conv_maxpool network structure extracts features at different levels from the input image; the SPPF network structure performs multi-scale feature extraction and dimensionality reduction on the output of the last layer of the ShuffleNetV2 network structure; and EMA performs smoothing on the model parameters.

[0040] More specifically, the ShuffleNetV2 network structure introduces a channel shuffle mechanism based on deep separable convolution. The channel shuffle mechanism breaks the independence between channels through the "channel rearrangement" operation, which enables the network to enhance the feature expression ability while maintaining low computational complexity. Specifically, the ShuffleNetV2 network structure uses "grouped convolution" and "channel shuffle" to reduce the amount of computation, and improves the model's feature extraction ability by rearranging the channels of the feature map.

[0041] The ShuffleNetV2 network structure is as follows Figure 3 As shown in the figure, different ShuffleNetV2 network structures are selected according to different input channel step sizes. For example, when the input channel step size is 1, the ShuffleNetV2 network structure is as follows: Figure 3 As shown in (a), the input is divided into two output paths after passing through the Channel Split layer. One is directly connected to the input of the Concat layer, and the other is connected to the Conv layer, DWConv layer, and Conv layer in sequence, and then connected to the input of the Concat layer. After the Concat layer, the input tensor is spliced ​​and input into the Channel Shuffle layer, and the Channel Shuffle layer outputs the rearranged feature map channels.

[0042] When the input channel stride is 2, the ShuffleNetV2 network structure is as follows Figure 3 As shown in (b), the input is processed through two paths, one path includes the DWConv layer and the Conv layer connected in sequence, and the other path includes the Conv layer, the DWConv layer, and the Conv layer connected in sequence. The output ends of the two paths are connected to the input of the Concat layer. The input tensors are concatenated by the Concat layer and then input into the Channel Shuffle layer, which outputs the rearranged feature map channels.

[0043] Furthermore, the ReLU activation function is used in the ShuffleNetV2 network structure to introduce nonlinearity, which can make the model fit more complex functions; on the one hand, it can speed up the training process because its derivative of the positive semi-axis is always 1, reducing the gradient disappearance; on the other hand, it can improve the sparsity of the model because the negative value output is zero; and reduce the model parameters by setting the negative part to zero. In addition, the expression of the ReLU activation function is as follows:

[0044] f(x)=max(0,x)

[0045] Among them, x is the feature value of the previous layer input.

[0046] 2. Concat_SDI is used in the Neck module to achieve multi-level feature fusion. For each level of the feature map, Concat_SDI will fuse high-level features that contain more semantic information with low-level features that capture finer details. The Neck module is composed of 2 Conv network structures, 4 C2f network structures, 2 upsampling, 4 Concat_SDI network structures, and 3 EMA attention mechanism network structures; among them, the first upsampling and the first Concat_SDI network structure, the first C2f network structure, the second upsampling, the second Concat_SDI network structure, the second C2f network structure, the first EMA attention mechanism network structure, the first Conv network structure, the third Concat_SDI network structure, the third C2f network structure, the second EMA attention mechanism network structure, the second Conv network structure, the fourth Concat_SDI network structure, the fourth C2f network structure, the first The three EMA attention mechanism network structures are connected in sequence, and the first C2f network structure is also directly connected to the third Concat_SDI network structure; the first upsampling receives the output of the EMA attention mechanism network structure in the Backbone module, the fourth Concat_SDI network structure receives the third effective feature layer output by the SPPF in the Backbone module, the first Concat_SDI network structure receives the second effective feature layer output by the sixth ShuffleNetV2 network structure in the Backbone module, and the second Concat_SDI network structure receives the first effective feature layer output by the fourth ShuffleNetV2 network structure in the Backbone module.

[0047] More specifically, the Concat_SDI network structure is as follows Figure 5 As shown, it consists of four feature extraction layers, forward propagation calculation and output layer.

[0048] like Figure 5 As shown, the three effective feature layers output by Backbone are input into the Concat_SDI network structure, and the first effective feature layer P1 output by the fourth ShuffleNetV2 network structure is connected to the second Concat_SDI network structure, and fused with the feature map N1 to be detected at the same scale;

[0049] The second effective feature layer P2 output by the sixth ShuffleNetV2 network structure is connected to the first Concat_SDI network structure, and is fused with the feature map N2 to be detected at the same scale;

[0050] The third effective feature layer P3 output by the SPPF module is connected to the fourth Concat_SDI network structure and fused with the feature map N3 to be detected at the same scale.

[0051] For the above-mentioned feature maps N1, N2, and N3 to be detected, N1 is the feature map obtained after the second upsampling, N2 is the feature map generated by upsampling and fusion of the feature map obtained by downsampling from the previous network layer N1, the second effective feature layer P2 output by the sixth ShuffleNetV2 network structure, and the feature extraction network part F2. The features of these three different network levels are fused to obtain the feature map, and N3 is the feature map obtained after the second Conv network structure.

[0052] More specifically, the EMA attention mechanism network structure is as follows Figure 4 As shown in the figure, EMA improves feature learning capabilities by parallelizing 1x1 and 3x3 convolutions. The 1x1 branch keeps the channel dimension unchanged and captures fine-grained channel information, while the 3x3 branch aggregates multi-scale spatial information. Through feature grouping and cross-space learning, EMA can effectively model local and global feature dependencies and improve pixel-level attention and performance. Ultimately, the output of EMA is the same size as the input. EMA's efficient multi-scale attention mechanism across space achieves feature weighting by learning the importance of different regions of the image, weightedly fusion the feature map and the corresponding attention weights, and obtains the final multi-scale feature representation.

[0053] Combination Figure 4 , the EMA attention mechanism network structure includes a groups-style branch and a Cross-spatial learning branch; in the groups-style branch, three parallel routes are used to extract the attention weight descriptor of the grouped feature map, and the three parallel routes include two parallel paths on the 1x1 branch and one path on the 3x3 branch; for any given input feature map, the X equivalent of the feature map is divided into G sub-features along the 1x1 branch (i.e., the channel dimension direction) to learn different semantics, and the Y equivalent of the feature map is learned along another 1x1 branch (i.e., the channel dimension direction). Specifically, two 1D global average pooling operations are used to encode the channels along two spatial directions in the 1x1 branch.

[0054] The outputs of the two 1x1 branches are decomposed into two vectors after Concat and conv processing, that is, the two encoded features are connected along the image height direction so that they share the same 1x1 convolution without reducing the dimension of the 1x1 branch. Two nonlinear Sigmoid functions are used to fit the two-dimensional binomial distribution after linear convolution, and different cross-channel interaction features are realized between the two parallel paths of the 1x1 branch; finally, the outputs of the two parallel paths of the 1x1 branch and the grouped outputs are input into Re-weight to re-weight the features.

[0055] The feature map also undergoes feature learning along the 3x3 branch, where only one 3x3 kernel is stacked to capture multi-scale feature representations to expand the feature space; it is then input into the Cross-spatial learning branch.

[0056] The groups-styl branch not only encodes inter-channel information to adjust the importance of different channels, but also preserves the precise spatial structure information into channels.

[0057] In the cross-spatial learning branch, the output of the 1x1 branch and the output of the 3x3 branch are taken as tensors.

[0058] The output of the 1x1 branch is sequentially connected to GroupNorm, Avg Pool, Softmax, and Matmul to output a vector of category probabilities; the output of the 3x3 branch is sequentially connected to Avg Pool, Softmax, and Matmul to output the probability of each category, and GroupNorm needs to be connected to Matmul in the 3x3 branch;

[0059] Finally, the nonlinear Sigmoid function is used to fuse the outputs of the two branches to obtain a probability value between 0 and 1, and then input it together with the grouped output into Re-weight to re-weight the features, finally obtaining the smoothed model parameters and features.

[0060] Then, the output of the 1x1 branch is used to encode the global spatial information using 2D global average pooling, and the output of the smallest branch is directly converted to the corresponding dimensional shape before the channel feature joint activation mechanism. The output of the above parallel processing is multiplied by the matrix dot product operation to obtain the first spatial attention map. In addition, the 3x3 branch is also used to encode the global spatial information using 2D global average pooling, and the 1x1 branch is directly converted to the corresponding dimensional shape before the channel feature joint activation mechanism. On this basis, the second spatial attention map that retains the entire precise spatial position information is derived. Finally, the output feature map within each group is calculated as a set of the two generated spatial attention weight values, and then the Sigmoid function is used. It captures the pairwise relationship at the pixel level and highlights the global context of all pixels. The final output of EMA is the same size as X.

[0061] More specifically, the Concat_SDI network structure of the present invention adds SDI on the basis of ordinary splicing to achieve multi-level feature fusion, and the formula is as follows:

[0062]

[0063] Among them, f0 i represents the original feature map of level i, and represent the parameters of the spatial and channel attention mechanisms respectively, and f1 i is the processed feature map.

[0064] The SDI module combines the semantic information of high-level features with the detailed information of low-level features, enhancing the representation ability of each level of feature maps, thereby improving the accuracy in the image segmentation task. Compared with the traditional feature map splicing method, the skip connections of this module are simpler and more efficient, reducing the computational complexity and GPU memory usage;

[0065] Furthermore, the intermediate principle process of SDI is explained in detail by the SDI formula:

[0066] First, the number of channels of f1 i is reduced to a hyperparameter through a 1x1 convolution, obtaining a new feature map f2 i ;

[0067] Furthermore, the feature map is sent to the decoder, and the SDI module adjusts each level of feature map to the same resolution. The adjusted feature map is denoted as f3 ij , where i represents the target level and j represents the source level of the feature map. The adjustment operations include:

[0068] Furthermore, for the case of j < i, adaptive average pooling is used to adjust the size;

[0069] For the case of j = i, the identity mapping is used;

[0070] For the case of j > i, bilinear interpolation is used to adjust the size;

[0071] Furthermore, the adjusted feature map is smoothed through a 3x3 convolution, denoted as f4 ij , and finally, all the adjusted feature maps are fused together using the Hadamard product to enhance the semantic information and detailed information of each level of feature map, obtaining f5 i .

[0072] 3. The head module is the segmentation head of YOLOv8, specifically including 3 Segments. The 3 Segments respectively correspond to the first EMA attention mechanism network structure, the second EMA attention mechanism network structure, and the third EMA attention mechanism network structure in the Neck module, and output the final feature information, including class labels, bounding box coordinates, bounding box areas, segmentation masks, and confidences.

[0073] Segment uses the feature map extracted by the feature extraction network to predict the category of each pixel, uses the IoU loss function to predict the IoU value between the mask and the true mask, and adds IoU as an additional loss term to the total loss to optimize the model's prediction in the boundary and overlapping areas and improve the overall segmentation performance. The formula is:

[0074]

[0075] Among them, Intersection(Pred,Target) is the intersection area of ​​the predicted segmentation area and the true annotation area, and Union(Pred,Target) is the union area of ​​the predicted segmentation area and the true annotation area.

[0076] Step 2: After completing the construction of the improved YOLOv8-seg segmentation detection model network structure, it is necessary to train, verify and test the built improved YOLOv8-seg segmentation detection model structure. The specific steps are as follows:

[0077] Step 2.1, dataset preparation.

[0078] First, 900 images with chip surface defects are acquired using an industrial camera, and the acquired chip surface defect images are labeled using Labelme software, with the labeled name being "Crack". In this embodiment, the labeling format is PascalVOC format.

[0079] The images in the data set in step 2.1 are preprocessed; the preprocessing specifically includes data enhancement of the chip surface defect data set by random cropping, rotation, flipping, color jitter and contrast, which can increase the robustness of the model to different data situations. The image is adjusted to the input size required by the model, with a uniform size of 640*640. The pixel values ​​are normalized to the range of 0 to 1, or standardized (minus the mean and divided by the standard deviation) to make the model training more stable. At the same time, the Mosaic data enhancement method is used to splice four pictures, which indirectly increases the batch_size, so that a single GPU can achieve a better training effect. The enhanced data set is divided into a training set, a validation set and a test set, and the division ratio is: (training set + validation set): test set = 9:1, where training set: validation set = 9:1.

[0080] Step 2.2, model training.

[0081] The improved YOLOv8-seg segmentation detection model is trained using the training set and validation set of the dataset. The training epoch is set to 500 times. The backbone feature extraction network is frozen in the first 150 epochs and the initial learning rate is set to 0.01. The backbone feature extraction network is unfrozen in the last 300 epochs and the initial learning rate is set to 0.0001. The segmentation detection is verified on the test set of the constructed chip surface defect dataset to complete the accurate segmentation of chip surface defects.

[0082] Example 2

[0083] The above method can be used to obtain a chip surface defect segmentation model based on the improved YOLOv8-seg.

[0084] Example 3

[0085] A chip surface defect segmentation detection method based on improved YOLOv8-seg includes the following steps:

[0086] Step 1: Use an industrial camera to collect images of surface defects of the chip to be identified;

[0087] Step 2: Input the image to be identified obtained in step 1 into a chip surface defect segmentation model based on an improved YOLOv8-seg as described in Example 2, perform segmentation prediction of chip surface defects on the image to be identified, and output information on chip surface defects in the industrial production process, including category label name, bounding box coordinates, bounding box area, segmentation mask, and category confidence.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

[0093] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A chip surface defect segmentation model training method based on improved YOLOv8-seg, characterized in that: The steps include: Step 1, build the network structure of the improved YOLOv8-seg segmentation detection model, the improved YOLOv8-seg segmentation detection model includes Backbone module, Neck module and head module; the Backbone module includes a multi-layer ShuffleNetV2 network structure, the input image is subjected to multiple convolution processing by the ShuffleNetV2 network structure, and 3 effective feature layers are extracted; the Neck module adopts the Concat_SDI network structure and the EMA attention mechanism network structure, and uses the Concat_SDI network structure to perform multi-level feature fusion on the effective feature layer output by the Backbone module to obtain a feature map, and uses the EMA attention mechanism network structure to perform feature weighting on the feature map; the head module is a segmentation head, which outputs the final feature information based on the output result of the Neck module; Step 2: After completing the network structure of the improved YOLOv8-seg segmentation detection model, use pictures with chip surface defects to train, verify and test the built improved YOLOv8-seg segmentation detection model structure.

2. According to claim 1, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: The Backbone module is composed of a Conv_maxpool network structure, a first ShuffleNetV2 network structure, a second ShuffleNetV2 network structure, a third ShuffleNetV2 network structure, a fourth ShuffleNetV2 network structure, a fifth ShuffleNetV2 network structure, a sixth ShuffleNetV2 network structure, an SPPF network structure and an EMA, which are connected in sequence; wherein the fourth ShuffleNetV2 network structure, the sixth ShuffleNetV2 network structure and the SPPF network structure output the first effective feature layer, the second effective feature layer and the third effective feature layer respectively.

3. According to claim 1, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: Different ShuffleNetV2 network structures are selected according to different input channel step sizes. When the input channel step size is 1, the input is output in two ways after passing through the Channel Split layer. One is directly connected to the input of the Concat layer, and the other is connected to the Conv layer, DWConv layer, and Conv layer in sequence, and then connected to the input of the Concat layer. After the Concat layer concatenates the input tensors, they are input into the Channel Shuffle layer, and the Channel Shuffle layer outputs the rearranged feature map channels. When the input channel stride is 2, the input is processed through two paths. One path includes the DWConv layer and the Conv layer connected in sequence, and the other path includes the Conv layer, the DWConv layer, and the Conv layer connected in sequence. The output ends of the two paths are connected to the input of the Concat layer. The input tensors are concatenated by the Concat layer and then input into the Channel Shuffle layer, which outputs the rearranged feature map channels.

4. According to claim 1, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: The Neck module is composed of 2 Conv network structures, 4 C2f network structures, 2 upsampling, 4 Concat_SDI network structures, and 3 EMA attention mechanism network structures; among them, the first upsampling is connected to the first Concat_SDI network structure, the first C2f network structure, the second upsampling, the second Concat_SDI network structure, the second C2f network structure, the first EMA attention mechanism network structure, the first Conv network structure, the third Concat_SDI network structure, the third C2f network structure, the second EMA attention mechanism network structure, the second Conv network structure, the fourth Concat_SDI network structure, the fourth C2f network structure, and the third EMA attention mechanism network structure in sequence, and the first C2f network structure is also directly connected to the third Concat_SDI network structure.

5. According to claim 4, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: The process of multi-level feature fusion in the Neck module is as follows: the Neck module receives the three valid feature layers output by Backbone, among which the first valid feature layer P1 output by the fourth ShuffleNetV2 network structure is connected to the second Concat_SDI network structure and fused with the feature map N1 to be detected at the same scale; The second effective feature layer P2 output by the sixth ShuffleNetV2 network structure is connected to the first Concat_SDI network structure, and is fused with the feature map N2 to be detected at the same scale; The third effective feature layer P3 output by the SPPF module is connected to the fourth Concat_SDI network structure and fused with the feature map N3 to be detected at the same scale.

6. According to claim 1, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: The EMA attention mechanism network structure includes a groups-style branch and a Cross-spatial learning branch; in the groups-style branch, three parallel routes are used to extract the attention weight descriptor of the grouped feature map, and the groups-style branch not only encodes the inter-channel information to adjust the importance of different channels, but also retains the precise spatial structure information into the channel.

7. According to claim 6, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: In the groups-style branch, the attention weight descriptor of the grouped feature map is extracted through three parallel routes, including two parallel paths on the 1x1 branch and one path on the 3x3 branch; for any given input feature map, the X equivalent of the feature map is divided into G sub-features along the 1x1 branch to learn different semantics, and the Y equivalent of the feature map is learned along another 1x1 branch; The outputs of the two 1x1 branches are decomposed into two vectors after Concat and conv processing. Two nonlinear Sigmoid functions are used to fit the two-dimensional binomial distribution after linear convolution to achieve different cross-channel interaction features. Finally, the outputs of the two parallel paths of the 1x1 branch and the grouped outputs are input into Re-weight to re-weight the features. The feature map is also input into the Cross-spatial learning branch after feature learning along the 3x3 branch.

8. According to claim 6, a chip surface defect segmentation model training method based on improved YOLOv8-seg is characterized in that: In the Cross-spatial learning branch, the output of the 1x1 branch and the output of the 3x3 branch are used as tensors; the output of the 1x1 branch is sequentially connected to GroupNorm, Avg Pool, Softmax, and Matmul to output a vector of category probabilities; the output of the 3x3 branch is sequentially connected to Avg Pool, Softmax, and Matmul to output the probability of each category, and GroupNorm needs to be connected to Matmul in the 3x3 branch; finally, the outputs of the two branches are fused using a nonlinear Sigmoid function to obtain a probability value between 0 and 1, and then input into Re-weight together with the grouped output to re-weight the features, and finally the smoothed model parameters and features are obtained.

9. A chip surface defect segmentation model based on improved YOLOv8-seg, characterized in that: The chip surface defect segmentation model is trained by using the improved YOLOv8-seg chip surface defect segmentation model training method as described in claim 1.

10. A chip surface defect segmentation detection method based on improved YOLOv8-seg, characterized in that: The steps include: Step 1: Use an industrial camera to collect images of surface defects of the chip to be identified; Step 2: Input the image to be identified obtained in step 1 into the chip surface defect segmentation model based on the improved YOLOv8-seg as described in claim 9, perform segmentation prediction of chip surface defects on the image to be identified, and output information on chip surface defects in the industrial production process.

Images