Chip surface defect detection method based on improved YOLOV8 network

Abstract

The invention discloses a chip surface defect detection method based on an improved YOLOV8 network, and the method comprises the steps: carrying out the improvement of a YOLOV8n model, and obtaining an improved model; the improvement method comprises the following steps: replacing a C2f module with a C2fRVB module; a SimAM attention mechanism is introduced after the spatial pyramid pooling; replacing the detection head with a dynamic task detection head; collecting chip surface defect images and training the improved model; a to-be-detected chip surface image is input into the trained model for defect detection, compared with other models, the method has remarkable advantages in the aspect of chip surface defect detection precision, the detection precision is improved, good balance is achieved between computing resource consumption and real-time performance, and the method is suitable for large-scale popularization and application. And a new technical path is provided for a chip surface defect detection task.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and particularly to a chip surface defect detection method based on an improved YOLOV8 network. Background Art

[0002] With the rapid development of information technology and the electronics manufacturing industry, chips, as the core components of electronic devices, play a crucial role. The quality of chips directly determines the performance and reliability of end products. Therefore, in the chip production process, ensuring that the chip surface is defect-free is a key link. However, with the continuous reduction of chip size and the increasing complexity of functions, the types and forms of chip surface defects have become more diverse, posing great challenges to traditional defect detection methods. Traditional chip surface defect detection methods mainly rely on manual visual inspection or traditional image processing techniques.

[0003] Firstly, although the manual detection method is intuitive and applicable, its disadvantages lie in low detection efficiency and being easily interfered by human factors. For example, factors such as operator fatigue and inattention may lead to misdetection or missed detection, thereby affecting production quality. In addition, the repeatability of manual detection is poor, and it is difficult to ensure the consistency of each detection result. Especially in the case of large-scale production, the detection method relying on manual means has a high cost and is difficult to meet the requirements of efficient and accurate detection. Secondly, the chip surface defect detection technology based on traditional machine vision also faces some challenges. These traditional technologies generally rely on simple image processing algorithms, such as edge detection, morphological operations, etc., to identify defects on the chip surface. However, when facing complex defect patterns, these methods often show low robustness and accuracy. Especially when there are relatively small and irregularly shaped defects on the chip surface, the effect of traditional methods is poor, and it is easy to generate a high misdetection rate and missed detection rate. In addition, due to the complex texture and reflected light on the chip surface, the traditional machine vision system has poor adaptability to light changes, resulting in poor stability in different environments and being difficult to meet the high-standard detection requirements in large-scale production environments. Although image processing technology has improved the detection efficiency to a certain extent, it still has certain limitations when facing problems such as complex defect forms, large scale differences, and a lot of background noise.

[0004] In order to overcome these problems, many scholars and researchers have proposed some improved methods, such as improved versions of YOLO and FasterR-CNN, and combined methods of convolutional neural networks and autoencoders, to try to improve the accuracy and robustness of defect detection. Although these methods have solved the shortcomings of traditional technologies to a certain extent, there are still problems with low recognition accuracy of extremely small scale defects and defects of different shapes. Therefore, how to further improve the robustness and accuracy of the detection algorithm, especially in complex backgrounds and small defect detection scenarios, is still a key technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a chip surface defect detection method based on an improved YOLOV8 network to solve the problem that in chip surface defect detection, traditional target detection algorithms are often difficult to achieve ideal accuracy due to similar defect features, small defect target size and large difference in defect scale.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a chip surface defect detection method based on an improved YOLOV8 network, comprising:

[0007] Improve the YOLOv8n model to obtain an improved model;

[0008] Improvements include:

[0009] Replace the C2f module with the C2f_RVB module;

[0010] The SimAM attention mechanism is introduced after spatial pyramid pooling;

[0011] Replace the detection head with a dynamic task detection head;

[0012] Collect chip surface defect images and train the improved model;

[0013] The chip surface image to be inspected is input into the trained model for defect detection.

[0014] Further: The C2f_RVB module processes the input data in the following manner:

[0015] Performing convolution and splitting operations on the input data in sequence to obtain first intermediate data;

[0016] Input the first intermediate data into three sequentially connected RepViTBlocks to obtain first RepViTBlock data, second RepViTBlock data and third RepViTBlock data in sequence;

[0017] Concatenate the first RepViTBlock data, the second RepViTBlock data, the third RepViTBlock data, and the first intermediate data to obtain concatenated data;

[0018] Perform a convolution operation on the concatenated data to obtain the output of the C2f_RVB module.

[0019] Furthermore: The RepViTBlock includes a token mixer and a channel mixer connected in sequence.

[0020] Furthermore: The SimAM attention mechanism regards pixels as neurons, where neurons with high weights have an inhibitory effect on neurons with low weights.

[0021] Furthermore: The weight of a neuron is characterized by an energy function, and its expression is:

[0022]

[0023] y t = w t x t + b t

[0024]

[0025] where, e t represents the energy function of the target neuron t, w t and b t represent the weight and bias of the linear transformation respectively, y represents the output of the neuron, y t represents the output of the target neuron t, y0 represents the sum of the outputs of other neurons, represents the output of the i-th other neuron, x i represents the i-th other neuron, x t represents the set of other neurons, M represents the number of neurons on this channel; i is a counting identifier, and are the intermediate parameters obtained by linearly transforming x i and t respectively.

[0026] Furthermore: The dynamic task detection head receives the output of the improved model neck and outputs the surface defect detection result. The specific steps include:

[0027] Receive the output of the improved model neck and perform feature scaling to obtain the aligned neck output features;

[0028] Perform shared convolution on the aligned neck output features, and respectively learn the localization task interaction features and the classification task interaction features from the convolutional layer through a feature extractor;

[0029] Concatenate the positioning task interaction feature and the classification task interaction feature to obtain a combined feature;

[0030] Input the combined feature into the positioning branch and the classification branch respectively to obtain the positioning prediction result and the classification prediction result;

[0031] Output the surface defect detection result according to the positioning prediction result and the classification prediction result.

[0032] Furthermore: The processing method of the positioning branch for the combined feature includes:

[0033] Generate a mask and an offset value of the combined feature through a generator;

[0034] Perform task decomposition on the combined feature to obtain the decomposed positioning interaction feature;

[0035] Perform deformable convolution according to the decomposed task interaction feature, the mask and the offset value of the combined feature to obtain the positioning prediction result.

[0036] Furthermore: The processing method of the classification branch for the combined feature includes:

[0037] Perform a convolution operation on the combined feature to obtain a convolved combined feature;

[0038] Perform task decomposition on the combined feature to obtain the decomposed classification interaction feature;

[0039] Multiply the convolved combined feature and the decomposed task interaction feature to obtain the classification prediction result.

[0040] Furthermore: When performing task decomposition on the combined feature, perform specific task decomposition by dynamically calculating different task-specific features at different levels.

[0041] The beneficial effects of the present invention are:

[0042] 1. Designed the C2f_RVB module, which uses the technology of lightweight vision transformers to optimize the feature expression ability, not only reducing the model parameter quantity and computational complexity, but also improving the detection accuracy and processing speed of the model;

[0043] 2. Introduce the SimAM attention mechanism after spatial pyramid pooling, enabling the spatial and channel parameters of the image to be jointly concerned and improving the perception ability of defect features;

[0044] 3. Aiming at the problem that small target defects are difficult to detect in object detection, designed a dynamic task detection head to promote task interaction between the positioning and classification detection heads, thereby improving the accuracy of small target detection. Description of the Drawings

[0045] Figure 1 It is a flowchart of a chip surface defect detection method based on an improved YOLOV8 network.

[0046] Figure 2 It is the network structure diagram of the improved model.

[0047] Figure 3 It is the schematic diagram of the RepViTBlock structure.

[0048] Figure 4 It is the schematic diagram of the RepVGGDW structure.

[0049] Figure 5 It is the schematic diagram of the SimAM attention mechanism structure.

[0050] Figure 6 It is the schematic diagram of the task decomposition structure. Specific implementation manners

[0051] The following describes the specific implementation manners of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation manners. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions created using the concept of the present invention are within the scope of protection.

[0052] As Figure 1 shown, in an embodiment of the present invention, there is provided a chip surface defect detection method based on an improved YOLOV8 network, including:

[0053] Improve the YOLOv8n model to obtain an improved model;

[0054] The improvement method includes:

[0055] Replace the C2f module with the C2f_RVB module;

[0056] Introduce the SimAM attention mechanism after the spatial pyramid pooling;

[0057] Replace the detection head with a dynamic task detection head;

[0058] Collect chip surface defect images and train the improved model;

[0059] Input the chip surface image to be detected into the trained model for defect detection.

[0060] The network structure diagram of the improved model is as Figure 2 shown, which also includes the specific structures of the C2f_RVB module and the dynamic task detection head (TADDH).

[0061] In Figure 2 it can be learned that the processing method of the C2f_RVB module for the input data includes:

[0062] Performing convolution and splitting operations on the input data in sequence to obtain first intermediate data;

[0063] Inputting the first intermediate data into three sequentially connected RepViTBlocks to obtain first RepViTBlock data, second RepViTBlock data, and third RepViTBlock data in sequence;

[0064] Concatenating the first RepViTBlock data, second RepViTBlock data, third RepViTBlock data, and the first intermediate data to obtain concatenated data;

[0065] Performing a convolution operation on the concatenated data to obtain the output of the C2f_RVB module.

[0066] The C2f module in YOLOv8n adopts multiple Bottleneck structures for feature fusion and information transmission. The Bottleneck module includes feature partitioning and residual connections. By repeatedly using the Bottleneck module for feature processing, the complexity and computational burden of the network structure are increased. C2f_RVB uses RepViTBlock to replace Bottleneck on the basis of the C2f module, enhancing the network's feature processing ability. This design enables C2f_RVB to more effectively integrate local and global information, improving the richness and accuracy of feature representation. The use of RepViTBlock optimizes the computational efficiency and reduces the number of parameters and computational requirements through depthwise separable convolutions.

[0067] Among them, RepViTBlock combines the design concept of lightweight vision transformers and the practical advantages of convolutional neural networks to improve the operation efficiency of the model and reduce the number of parameters of the model, in order to solve the problem that as the model structure becomes more complex, convolutional neural networks usually require a large amount of computational resources and storage space. The structural schematic diagram of RepViTBlock is as Figure 3 shown, including a token mixer and a channel mixer connected in sequence. In RepViTBlock, the data stream first passes through the token mixer. In this embodiment, the token mixer uses a depthwise separable convolution unit;

[0068] As Figure 4As shown, the depthwise separable convolution unit includes two convolutional layers, depth convolution and 1×1 group convolution. The outputs of these two convolutions are added together, then fused with the original input to achieve residual connection, and then output after normalization through the BN layer. The token mixer mainly processes the features in local or neighboring regions, enhances the model's ability to capture spatial details, and reduces the number of parameters and computational requirements, enabling the module to maintain a low computational cost while maintaining efficient feature processing.

[0069] In the channel mixer, feature fusion between channels is performed through two 1×1 convolutional layers. This part not only improves the feature expression ability but also performs dimensional transformation and feature compression, helping the model integrate information from different channels and achieve global feature integration.

[0070] Generally speaking, the attention mechanism can generally be divided into spatial attention mechanism and channel attention mechanism. The spatial attention mechanism scores each pixel point in the image and then performs weighted averaging on the scoring results to obtain a more attention - focused area. The channel attention mechanism, on the other hand, correlates and integrates the information between different channels to enhance the model's representation ability.

[0071] The SimAM attention mechanism used in this application regards pixel points as neurons and can directly calculate three - dimensional attention weights. The fusion process of the three - dimensional attention mechanism is as Figure 5 shown. In the SimAM attention mechanism, high - weight neurons have an inhibitory effect on low - weight neurons. Therefore, neurons showing obvious spatial inhibitory effects should be given higher priority. The weight of a neuron is characterized by an energy function, that is, the higher the energy, the higher the weight. A high - energy neuron may correspond to the key features of a defective area in the image, while a low - energy neuron corresponds to the background or secondary area. The expression of the energy function of the neuron is:

[0072]

[0073] y t =w t x t +b t

[0074]

[0075] where, e t represents the energy function of the target neuron t, w t and b t represent the weights and biases of the linear transformation respectively, y represents the output of the neuron, y t represents the output of the target neuron t, y0 represents the sum of the outputs of other neurons, represents the output of the i - th other neuron, xi represents the i-th other neuron, x t represents the set of other neurons, M represents the number of neurons on this channel; i is a counting identifier, and are intermediate parameters obtained by linearly transforming x i and t respectively.

[0076] Solving the minimum value of the energy function can find the linear separability between the target function t and the remaining neurons in the same channel. Usually, y t and y0 adopt binary labels, that is, -1 and 1. At the same time, regularization is added to the equation, and the final energy function is given by the formula:

[0077]

[0078] e t2 (w t , b t , y, x i ) = (1 - (tw t + b t )) 2 + λw t 2

[0079] e t (w t , b t , y, x i ) = e t1 (w t , b t , y, x i ) + e t2 (w t , b t , y, x i )

[0080] where e t1 (w t , b t , y, x i ) and e t2 (w t , b t , y, x i ) are both intermediate calculation parameters, λ is the regularization coefficient, used to balance the complexity of the model and the sensitivity to the uniqueness of neurons. Through the above formula, a fast closed-form solution for w t and b t can be obtained:

[0081]

[0082] where, and are the mean and variance of neurons obtained on a single channel. Therefore, it is assumed that all pixels in a single channel follow the same distribution. Based on this, the mean and variance of all neurons are calculated and reapplied to all neurons on this channel. Thus, the minimum energy e t * can be calculated using the following equation:

[0083]

[0084] where, and represent the mean and variance of all neurons respectively. The above equation shows that the lower the energy, the greater the difference between neuron t and its surrounding neurons, and the higher its weight.

[0085] In the refinement stage of the entire module, the following equation is also included:

[0086]

[0087] where sigmoid is the activation function used to limit from being too large. E represents the sum of all across the channel and spatial dimensions. X is the feature of the image input, represents the result after weighting the input feature, represents element-wise multiplication.

[0088] Since the traditional YOLOv8 algorithm has problems such as high false detection and missed detection rates in detecting small targets in chip surface defects, this study proposes adding a dynamic detection head (TADDH) to the head of YOLOv8 to solve this problem. The single-stage object detection optimizes the two tasks of classification and localization and adopts a dual-branch head design, but it may cause spatial inconsistency between predictions. At the same time, this dual-branch design may lead to a lack of interaction between the two tasks, resulting in inconsistent predictions when performing tasks. We combine a task-aligned single-stage object detection, aiming to more accurately align the two tasks by designing a new head structure and an alignment-oriented learning method.

[0089] The dynamic task detection head receives the output of the improved model neck and outputs the surface defect detection result. Its specific steps include:

[0090] Receiving the output of the improved model neck and performing feature scaling to obtain the aligned neck output feature;

[0091] Performing shared convolution on the aligned neck output feature, and learning the localization task interaction feature and the classification task interaction feature from the convolutional layer through the feature extractor respectively;

[0092] Concatenate the location task interaction features and the classification task interaction features to obtain joint features;

[0093] Input the joint features into the location branch and the classification branch respectively to obtain the location prediction result and the classification prediction result;

[0094] Output the surface defect detection result according to the location prediction result and the classification prediction result.

[0095] On the one hand, perform feature scaling to deal with the problem of inconsistent target scales detected by each detection head; on the other hand, using shared convolutions can significantly reduce the number of parameters, making the model lighter, especially on resource-constrained devices.

[0096] The processing method of the location branch for the joint features includes:

[0097] Generate the mask and offset values of the joint features through a generator;

[0098] Decompose the joint features to obtain the decomposed location interaction features;

[0099] Perform deformable convolution according to the decomposed task interaction features, the mask and offset values of the joint features to obtain the location prediction result.

[0100] Deformable convolution flexibly samples on multi-layer features through multi-layer offset learning. Its modulation mechanism adjusts the position and intensity of the sampling points, enabling the network to dynamically adjust the sampling layout and influence weights, improving adaptability, which makes these two tasks easier to cooperate, thus more accurately adjusting their predictions.

[0101] The processing method of the classification branch for the joint features includes:

[0102] Perform a convolution operation on the joint features to obtain the convolution joint features;

[0103] Decompose the joint features to obtain the decomposed classification interaction features;

[0104] Multiply the convolution joint features and the decomposed task interaction features to obtain the classification prediction result.

[0105] When decomposing the joint features, perform object classification and location on the calculated task interaction features. These two tasks can well perceive each other's states. However, due to the single-branch design, the task interaction features will inevitably introduce a certain degree of feature conflict between the two different tasks because the goals of classification and location are different, so they focus on different types of features, such as different levels and receptive fields. Therefore, this paper proposes a layer attention mechanism to perform task decomposition by dynamically calculating different task-specific features at different levels.

[0106] As Figure 6 shown, for classification and localization, their task-specific features are calculated separately:

[0107]

[0108] Among them, is the task interaction feature, which is used to capture the interaction information between the classification and localization tasks, is the task-specific feature, which is used to weight the task interaction feature, emphasizing the features that are more important for a specific task (classification or localization), ω k is the k-th element of the learned layer attention weight ω ∈ R N , and ω is calculated based on the cross-layer task interaction feature and can capture the interaction relationship between layers:

[0109] ω = σ(f c2 (σ(f c1 (x inter ))))

[0110] Among them, f c1 and f c2 are two fully connected layers, σ is the sigmoid activation function, x inter is obtained by applying average pooling to X inter , and X inter is obtained by concatenating X inter k .

[0111] According to the improved YOLOv8 chip surface defect detection method proposed in this study, compared with the existing technology, the present invention has significant advantages, especially outstanding performance in the detection of small-sized and low-contrast defects. First of all, by designing a new feature extraction module C2f_RVB, the feature expression ability of the model is enhanced, making the detection of chip surface defects more accurate, thereby reducing the need for manual intervention and improving the stability and reliability of the production process. At the same time, the SimAM attention mechanism and the aligned dynamic detection head TADDH are adopted, which especially improve the detection accuracy of small target defects, optimize the network resource consumption in the detection process, and make the overall system more efficient. This method not only improves the detection accuracy and recall rate, but also optimizes the use of network resources by reducing unnecessary calculations, reduces the operation burden, and further improves the technical effect. These innovations make the chip surface defect detection more intelligent and automated, reduce the dependence on manual intervention, and significantly improve the efficiency of the production process.

[0112] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A chip surface defect detection method based on an improved YOLOV8 network, characterized in that, Including: Improve the YOLOv8n model to obtain an improved model; The improvement method includes: Replace the C2f module with the C2f_RVB module; Introduce the SimAM attention mechanism after spatial pyramid pooling; Replace the detection head with a dynamic task detection head; Collect chip surface defect images and train the improved model; Input the chip surface image to be detected into the trained model for defect detection.

2. The chip surface defect detection method based on the improved YOLOV8 network according to claim 1, characterized in that, The processing method of the C2f_RVB module for input data includes: Perform convolution and splitting operations on the input data in sequence to obtain first intermediate data; Input the first intermediate data into three consecutive RepViTBlocks to obtain first RepViTBlock data, second RepViTBlock data, and third RepViTBlock data in sequence; Concatenate the first RepViTBlock data, second RepViTBlock data, third RepViTBlock data, and the first intermediate data to obtain concatenated data; Perform a convolution operation on the concatenated data to obtain the output of the C2f_RVB module.

3. The chip surface defect detection method based on the improved YOLOV8 network according to claim 2, wherein, The RepViTBlock includes a token mixer and a channel mixer connected in sequence.

4. The chip surface defect detection method based on the improved YOLOV8 network according to claim 1, characterized in that, The SimAM attention mechanism regards pixel points as neurons, where high-weight neurons have an inhibitory effect on low-weight neurons.

5. The chip surface defect detection method based on the improved YOLOV8 network according to claim 4, wherein, The weight of the neuron is characterized by an energy function, and its expression is: y t = w t x t + b t Among them, e t represents the energy function of the target neuron t, w t and b t represent the weight and bias of the linear transformation respectively, y represents the output of the neuron, y t represents the output of the target neuron t, y0 represents the sum of the outputs of other neurons, represents the output of the i-th other neuron, x i represents the i-th other neuron, x t represents the set of other neurons, M represents the number of neurons on this channel; i is a counting identifier, and are the intermediate parameters obtained by linearly transforming x i and t respectively.

6. The chip surface defect detection method based on the improved YOLOV8 network according to claim 1, characterized in that, The dynamic task detection head receives the output of the neck of the improved model and outputs the surface defect detection result. The specific steps include: Receive the output of the neck of the improved model and perform feature scaling to obtain aligned neck output features; Perform shared convolution on the aligned neck output features, and learn localization task interaction features and classification task interaction features from the convolutional layer through a feature extractor respectively; Concatenate the localization task interaction features and classification task interaction features to obtain joint features; Input the joint features into the localization branch and the classification branch respectively to obtain localization prediction results and classification prediction results respectively; Output the surface defect detection result according to the localization prediction result and the classification prediction result.

7. The chip surface defect detection method based on the improved YOLOV8 network according to claim 6, characterized in that, The processing method of the localization branch for the joint features includes: Generate a mask and offset values of the joint features through a generator; Perform task decomposition on the joint features to obtain decomposed localization interaction features; Perform deformable convolution according to the decomposed task interaction features, the mask and offset values of the joint features to obtain localization prediction results.

8. The chip surface defect detection method based on the improved YOLOV8 network according to claim 6, characterized in that, The processing method of the classification branch for the joint features includes: Perform a convolution operation on the joint features to obtain convolution joint features; Perform task decomposition on the joint features to obtain decomposed classification interaction features; Multiply the convolution joint features and the decomposed task interaction features to obtain classification prediction results.

9. The chip surface defect detection method based on the improved YOLOV8 network according to any one of claims 7 or 8, characterized in that, When performing task decomposition on the joint features, specific task decomposition is performed by dynamically calculating different task-specific features at different levels.

Images