A PCB defect image detection method for improving YOLOv5 by fusing GAM and SimAM attention mechanisms

Abstract

The application discloses a PCB defect image detection method for improving YOLOv5 by fusing GAM and SimAM attention mechanisms, and the method is improved in view of the insufficient detection capability of a YOLOv5 deep convolutional neural network model for small target defects of a PCB, a design idea of fusing GAM and SimAM attention mechanisms by using an alternating mobile convolution and attention (MOAT) module is proposed, a GSAM attention mechanism is proposed, a GSAM module is embedded into the YOLOv5 model, and an improved YOLOv5-GSAM model is proposed.Compared with the original YOLOv5 model, the improved model has improved sensitivity, specificity and other indicators, the detection precision of the model for small target defects of a PCB is improved, and the detection model has good robustness and generalization capability.

Description

Technical Field

[0001] This invention relates to the field of printed circuit board (PCB) defect detection, specifically to a PCB defect image detection method that improves YOLOv5 by integrating GAM and SimAM attention mechanisms. Background Technology

[0002] Printed circuit boards (PCBs) are the carriers connecting other electronic components and are a crucial foundational component of the electronics and information industry. Their manufacturing process is complex and precise, and the requirements for PCB inspection in industrial production are constantly increasing. Traditional inspection methods, such as manual visual inspection and machine electrical testing, suffer from low efficiency, high false negative rates, high costs, and easy damage to PCBs, and can no longer meet the demands for high-efficiency, high-precision, and high-performance PCB production. Image processing-based defect detection methods mainly employ principal component analysis, Fourier transform, wavelet transform, and image pattern matching to extract features from PCB images, and then classify defects based on matching templates to these features. However, existing technologies are sensitive to factors such as location and lighting, limiting their application in real industrial production environments. While traditional machine learning and image processing techniques have shown good results in image recognition and defect detection, deep learning-based defect detection methods have surpassed the defect classification capabilities of traditional machine learning. Therefore, utilizing deep learning technology for PCB defect detection is particularly important.

[0003] Currently, deep learning algorithms show great promise in fields such as computer vision, natural language processing, and robotics. In the field of PCB image inspection, deep learning algorithms automatically extract features from PCB images to construct optimal parameter models, thereby locating defects and achieving defect detection. Compared with traditional image processing and machine learning algorithms, deep learning-based algorithms overcome the sensitivity of traditional defect detection to location and lighting conditions. They effectively reduce dependence on hyperparameter selection, avoid complex manual feature extraction operations, improve the generalization ability of algorithm models, and have significant advantages in accuracy, exhibiting optimal overall performance and demonstrating outstanding results in image recognition and defect detection. Summary of the Invention

[0004] To address the problems in the existing technology, the present invention aims to overcome the shortcomings of the existing technology and provide an improved YOLOv5 PCB defect image detection method that integrates GAM and SimAM attention mechanisms. This method utilizes the attention mechanism to focus on PCB defect areas, effectively selects feature information of small defect areas, assigns higher weights to defect areas to prevent defect information from being ignored during feature extraction, thereby improving the performance of the network model in detecting small target defects on PCBs.

[0005] An improved PCB defect image detection method for YOLOv5 by integrating GAM and SimAM attention mechanisms includes the following two steps:

[0006] S1: Establish an improved image detection model, specifically: based on the original YOLOv5 model, integrate the global attention mechanism GAM and the simple parameterless attention mechanism SimAM into the original YOLOv5 model to obtain an improved YOLOv5 PCB defect image model.

[0007] S2: Establish an image dataset for training the improved YOLOv5 PCB defect image model to obtain a trained improved image model.

[0008] Preferably, in step S1, a GAM / SimAM module is embedded between the two Conv modules in the C3_2 module connected by the concatenation of layers 4 and 6 of the original YOLOv5 backbone network. Preferably, in step S1, the GAM attention mechanism and the SimAM attention mechanism are fused into a GSAM structural attention mechanism, which is then integrated into the YOLOv5 model to obtain the improved YOLOv5 PCB defect image model.

[0009] In a further preferred embodiment, the GSAM structure replaces the SE-Net module in the MBConv block with the GAM module, and replaces the self-attention operation in the Transformer with the SimAM attention mechanism.

[0010] In a further preferred embodiment, the GSAM structure attention mechanism is designed based on the inverse residual structure and linear bottleneck structure in the MOAT structure. Before the GAM attention mechanism, a 1×1 convolution is applied to expand the input channel by 4 times. Then, a 3×3 depthwise separable convolution is used to effectively improve the local spatial interaction between pixels. After the attention mechanism focuses on the information of interest, the features are mapped back to the original channel size through a 1×1 convolution. Finally, residual connections are realized.

[0011] Preferably, in step S2, the image set consists of a training dataset and a test dataset.

[0012] More preferably, the training dataset consists of defect images and background images, wherein the defect images consist of images of different colors, images of residue, images of foreign objects, and other defect images, and the background images are used to suppress interference caused by background information.

[0013] In a further preferred embodiment, the training dataset is divided into a training image set and a validation image set, wherein the training image set and the validation image set are formed by dividing all types of images in the training dataset in the same proportion.

[0014] More preferably, the training process of the improved YOLOv5 PCB defect image model is as follows: the training image set is input into the improved YOLOv5 PCB defect image model for training, and the verification image set is input into the trained improved YOLOv5 PCB defect image model for verification testing, so as to obtain the trained improved image model.

[0015] More preferably, the test image set is input into the trained improved image model to test the generalization ability of the trained improved image model.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] This invention proposes an improved YOLOv5 PCB defect image detection method that integrates GAM and SimAM attention mechanisms. By effectively combining the high specificity of GAM with the high sensitivity of SimAM, more accurate three-dimensional attention weights are obtained, which improves the model's detection accuracy for small target defects on PCBs. This results in the PCB defect detection algorithm of this invention having better robustness and generalization ability.

[0018] This invention proposes an improved PCB defect image detection method based on YOLOv5, which integrates GAM and SimAM attention mechanisms. The improved YOLOv5-GSAM model is used to detect PCB defects. Compared to the original model, the improved model demonstrates higher sensitivity and specificity, indicating better PCB defect detection performance. In experimental comparisons under identical conditions, the model achieves a sensitivity of 89.79% at a threshold of 0.5, demonstrating superior performance in PCB defect detection tasks. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the improved YOLOv5 PCB defect image detection method that integrates GAM and SimAM attention mechanisms provided in this invention.

[0020] Figure 2 This is a schematic diagram of the YOLOv5 original target detection network in this invention;

[0021] Figure 3 This is a schematic diagram illustrating the design concept of the MOAT structure in this invention;

[0022] Figure 4 This is a structural diagram of the GSAM module in this invention;

[0023] Figure 5 This is a structural diagram of the improved YOLOv5-GSAM model in this invention;

[0024] Figure 6 This is a schematic diagram illustrating the composition of the images in the image set of this invention;

[0025] Figure 7 This refers to the PCB defect sample type in this invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] like Figure 1 As shown, a method for improving YOLOv5 PCB defect image detection by integrating GAM and SimAM attention mechanisms includes the following two steps:

[0028] Step 1: Establish an improved image detection model. Specifically, based on the original YOLOv5 model, integrate the Global Attention Mechanism (GAM) and the Simple Parameter-Free Attention Module (SimAM) into the YOLOv5 model.

[0029] Specifically, such as Figure 2 As shown, the entire YOLOv5 network structure is mainly composed of two different C3 modules. The C3_1 and C3_2 modules are combinations of CSPNet with ResNet and Conv structures, respectively. However, Conv is not ideal for feature focusing on small target defects. Therefore, in this embodiment, the structure of Conv in the original C3_2 module is improved. Specifically, a GAM / SimAM module is embedded between the two Conv modules in the C3_2 module connected by concatenation at layers 4 and 6 of the YOLOv5 backbone network, which introduces an attention mechanism. Here, GAM and SimAM are briefly introduced:

[0030] Global Attention (GAM) is an attention mechanism capable of capturing important features across all three dimensions, primarily composed of two sub-modules: channel attention and spatial attention. Its channel attention sub-module does not use pooling operations to downsample and compress two-dimensional features in the H×W direction; instead, it uses a dimension swapping arrangement to retain all information across all three dimensions. Then, it uses a two-layer MLP network to amplify cross-dimensional channel and spatial dependencies, and finally restores the original state by reversing the order before the swapping. In GAM's spatial attention sub-module, two 7×7 convolutional layers are used to fuse spatial information. Similar to the SE-Net attention mechanism, it first reduces the number of channels to 1 / r of the original number, then restores the number of channels, thus reducing network computation while increasing the network's non-linearity.

[0031] SimAM, a simple, parameter-free attention mechanism, is a three-dimensional attention mechanism proposed based on neuroscience theory. In neuroscience, each neuron is analogous to an element in a feature matrix; activating a neuron inhibits surrounding neurons (i.e., spatial inhibition), similar to the weighting process in attention mechanisms. Therefore, the SimAM module defines an energy function guided by neuroscience knowledge, assigning higher weights to each neuron with spatial inhibition effects, ultimately deriving a closed-form solution. Thus, this module directly calculates the three-dimensional attention weights corresponding to each element in the feature map using mathematical formulas, without increasing the parameters of the original network. The closed-form solution is as follows:

[0032]

[0033]

[0034] In the formula, and These are the mean and variance of all neurons except t in that channel, respectively. Assuming all pixels in a single channel follow the same distribution, the minimum energy can be calculated using the following formula.

[0035]

[0036] In the formula, Let be the energy represented by the target neuron, and t be the target neuron in a single channel of the input feature map. and These represent the mean and variance of all neurons, and λ is the compensation coefficient. The lower the energy of neuron t, the greater its difference from surrounding neurons, and the higher its importance. Its importance can be determined by... We obtain the result. According to the definition of the attention mechanism, the features need to be enhanced by adding a sigmoid function to limit excessively large values ​​in E.

[0037]

[0038] In the formula, E represents all Grouping based on channel and spatial dimensions.

[0039] Both GAM and SimAM attention mechanisms are used to directly estimate complete 3D weights, reduce information diffusion, and enhance the dimensionality interaction of features. Furthermore, the attention mechanism is a plug-and-play module that can be placed after the feature layers of any convolutional neural network, whether in the backbone or feature enhancement network. Therefore, determining the placement of the attention mechanism is crucial. In this embodiment, the attention mechanism is applied to the feature fusion network. The feature maps of layers 4 and 6 of the YOLOv5 backbone network have rich information, containing both low-level detailed features and high-level semantic information. Therefore, focusing the attention mechanism at this location is beneficial for effectively finding salient regions in complex scenes and improving the detection accuracy of small target defects. Therefore, in this embodiment, the structure of Conv in the original C3_2 module is improved by embedding a GAM / SimAM module between the two Conv modules, enhancing the feature extraction capability of the C3_2 module.

[0040] It should be noted that, compared to applying the attention mechanism to the feature fusion network, placing the attention mechanism in the backbone network may result in the inability to use the pre-trained weights. In addition, the feature map size of the backbone network is large, and embedding the attention mechanism here may increase the computational cost.

[0041] Furthermore, the two attention mechanisms, GAM and SimAM, are merged into the GSAM structural attention mechanism, which is then incorporated into the YOLOv5 model (referred to as YOLOv5-GSAM).

[0042] Specifically, such as Figure 3 As shown, the design philosophy of the MOAT structure is to effectively merge the best blocks from the Moving Convolutional (MBConv) block and the Transformer block. Depthwise separable convolutions from the moving convolutional block are incorporated into the MLP module to promote local interactions between pixels, and the optimized MLP module is rearranged before its self-attention operation. Depthwise separable convolutions not only enhance the network's representational ability but also generate better downsampled features. Simultaneously, the removal of the SE-Net module from the MBConv block allows the Transformer to maintain its self-attention operation to capture global information. Furthermore, the optimized MLP module retains the original inverse residual structure and linear bottleneck structure design to prevent ReLU from destroying the feature map, thus preserving more feature information and ensuring the model's expressive power.

[0043] In this embodiment, drawing inspiration from the design principles of the MOAT structure, the C3_2 module is optimized by integrating the GAM and SimAM attention mechanisms into the Bottleneck portion, thus proposing a GSAM structure attention mechanism, such as... Figure 4 As shown, the GSAM structure follows the design of the inverse residual structure and linear bottleneck structure in the MOAT structure. Before the GAM attention mechanism, a 1×1 convolution is applied to expand the input channels by 4 times. Then, a 3×3 depthwise separable convolution is used to effectively improve the local spatial interaction between pixels. After the attention mechanism focuses the information of interest, the features are mapped back to the original channel size through a 1×1 convolution. Finally, residual connections are realized.

[0044] The GSAM architecture replaces the SE-Net module in the MBConv block with the GAM module. While the SE-Net module only focuses on channel features, the GAM module allows information from different dimensions to interact, thus integrating more information. The GSAM architecture replaces the self-attention operation in the Transformer with the SimAM attention mechanism. Firstly, the multi-head attention module in the Transformer has more parameters, while the SimAM attention mechanism can infer three-dimensional attention weights without increasing the number of model parameters, improving computational speed and efficiency. Secondly, the multi-head attention module in the Transformer divides the raw data into different spaces, calculates separately, and then merges the outputs to improve attention representation capabilities. Essentially, this is achieved by adding parallel attention layers and using matrix multiplication for convenient parallel computation. The SimAM attention mechanism, based on neuroscience theory, proposes an optimized energy function and derives a fast analytical solution, essentially solving for the attention weights corresponding to each pixel in the feature map. Multi-head attention in Transformer uses parallel computation to extract feature information as close to the optimal solution as possible, while SimAM calculates the optimal value through a single analytical solution. Therefore, this invention replaces the multi-head attention module of Transformer with the SimAM attention mechanism. In short, by utilizing the design concept of alternating moving convolution and attention (MOAT) modules, it integrates the GAM and SimAM attention mechanisms to propose the GSAM attention mechanism. Furthermore, it embeds the GSAM module into the YOLOv5 model, proposing an improved YOLOv5 PCB defect image detection model, YOLOv5-GSAM, as follows. Figure 5 As shown.

[0045] Step 2: Establish an image set for training the improved image model to obtain a well-trained improved image model.

[0046] Furthermore, the image set consists of a training dataset and a test dataset, such as... Figure 6 As shown.

[0047] Specifically, in this embodiment, PCB images output by the PCB appearance defect inspection machine are collected and organized to obtain a raw dataset of PCB defect images. Suitable images are selected from this dataset as an image set, and the open-source image annotation tool LabelImg is used to annotate them. By selecting the corresponding defect area with a rectangle, inputting the defect type, and saving, a corresponding XML format annotation file is generated in a specified folder. Finally, the annotation files for all PCB defect images in the raw dataset are obtained. The PCB defect sample types in this invention are as follows: Figure 7 As shown.

[0048] The training dataset consists of defect images and background images. The defect images are categorized as: images of different colors, images of residue, images of foreign objects, and other defect images (scratches, holes, residue, etc.). The background images are used to suppress interference from background information.

[0049] Specifically, in this embodiment, the training dataset consists of 7,000 images of different colors, 25,000 images of residue, 13,000 images of foreign objects, and 7,000 images of other defects such as scratches, cavities, and residue, totaling 52,000 defect images. A 10% background image is added, resulting in a total of 57,200 images. Each category of images is divided in an 8:2 ratio to form a training image set (45,760 images) and a validation image set (11,440 images).

[0050] The test dataset, also known as the test image set, consists of 140,000 defective images (NG images) and 180,000 defect-free images (OK images). The distribution of PCB defect types in the NG images is basically consistent with that in the training image set, consisting of four types: discoloration, residue, foreign matter, and other defects.

[0051] The training process is as follows: the training image set is input into the improved image model for training, and the verification image set is input into the improved image model during training for verification testing, so as to obtain the trained improved image model.

[0052] After training, the test image set is input into the trained improved image model to test its performance, i.e., its generalization ability, which needs to be measured by a certain metric. A series of evaluation metrics can be calculated using the confusion matrix to measure the model, such as precision, accuracy, recall, and F-score.

[0053] In this embodiment, the above image set was used for training, validation and testing of YOLOv5-GSAM. The detection comparison results of YOLOv5-GSAM and the original YOLOv5 model when the classification threshold is 0.5 are shown in Table 1.

[0054] Table 1 Results of various indicators for different models

[0055]

[0056] As shown in Table 1, the improved YOLOv5-GSAM model demonstrates improved performance across all metrics, with a sensitivity rate reaching 89.79%. Regarding training convergence speed, the GSAM mechanism, which integrates GAM and SimAM attention mechanisms, effectively inherits the convergence speed of SimAM, enhancing the model's learning ability and making the YOLOv5-GSAM model slightly faster than the YOLOv5 model. In terms of training stability, both models converged smoothly to their final values ​​without significant fluctuations. The curve of the YOLOv5-GSAM model essentially encircles that of the YOLOv5 model, indicating that the improved YOLOv5 model performs better than the original model in PCB defect detection.

[0057] In summary, this specific implementation provides an improved PCB defect image detection method for YOLOv5 by integrating GAM and SimAM attention mechanisms. Addressing the insufficient small-target defect detection capability of the YOLOv5 model, an improvement is proposed: a GSAM structure attention mechanism is implemented by embedding the GSAM module into the YOLOv5 model. This attention mechanism focuses on PCB defect regions, effectively selecting feature information from small defect areas and assigning higher weights to these regions to prevent defect information from being ignored during feature extraction. This improves the network model's performance in detecting small-target PCB defects. This method can detect different types of PCB defect images. Compared to the original YOLOv5 model, it shows improvements in sensitivity, specificity, and other indicators, demonstrating superior performance in PCB defect detection tasks. It addresses many problems associated with traditional manual and mechanical inspection, such as misjudgment, missed detections, and damage to product surface structures, thereby improving detection accuracy and production efficiency, reducing the cost of manual inspection, and better realizing automated and intelligent PCB defect detection.

Claims

1. A method for PCB defect image detection that integrates GAM and SimAM attention mechanisms to improve YOLOv5, characterized in that, It includes the following two steps: S1: Establish an improved image detection model, specifically: based on the original YOLOv5 model, integrate the global attention mechanism GAM and the simple parameterless attention mechanism SimAM into the original YOLOv5 model to obtain an improved YOLOv5 PCB defect image model. The attention mechanism GAM and the attention mechanism SimAM are fused into a GSAM structure attention mechanism, which is then incorporated into the YOLOv5 model to obtain the improved YOLOv5 PCB defect image model. The GSAM structure replaces the SE-Net module in the MBConv block with the GAM module, and replaces the self-attention operation in the Transformer with the attention mechanism SimAM. The GSAM structure attention mechanism is designed based on the inverse residual structure and linear bottleneck structure in the MOAT structure. Before the GAM attention mechanism, a 1×1 convolution is applied to expand the input channel by 4 times. Then, a 3×3 depthwise separable convolution is used to effectively improve the local spatial interaction between pixels. After the attention mechanism focuses on the information of interest, the features are mapped back to the original channel size through a 1×1 convolution. Finally, residual connections are realized. S2: Establish an image dataset for training the improved YOLOv5 PCB defect image model to obtain a trained improved image model.

2. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 1, characterized in that, In step S1, a GAM / SimAM module is embedded between the two Conv modules in the C3_2 module connected by Concat at layers 4 and 6 of the original YOLOv5 backbone network.

3. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 1, characterized in that, In step S2, the image set consists of a training dataset and a test dataset.

4. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 3, characterized in that, The training dataset consists of defect images and background images. The defect images are composed of images of different colors, images of residue, images of foreign objects, and other defect images. The background images are used to suppress interference caused by background information.

5. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 4, characterized in that, The training dataset is divided into a training image set and a validation image set, wherein the training image set and the validation image set are formed by dividing all types of images in the training dataset in the same proportion.

6. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 5, characterized in that, The training process of the improved YOLOv5 PCB defect image model is as follows: the training image set is input into the improved YOLOv5 PCB defect image model for training, and the verification image set is input into the improved YOLOv5 PCB defect image model during training for verification testing, so as to obtain the trained improved image model.

7. The PCB defect image detection method for YOLOv5 that integrates GAM and SimAM attention mechanisms according to claim 3, characterized in that, The test dataset is input into the trained improved image model to test the generalization ability of the trained improved image model.

Images