PCBA component defect detection model based on MaShuff
By combining the backbone network and WMIOU loss function of Mask R-CNN and ShuffleNetV2, the low detection rate and false alarm rate problems of traditional PCBA component defect detection methods are solved, and efficient and accurate defect detection is achieved, adapting to different production environments and reducing Manual review to improve production efficiency.
Patent Information
- Application Number
- CN202410365534.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-28
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional PCBA component defect detection methods have low detection rate and high false alarm rate, and require manual re-inspection to increase cost and time, and frequent replacement of detection templates affects production efficiency.
The MSS object detection model based on deep learning is adopted, combined with the backbone network of Mask R-CNN and ShuffleNetV2 for deep fusion to form a MaShuff model, and a WMIOU loss function is proposed to improve detection accuracy.
It improves the accuracy and efficiency of PCBA component defect detection, reduces the error detection rate, adapts to the differences in different production batches and PCBA boards, reduces the burden of manual review, and improves production efficiency.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image recognition, and in particular to a target detection model for PCBA (Printed Circuit Board Assembly) component defect detection. Background Art
[0002] As the core engineering process of electronic manufacturing, PCBA is responsible for cleverly installing various electronic components on printed circuit boards, providing solid support for diverse electronic devices. With the continuous innovation of technology and the improvement of automation, the field of PCBA manufacturing is undergoing rapid evolution to meet the rapidly growing needs of the electronics market.
[0003] At present, the traditional PCBA component defect detection method mainly adopts the template matching method of AOI (Automated Optical Inspection). This method relies on a set of templates set in advance to match the actual image on the PCBA. However, there are a series of problems with the traditional AOI system: First, its detection rate is low, up to about 50%, which means that it is easy to miss or falsely report, which may cause quality problems to go undetected or normal parts to be mistakenly reported as defects. Secondly, due to the high error rate, each AOI device needs to be equipped with experienced employees for manual re-inspection to ensure accuracy, which further increases labor costs and production time. Finally, as the production line switches to different types of PCBA boards, the corresponding inspection templates must be replaced and re-matched. This process is not only time-consuming and cumbersome, but also affects production efficiency.
[0004] To solve these problems, a new generation of PCBA component defect detection technology has gradually emerged. Among them, detection methods based on deep learning and artificial intelligence have gradually become mainstream. These advanced technologies can automatically identify potential defects by learning a large amount of data, greatly improving the accuracy and efficiency of detection. Compared with traditional AOI systems, they are no longer limited to fixed detection templates, thus avoiding the trouble of frequent replacement and re-matching. This not only saves time, but also reduces production costs, making PCBA manufacturing more intelligent and efficient. The present invention constructs a target detection model called MSS (A lightweight detection model based on Mask ShuffleNet). The model ensures the detection accuracy of PCBA component defects while achieving a small amount of parameters and calculations. The backbone network of the model adopts a deep fusion strategy, combining the convolutional layers and pooling layers of two different backbone networks Mask R-CNN and ShuffleNetV2 to construct a new backbone network model, called MaShuff (Mask ShuffleNet).
[0005] Mask R-CNN and ShuffleNetV2 have obvious advantages and disadvantages in PCBA component defect detection tasks: Mask R-CNN has excellent instance segmentation accuracy in PCBA component defect detection tasks, can locate components at the pixel level, and achieve accurate detection and location of defects. Its multi-task processing capability enables simultaneous target detection and target binary mask generation in one model, providing more comprehensive detection results. The model is highly adaptable and can adapt to the shape and size changes of different PCBA components through training. The use of advanced backbone networks (such as ResNet) helps to extract richer features and further improve detection performance. In the industrial field, especially in PCBA defect detection, Mask R-CNN is widely used.
[0006] The Mask R-CNN model also faces some challenges. Its computational complexity is high, and relatively large models require more computing resources, which may be limited in resource-constrained devices or real-time detection scenarios. The long training time and complexity make it too heavy for some simple tasks, and it may face overfitting problems when there is less training data.
[0007] ShuffleNetV2 has significant advantages in PCBA component defect detection tasks, first of all, its lightweight structure, which makes it an ideal choice for real-time detection on computing resource-constrained devices. By introducing technologies such as channel rearrangement, ShuffleNetV2 achieves efficient computing, effectively reduces the computational complexity of the model, and improves the operating efficiency on resource-constrained devices. In addition, its relatively small model size makes it suitable for embedded systems and mobile devices, providing convenience for PCBA defect detection.
[0008] ShuffleNetV2 also has some challenges in PCBA component defect detection tasks. Its lightweight design makes the accuracy relatively low in some complex tasks, which may be slightly insufficient. Especially when dealing with complex scenarios that require higher accuracy and complex feature representation, ShuffleNetV2 may not perform as well as some large deep learning models. Therefore, when choosing a model suitable for PCBA component defect detection, it is necessary to weigh the advantages of its lightweight design with the relatively low accuracy in complex scenarios. Summary of the invention
[0009] The present invention proposes an innovative target detection model, which is specially designed for PCBA component defect detection. The model can not only accurately identify various defect problems existing in PCBA boards, but also accurately locate and identify various defects in a small PCBA board area through deep learning and advanced image processing technology, providing manufacturers with efficient and reliable quality control methods. Not only that, the model also greatly improves the accuracy of PCBA component defect detection through the application of intelligent algorithms while taking into account the actual production environment, further promoting the automation and intelligence of the industrial manufacturing process.
[0010] It is worth mentioning that the present invention not only focuses on improving the accuracy of PCBA component defect detection, but also makes significant progress in reducing the model's false positive rate. Through carefully designed algorithms and training methods, the model can better adapt to the differences between different production batches and PCBA boards, thereby avoiding unnecessary misjudgments, reducing the burden of manual review, and improving production efficiency.
[0011] In view of the difficulties encountered in the PCBA component defect detection task, the present invention makes two precise improvements to the traditional target detection model to achieve more excellent performance: First of all, the present invention has created a target detection model called MSS, which aims to achieve fewer parameters and less computation while ensuring the accuracy of PCBA component defect detection. The MaShuff backbone network was proposed after an in-depth analysis of the characteristics of MaskR-CNN and ShuffleNetV2, as well as their advantages in different scenarios. A deep fusion strategy is adopted to fuse the convolutional layer and pooling layer of the two obviously different backbone networks, Mask R-CNN and ShuffleNetV2, to form a new backbone network model - MaShuff. This sophisticated fusion enables the new model to give full play to the advantages of both and make up for their respective shortcomings, thereby showing more comprehensive and excellent performance in PCBA component defect detection tasks.
[0012] Secondly, in the process of implementing the PCBA component defect detection task, the defects of the components may exist in various forms, including irregular shapes and various sizes. This diversity makes the traditional loss function not flexible enough in capturing and interpreting these specific defect modes, resulting in a decrease in detection performance. Especially when the defect target is very small, it may be difficult for traditional models to accurately locate and identify these defects. To solve the above problems, the present invention proposes a WMIOU (Weighted Fusion Intersection over Union) loss function, which is a cross-combination loss function based on the minimum overlap area proposed by the present invention, which aims to improve the accuracy of bounding box prediction in object detection tasks and solve the problem that the original IOU overlap area is the same but the component directions are different. It measures the similarity between the predicted bounding box and the true bounding box, and uses it as a loss to train the model. BRIEF DESCRIPTION OF THE DRAWINGS In order to more clearly explain the implementation details and technical features of the "PCBA component defect detection method based on MaShuff model" of the present invention, the accompanying drawings involved in the description will be briefly introduced below. Figure 1 The overall process diagram of the PCBA component defect detection method based on the MaShuff model is shown, which intuitively presents the main steps and logical flow of the method in the form of a block diagram. Figure 2 The Mask R-CNN network structure diagram is described in detail, which reveals the key components of the model for feature extraction and instance segmentation and their interrelationships. Figure 3 The network structure diagram of ShuffleNetV2 is further shown, and its lightweight design characteristics are clearly reflected in the figure. Figure 4 Comprehensive Figure 2 and Figure 3 The figure comprehensively demonstrates how the model combines the high precision of Mask R-CNN and the high efficiency of ShuffleNetV2 to achieve efficient and accurate detection of PCBA component defects.
Claims
1. A PCBA component defect detection model based on MaShuff, It is characterized in that include: (1) Collect PCBA board images from the actual production line and perform image processing and annotation; (2) The dataset is randomly divided into training set, validation set and test set in a ratio of 6:
4. (3) Using a deep fusion strategy, we combined the convolutional layers and pooling layers of two different backbone networks, Mask R-CNN and ShuffleNetV2, to construct the MaShuff backbone network model. (4) The WMIOU loss function is introduced to improve the accuracy of bounding box prediction in object detection tasks.
2. According to the PCBA defect detection method based on MaShuff according to claim 1, It is characterized in that In step (1), the labelImg annotation tool is used to annotate the sample images in categories. The annotation format is ".xml". The resulting ".xml" file is converted into a ".tfrecord" file, and the annotation results are corrected to reduce the noise introduced by the annotation errors.
3. According to the PCBA defect detection method based on MaShuff in claim 2, It is characterized in that In step (2), the random partitioning of the data set ensures that the data of the training set, validation set, and test set are evenly distributed, which helps the model to have good generalization ability in various situations. At the same time, the 6:4 ratio partitioning ensures that the model has enough training data, as well as enough validation and test data to evaluate the performance of the model.
4. The PCBA defect detection method based on MaShuff according to claim 1, It is characterized in that In step (3), by comprehensively considering the advantages and disadvantages of Mask R-CNN and ShuffleNetV2, the MaShuff model is constructed, which is a lightweight, efficient, and adaptable target detection model. It improves the current PCBA component defect detection task, which has low accuracy and large computational complexity for small targets, making the model more robust when dealing with complex backgrounds, small target detection, and other problems. It provides an advanced and reliable solution for the electronics manufacturing industry. The MaShuff network structure consists of 5 convolutional layers, 5 pooling layers, 1 fully connected layer, and 5 activation functions.
5. In the first convolutional layer, a 3×3×3×24 convolution kernel with a stride of 1 is used for cross-channel processing, mapping an image with 3 channels to features with 24 channels, and performing dimensionality increase on the input image; In high-dimensional space, the ReLU linear activation function is used to perform 3×3 maximum pooling with a step size of 2 and local pooling (RoIAIign) processing to obtain a 14×14×24 feature map; Through channel splitting, two feature maps of the same size of 14×14×12 are obtained. One of the feature maps is processed by DW (Depthwise Convolution), while the other is not processed. The two feature maps are channel-superimposed and rearranged to obtain a feature map of 14×14×24. Repeat the above steps, process the two feature maps of size 14×14×12 obtained by channel segmentation in the first layer, use a 3×3 convolution kernel with a stride of 2 for DW processing in the second layer, and use 1×1 convolution for dimensionality increase in the third layer; After channel stacking and rearrangement of the obtained feature map, a 7×7×48 feature map is finally obtained, and then a transposed convolution (ConvTranspose2d) calculation is performed on it; The last layer is a fully connected layer, which is used to output the classification results.
6. The PCBA defect detection method based on MaShuff according to claim 1, It is characterized in that In step (4), an innovative WMIOU loss function is introduced, which directly predicts the matching degree between the bounding box and the true bounding box based on the concept of minimum overlapping area.
7. Let, respectively denote the predicted bounding box and the actual bounding box, Calculate the intersection and union (IOU) ratio: (1) in, Expressed as , The part where the two bounding boxes overlap. Expressed as , All parts of both bounding boxes. express , which is the union of the predicted box and the true box. express , which is represented as the intersection of the predicted box and the true box.
8. Set It means including and The minimum box area of the WMIOU loss function can be obtained as follows: (2) Then we can get its loss function have to: (3) That is to simplify: (4)。 9. The PCBA defect detection method based on MaShuff according to claim 6, It is characterized in that The WMIOU loss function not only optimizes the matching degree between the model's predicted bounding box and the true bounding box, but also improves the model's ability to detect small targets by considering the concept of minimum overlapping area. This innovative loss function design allows the model to pay more attention to the recognition and positioning of small targets during the training process, thereby improving the accuracy of small target detection. In practical applications, PCBA defect detection often faces the challenge of small target detection, and the WMIOU loss function reflects the matching degree between the predicted bounding box and the true bounding box through a more detailed measurement method. By better capturing the geometric relationship between the target and the predicted box, it improves the accuracy of target detection and further enhances the overall detection performance.
10. In summary, the invention of the PCBA component defect detection model based on MaShuff adopts a target detection model based on deep learning technology, and is committed to promoting innovative technological progress in the field of electronic manufacturing. Through deep convolutional neural networks for image recognition and defect detection, this new method can automatically learn and identify the characteristics of various PCBA component defects, thereby significantly improving the detection accuracy; by constructing the MaShuff model, it achieves high accuracy in PCBA component defect inspection and detection with fewer parameters and calculations, better meeting the needs of actual production. The application of this technology is not only expected to reduce employment costs and provide opportunities for capital savings, but also accelerate problem identification through automated PCBA board defect detection methods, reduce stagnation and delays in production, and thus comprehensively improve overall production efficiency. This innovation will have a far-reaching impact on the electronics manufacturing industry and drive the industry forward.
