A printed circuit board surface defect detection method based on YOLO

Abstract

This invention discloses a YOLO-based method for detecting surface defects on printed circuit boards, belonging to the fields of computer vision and intelligent manufacturing technology. The method includes the following steps: deploying an industrial area scan camera at the inspection station of an automated production line; when a PCB enters the inspection station, the camera is triggered by a sensor to acquire and store images; the acquired image data is processed using data cleaning, data augmentation, and target annotation methods; the processed PCB defect dataset undergoes detailed statistical analysis to optimize the model training strategy; based on the data analysis results, optimization is performed using the YOLOv5 target detection framework to establish a surface defect detection model suitable for PCB production scenarios. This invention can effectively improve the automation level and detection accuracy of defect detection, is suitable for modern large-scale batch production of PCBs, and has good engineering application prospects and industrial promotion value.

Description

Technical Field

[0001] This invention discloses a YOLO-based method for detecting surface defects on printed circuit boards, belonging to the field of computer vision and intelligent manufacturing technology. Background Technology

[0002] As a key component in modern electronic devices, PCBs are complex in structure and highly integrated, and their manufacturing quality directly affects the performance stability and operational reliability of electronic products. However, during the large-scale, automated production of PCBs, various types of defects and flaws are easily generated on their surface due to factors such as processing technology, material properties, and environmental factors. If these defects are not detected in a timely and accurate manner, they may lead to product malfunctions or even potential safety hazards. Currently, the manual visual inspection methods commonly used in industrial production have significant shortcomings in terms of consistency, real-time performance, and adaptability, making it difficult to meet the urgent needs of the electronics manufacturing industry for high-efficiency, high-precision, and intelligent inspection. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a YOLO-based method for detecting surface defects on printed circuit boards (PCBs). It aims to solve the problems of high reliance on manual visual inspection, low detection efficiency, and insufficient accuracy to meet the demands of modern large-scale production in existing PCB surface defect detection processes. This invention effectively improves the automation level and detection accuracy of defect detection, is suitable for modern large-scale PCB production scenarios, and has promising engineering application prospects and industrial promotion value.

[0004] The specific technical solution is as follows:

[0005] A YOLO-based method for detecting surface defects on printed circuit boards includes four steps: data acquisition, data processing, data analysis, and model building. Specifically:

[0006] Step 1: In the data acquisition stage, an industrial area scan camera is deployed at the inspection station of the automated production line. When the PCB enters the inspection station, the camera is triggered by the sensor to acquire and store the image.

[0007] Step 1.1: The area scan camera will continuously collect data at different times and on different production lines to obtain PCB surface image data under different production environments, ensuring the diversity and representativeness of the data.

[0008] Step 1.2: Statistically organize the PCB surface images collected in Step 1.1, filter images containing defects and determine the defect categories on their surfaces to facilitate later annotation.

[0009] Step 1.3: Based on the dataset compiled in Step 1.2, some publicly available datasets are introduced to supplement and expand the dataset, thereby enhancing its richness and generalization ability.

[0010] Step 2: In the data processing stage, the image data collected in Step 1 is processed through methods such as data cleaning, data augmentation, and target annotation.

[0011] Step 2.1: Data cleaning mainly involves quality screening of the original PCB image data collected in Step 1, removing defective or blurry images to ensure the clarity and usability of the dataset.

[0012] Step 2.2: Data augmentation mainly involves performing geometric transformations, color transformations, image sharpening, and other operations on the dataset selected in Step 2.1 to further improve the diversity of the dataset and enhance the model's generalization ability in different production line scenarios.

[0013] Step 2.3: Data augmentation mainly involves a combination of manual and semi-automatic annotation of the image data from Step 2.2 to label defects in the images. A total of 8 different types of defects are included to ensure the accuracy and high quality of the training data.

[0014] Step 3: In the data analysis phase, a detailed statistical analysis is performed on the PCB defect dataset processed in Step 2 to optimize the model training strategy.

[0015] Step 3.1: For the dataset processed in Step 2, perform a category distribution analysis of different defects and statistically analyze the distribution of the number of defects in 8 different categories.

[0016] Step 3.2: For the dataset processed in Step 2, perform size analysis on different defects, count the number of defects of different sizes, and analyze the distribution of target sizes for all categories.

[0017] Step 4: In the model building stage, based on the data analysis results of Step 3, optimize the YOLOv5 target detection framework to build a surface defect detection model suitable for PCB production scenarios.

[0018] Step 4.1: Add a CBMA attention mechanism module to the backbone network to extract image features from different channels, and finally fuse the features from different channels to improve the model's detection performance.

[0019] Step 4.2: The backbone network extracts image features using deformable convolution. By introducing offsets, the receptive field of the model is enhanced, improving the model's adaptability to targets of different sizes and deformations.

[0020] Step 4.3: Loss function optimization. A smoothing function is introduced into the loss function to maintain its continuous and smooth characteristics during numerical changes, avoid sharp fluctuations in gradients, and enhance model stability.

[0021] This method is based on the YOLOv5 framework and is improved and optimized for actual PCB production scenarios to improve the detection efficiency and accuracy of PCB surface defects, enhance the stability of the detection model, and ensure the efficiency and adaptability of the model in different scenarios. The method includes four stages: data acquisition, data processing, data analysis, and model building.

[0022] During the data acquisition phase, industrial area scan cameras are deployed at the inspection stations of the PCB automated production line to continuously acquire PCB surface images from different production lines and at different times. The acquired image samples are screened to retain images containing defect targets. At the same time, some publicly available PCB defect image datasets are introduced as supplements to expand the data samples and construct the original dataset for model training and validation.

[0023] In the data processing stage, the original dataset constructed in claim 2 is preprocessed, the image samples are quality-screened, and images with unclear defect areas or low imaging quality are removed; various data augmentation methods are used to expand the samples; and the defect targets in the images are labeled with categories and locations, finally forming a processed dataset for model training.

[0024] In the data analysis phase, statistical analysis is performed on the processed dataset constructed according to claim 3, the category distribution of different types of defects is analyzed, and the sample quantity distribution including eight defect categories is statistically analyzed; at the same time, the size characteristics of different defect targets are analyzed, the number of defect targets in different size ranges is statistically analyzed, and the size distribution characteristics of each category of defect targets are comprehensively analyzed.

[0025] In the model building phase, based on the YOLOv5 object detection framework, the original YOLOv5 network structure was optimized in combination with the detection requirements of the actual PCB production environment to build a defect detection model suitable for PCB automated production lines. The optimization included introducing a CBAM attention mechanism module into the backbone feature extraction network to enhance the expression of key defect features, using deformable convolution to adaptively extract image features, and introducing a smoothing function into the loss function to improve the stability of the model training process.

[0026] A CBAM attention mechanism module is introduced into the backbone feature extraction network. The CBAM attention mechanism includes a channel attention submodule and a spatial attention submodule. By weighting the feature maps in the channel dimension and the spatial dimension, the fusion of multi-dimensional feature information is achieved, thereby highlighting the salient areas of defective targets in the image and suppressing irrelevant background areas, enhancing the model's ability to model and represent key features.

[0027] Deformable convolutional structures are used in the backbone feature extraction network for image feature extraction. By introducing learnable offsets in the standard convolution sampling positions, adaptive sampling of surrounding pixel features is achieved, thereby effectively expanding the receptive field of the network, enhancing feature representation capabilities, and improving the model's adaptability to targets of different scales and deformation defects.

[0028] Introducing a smoothing function into the loss function allows the loss value to maintain continuous and differentiable smoothness during numerical changes, thereby effectively alleviating the problem of drastic gradient fluctuations caused by sudden loss changes during training, improving the stability of gradient updates, improving the convergence behavior of the model, and enhancing the numerical stability and overall robustness of the training process. Attached Figure Description

[0029] Figure 1 This is the overall structure of the model of the present invention;

[0030] Figure 2 This is the CBMA attention mechanism module of the present invention;

[0031] Figure 3 This is a diagram of the deformable convolution structure of the present invention. Detailed Implementation

[0032] As the preferred embodiment of the present invention, the present invention includes a method for detecting surface defects of printed circuit boards based on YOLO.

[0033] Refer to the instruction manual appendix Figure 1 The specific implementation includes the following steps:

[0034] Step S1: In the image data acquisition stage of the PCB automated production line, an industrial area scan camera is deployed at the inspection station of the production line to continuously acquire images of the PCB surface at different production lines and time periods, in order to cover as many production conditions and environmental conditions as possible, thereby obtaining PCB surface image data with high diversity and representativeness. Subsequently, the acquired image samples are screened, retaining valid samples containing defect targets, and introducing some publicly available PCB defect image datasets as supplements to expand and enrich the data samples. In this embodiment, approximately 20,000 PCB surface images containing defect targets are acquired, with an image acquisition frame rate of 30 FPS and a resolution of 1920×1080. The samples are classified according to the type of defects in the images, into eight categories: film anomalies, film corrosion, film peeling, hole anomalies, broken wires, foreign objects on the film, foreign objects in or under the film, and oil stains. A unique category code is assigned to each category of defects to meet the needs of subsequent target detection model training and label management. The introduced public datasets include the DeepPCB dataset, which is fused and expanded with self-collected PCB surface defect image data to finally construct the original dataset, providing reliable data support for subsequent training and performance verification of the target detection model.

[0035] Step S2: Data processing, which involves processing the original dataset formed in step S1, mainly including data cleaning, data augmentation, and target labeling.

[0036] The data cleaning phase primarily employs a combination of manual review and automated quality assessment algorithms to screen image samples in the original dataset. First, based on indicators such as image sharpness, contrast, and image integrity, images with severe blurring, abnormal exposure, or low image quality are removed to ensure the overall quality of the dataset. Second, for duplicate or highly similar images generated during PCB surface image acquisition due to continuous acquisition or similar operating conditions, a combination of manual discrimination and similarity detection is used to remove redundant samples. This reduces data redundancy, improves the effectiveness and representativeness of the dataset, and provides a reliable data foundation for subsequent model training.

[0037] In the data augmentation stage, to improve the model's adaptability and generalization performance to complex production environments, various data augmentation and image transformation strategies were employed to expand the training samples. These included geometric transformations, color space transformations, and image sharpening. Specifically, geometric transformations enhanced the images through operations such as random rotation, translation, scaling, and horizontal or vertical flipping to simulate the posture changes that PCBs may undergo during actual production, thereby improving the model's robustness to changes in target position and scale. Secondly, a color space transformation strategy was introduced to randomly perturb parameters such as brightness, contrast, and saturation of the images to simulate visual differences under different lighting conditions and imaging environments, enhancing the model's adaptability to lighting variations. Furthermore, image sharpening and other enhancement techniques highlighted the edge and texture features of defect areas, further strengthening the model's ability to perceive subtle defect features. The comprehensive application of these multiple data augmentation strategies effectively enriched the distribution of training samples, increased the diversity of the dataset, and provided strong support for the stable training and performance improvement of the subsequent object detection model.

[0038] In the data annotation stage, a combination of manual and semi-automatic annotation methods was used to accurately annotate targets in the images, ensuring the accuracy and consistency of the annotation results. Manual annotation primarily utilized tools such as LabelImg, Labelme, Labelbox, VGG Image Annotator, RectLabel, and CVAT to annotate target regions one by one in the images. The semi-automatic annotation method introduced a deep learning pre-trained model to make preliminary predictions of the images, which were then manually corrected and confirmed, thus significantly improving annotation efficiency while ensuring annotation quality. The annotated target categories included eight types of defects: membrane anomalies, membrane corrosion, membrane detachment, pore anomalies, broken lines, foreign objects on the membrane, foreign objects in or under the membrane, and oil contamination, providing high-quality annotated data for subsequent model training and performance evaluation.

[0039] Step S3: Data Analysis. Based on the dataset processed in Step S2, perform statistical analysis and organize the overall situation and detailed information of defect distribution in the PCB dataset to understand the detailed situation of the dataset and provide reference value for subsequent model building, including defect category statistics and defect size analysis.

[0040] The defect category statistics mainly involve counting the number of different types of defects and their proportions, including eight types of defects: membrane anomalies, membrane corrosion, membrane detachment, pore anomalies, broken wires, foreign matter on the membrane, foreign matter in or under the membrane, and oil contamination. Their proportions are 12.5%, 12.3%, 11.4%, 15.2%, 12.2%, 10.7%, 13.6%, and 12.1%, respectively. As can be seen from the proportions, the overall distribution of these eight types of defects is relatively stable, with pore anomalies having the highest proportion at 15.2% and foreign matter on the membrane having the lowest proportion at 10.7%. This provides a reference for subsequent model training and continuous optimization of the target detection model.

[0041] Defect size analysis aims to systematically statistically analyze and evaluate the scale distribution characteristics of different categories of defect targets. Specifically, based on the relative size or bounding box area of ​​the defect in the image, defect targets are divided into three categories: large-sized defects, medium-sized defects, and small-sized defects. The sample size distribution of each category of defects within different scale intervals is then statistically analyzed. By analyzing the defect scale distribution characteristics, the scale characteristics of defect targets in the dataset can be comprehensively characterized, providing important reference for the subsequent network structure design, anchor box scale setting, and multi-scale feature fusion strategies of target detection models. This also helps improve the model's detection performance and stability for multi-scale defect targets.

[0042] Step S4: Build the model and optimize it based on the YOLOv5 object detection framework, as shown in the attached figure. Figure 1 As shown, a CBAM attention mechanism module is introduced into the backbone feature extraction network, deformable convolution is used to adaptively extract image features, and a smoothing function is introduced into the loss function.

[0043] Add a CBAM attention mechanism module to the backbone network, the structure of which is shown in the attached figure. Figure 2 As shown, the module consists of two sub-modules: the Channel Attention Module (CAM) and the Spatial Attention Module (SAM). The input feature map is first processed by CAM, undergoing max pooling and average pooling. Then, the two are added together and passed through the sigmoid activation function to obtain the one-dimensional channel weights. , is used to weight each channel of the feature map. CAM weights different features along the channel dimension to highlight channel information with higher semantic contribution. Its processing method is shown in Equation (1):

[0044] (1)

[0045] in and These represent the original feature map and the feature map after processing by the attention module, respectively. and These are global average pooling and global max pooling, used to compress spatial dimensions. The Sigmoid activation function maps the output to the (0,1) interval. It is a multilayer perceptron used to learn the nonlinear relationships between channels.

[0046] Subsequently, the generated feature map is input into SAM, where attention is applied in the spatial dimension. This module processes the input feature map through MaxPool and AvgPool to form a complementary relationship, and then passes it through the Sigmoid activation function to obtain spatial weights. CAM can enhance the response in significant regions, and its processing method is shown in equation (2):

[0047] (2)

[0048] in, Representative at The feature map with added spatial attention is processed by the CBAM attention mechanism module, and the output feature map simultaneously integrates attention information from both the channel and spatial dimensions.

[0049] Deformable convolution is used to extract image features, as shown in the attached figure. Figure 3 As shown, with the sampling point as the center, after adding an offset, feature extraction is performed on the surrounding pixels. The offset and offset position are randomly generated to ensure that the deformable convolution method can extract information from the image to the greatest extent. Its formula is shown in equation (3):

[0050] (3)

[0051] in, To output the current position on the feature map, i.e., the sampling center point. For predefined offset positions in standard convolution kernels A learnable offset is used to adjust the sampling position. For the convolution kernel at position Weight at each location.

[0052] By introducing learnable offsets into the standard convolution sampling positions, adaptive sampling of surrounding pixel features is achieved, thereby effectively expanding the receptive field of the network, enhancing feature representation capabilities, and improving the model's adaptability to targets of different scales and deformation defects.

[0053] Introducing a smoothing function into the loss function ensures that the loss function maintains continuous and smooth numerical changes, thereby alleviating the gradient abruptness problem caused by the large difference between the predicted and true bounding boxes during training. This improvement helps stabilize gradient updates during backpropagation, improves the numerical stability and convergence efficiency of model training, and enhances the regression accuracy of the model for targets with different scales and positional deviations in complex scenes, as shown in Equation (4):

[0054] (4)

[0055] in, This represents the result after smoothing GIoU. Represents a positive scaling factor. This represents the generalized intersection-union ratio (CUI) between the predicted bounding box and the ground truth bounding box.

Claims

1. A method for detecting surface defects on printed circuit boards based on YOLO, characterized in that, Includes the following steps: Step 1: Data Acquisition; Deploy an industrial area scan camera at the inspection station of the automated production line. When the PCB enters the inspection station, the camera is triggered by the sensor to acquire and store the image. Step 2: Data Processing; The image data acquired in Step 1 is processed using data cleaning, data augmentation, and target annotation methods. Step 3: Data Analysis; A detailed statistical analysis is performed on the PCB defect dataset processed in Step 2 to optimize the model training strategy; Step 4: Model building; Based on the data analysis results of Step 3, optimize the YOLOv5 target detection framework to build a surface defect detection model suitable for PCB production scenarios.

2. The method for detecting surface defects of printed circuit boards based on YOLO according to claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: The area scan camera will continuously collect data at different times and on different production lines to obtain PCB surface image data under different production environments, ensuring the diversity and representativeness of the data; Step 1.2: Statistically organize the PCB surface images collected in Step 1.1, filter images containing defects and determine the defect categories on their surfaces to facilitate subsequent annotation; Step 1.3: Based on the dataset compiled in Step 1.2, some publicly available datasets are introduced to supplement and expand the dataset, thereby enhancing its richness and generalization ability.

3. The method for detecting surface defects on printed circuit boards based on YOLO according to claim 1, characterized in that, Step 2 includes the following sub-steps: Step 2.1: Data cleaning mainly involves quality screening of the raw PCB image data collected in Step 1, removing defective or blurry images to ensure the clarity and usability of the dataset; Step 2.2: Data augmentation mainly involves performing geometric transformations, color transformations, and image sharpening on the dataset selected in Step 2.1 to increase the diversity of the dataset and enhance the model's generalization ability in different production line scenarios. Step 2.3: Data augmentation mainly involves a combination of manual and semi-automatic annotation of the image data from Step 2.2 to label defects in the images. A total of 8 different types of defects are included to ensure the accuracy and high quality of the training data.

4. The method for detecting surface defects of printed circuit boards based on YOLO according to claim 1, characterized in that, Step 3 includes the following sub-steps: Step 3.1: For the dataset processed in Step 2, perform a category distribution analysis of different defects and statistically analyze the distribution of the number of defects in 8 different categories; Step 3.2: For the dataset processed in Step 2, perform size analysis on different defects, count the number of defects of different sizes, and analyze the distribution of target sizes for all categories; Step 4: In the model building stage, based on the data analysis results of Step 3, optimize the YOLOv5 target detection framework to build a surface defect detection model suitable for PCB production scenarios.

5. The method for detecting surface defects of printed circuit boards based on YOLO according to claim 1, characterized in that, Step 4 includes the following sub-steps: Step 4.1: Add a CBMA attention mechanism module to the backbone network to extract image features from different channels, and finally fuse the features from different channels to improve the model's detection performance; Step 4.2: The backbone network extracts image features using deformable convolution. By introducing offsets, the receptive field of the model is enhanced, and the model's adaptability to targets of different sizes and deformations is improved. Step 4.3: Loss function optimization. A smoothing function is introduced into the loss function to maintain its continuous and smooth characteristics during numerical changes, avoid sharp fluctuations in gradients, and enhance model stability.

Images