PCB surface defect detection method and system based on deep learning

Abstract

The invention belongs to the field of target detection, and provides a PCB surface defect detection algorithm and system based on deep learning, and the algorithm comprises an image collection module, a defect labeling module, a detection model training module, a defect detection module, and a defect data management module. A YOLOv8 target detection network is improved, and the target detection network is improved for the problems that manual visual inspection is low in efficiency, traditional machine detection is low in precision and high in false detection rate, and a deep learning model is difficult to deploy on an industrial production line. A lightweight convolution model is used for replacing an original convolution structure, so that the parameter calculation amount of the model and the deployment difficulty of the model in an industrial production line are reduced; a GAM attention mechanism is introduced into an original target detection network, and the PCB defect detection capability of the detection network is improved; and the positioning loss is optimized, and the detection precision of the target network is further improved by introducing an Inner-CIoU mode. Finally, through testing, the average precision mAP (at) 0.5 of the method for detecting the defects of the PCB is 99.2%.

Description

Technical Field

[0001] The present invention relates to a PCB surface defect detection method based on deep learning, belonging to the field of target detection. Background Art

[0002] Compared with traditional target detection methods, target detection methods based on deep learning have higher detection accuracy and stronger adaptability, and can be more widely used in industrial scenarios. In the task of detecting surface defects of PCB boards, due to the influence of factors such as complex industrial environment, the detection difficulty increases, the false detection rate and missed detection rate are high, and the detection efficiency is low. Therefore, it is of great significance to accurately identify surface defects of PCB boards.

[0003] With the continuous development of science and technology and the continuous improvement of productivity, my country has made great progress in industrial intelligence and related fields. The market has also put forward higher requirements on the appearance and quality of industrial products, which requires the traditional production model to transform into an intelligent production model. From a macro perspective, the intelligence and automation of the manufacturing industry will be an inevitable trend.

[0004] The PCB board production process is an extremely precise and tedious process, with as many as a dozen processes, including cutting, lamination, drilling, electroplating, etching, silk screen, cleaning, packaging, testing, etc. Any problem or error in any of these processes will lead to defects of varying degrees and types in the final product. In order to detect defects and identify defect types as early as possible and avoid greater waste of resources in subsequent production, timely and accurate defect detection is an extremely important process. Among them, common PCB defect types are divided into six categories: holes, gaps, open circuits, short circuits, burrs, and residual copper. PCB boards with defects are extremely likely to affect the performance of printed circuit boards and may cause the scrapping of electronic components and cause safety hazards. Therefore, PCB board defect detection is an indispensable detection link for electronic manufacturers.

[0005] Initially, PCB board surface defect detection mainly relied on manual visual inspection. This method is not only costly and inefficient, but more importantly, it will result in a large number of false detections and missed detections, which is not applicable in actual industrial environments. Later, some specific product defect detection methods emerged to replace manual inspection, such as electrical inspection, flying probe inspection, automatic optical inspection, etc. However, these detection methods still have problems such as the possibility of causing secondary damage to PCB boards and the inability to classify defects, and cannot meet the requirements of accurate and efficient detection in PCB board surface inspection.

[0006] At present, computer vision detection methods have been applied in many industrial scenarios, among which target detection based on deep learning is very common. The general process is: first, use imaging equipment to shoot the surface image of the PCB board during production; then input the obtained image data set into the deep learning network model; finally, through multiple iterative reasoning, the best target detection network model is obtained, and the target detection network model can automatically detect the defect category and location in the application. Compared with the traditional method, the related method based on deep learning is to autonomously learn the defect samples based on the Convolutional Neural Network (CNN), thereby overcoming the problem that a traditional algorithm can only detect one defect. The network model can also be optimized by improving the network structure, adjusting parameters, etc., so that it can detect defects with small areas and fuzzy features. Therefore, the PCB surface defect detection method based on deep learning has great application prospects. Summary of the invention

[0007] The purpose of the present invention is to improve the accuracy and speed of PCB surface defect detection, and to propose a PCB surface defect detection algorithm and system based on deep learning.

[0008] The above invention object is achieved through the following technical solutions:

[0009] A PCB board surface defect detection algorithm based on deep learning, the method is implemented by the following steps:

[0010] Step S1, defect image acquisition: using an industrial camera to acquire defect images generated during the production process of a bare PCB on a built PCB inspection platform, and obtaining a defect dataset of the bare PCB original image;

[0011] Step S2, image preprocessing: expanding the data set by performing basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity on the original data set in step S1;

[0012] Step S3, annotating and dividing the data set: using the image annotation tool labelimg to annotate the defect locations and categories of the data set obtained after the image preprocessing in step S2, and dividing the data set and the annotation file into an image training set and an image test set at a ratio of 9:1;

[0013] Step S4, improvement, optimization and training of the target detection network model: using the improved YOLOv8 target detection network model as the network model for PCB board surface defect detection, inputting the battery image data set prepared in step S3 into the network model for training and testing, and obtaining a trained PCB board surface defect detection network model;

[0014] Step S5, online detection of PCB board surface defects: the PCB board surface defect detection network model with the highest average accuracy obtained after training in step S4 is placed in an online defect detection device. If any defect is detected on the surface of the PCB board input into the device, the defect detection device marks the detected defect in the annotation style in step S3, and saves the defect data. The defective products can be found by comparing the product number with the defect detection data.

[0015] In the above scheme, the image acquisition in step S1 refers to the acquisition of defect images generated during the production of PCB bare boards using an industrial camera and a light source on a built PCB inspection platform. The acquired defect images are divided into six types of defects: holes, rat bites, open circuits, short circuits, burrs, and excess copper.

[0016] In the above scheme, expanding the original data set in step S2 refers to expanding the original data set by randomly rotating, randomly scaling, randomly adjusting brightness or contrast, etc. the data set images using the pytorch framework.

[0017] In the above scheme, the labeling in step S3 refers to using the labelimg image labeling tool to select different defects on the surface of the PCB board with a rectangular box and label them with the defect name. The defect name is the same as the label name, which are mouse_bite, spur, missing_hole, short, open_circuit, and spurious_copper.

[0018] In the above scheme, the PCB defect detection network model in step S4 is an improved YOLOv8 target detection network model, and the improvement of the YOLOv8 target detection network includes:

[0019] Introducing lightweight convolution into the network model structure to reduce the number of network parameters;

[0020] Introduce the GAM attention mechanism in the backbone of the network model;

[0021] Improve the loss function of the original network model and use Inner-CIoU as the positioning loss of the improved model;

[0022] In the above scheme, the lightweight convolution is introduced into the network model structure to reduce the number of network parameters as follows:

[0023] The C2f module in YOLOv8 uses a multi-branch design with multiple Bottleneck modules in series. Although it improves the model's ability to extract features of different scales, it will generate too much channel redundant information. By replacing the Bottleneck module with the PConv Block module, the lightweight improvement of the network structure can be achieved; the PConv Block module consists of two 3×3 PConv modules in series. The first PConv compresses the feature map, and the second PConv performs high-dimensional regression on the feature map and removes residual connections to reduce excessive redundant information, thereby reducing the model's parameters and improving the model's detection speed. PConv parameter quantity P pc , calculation amount F pc 、Memory access amount B pc The specific calculation formula is:

[0024]

[0025]

[0026]

[0027] Among them, k represents the size of the convolution kernel, w and h represent the width and height of the feature layer respectively, and c p Indicates the number of channels for convolution operation.

[0028] In the above scheme, the GAM attention mechanism is introduced into the bankbone part of the network structure as follows:

[0029] The GAM attention module is introduced after the SPPF module in the YOLOv8 network structure. The GAM module first passes the input features through the channel attention module, which maintains the three-dimensional channel information of the input features. The channel attention module performs a permutation operation to convert the input feature map from the C×W×H format to the W×H×C format, and then generates a new feature map through inverse permutation and activation function. Next, the output feature map is input into the spatial attention module. This module consists of two 7×7 convolutional layers and activation functions to process the feature map. After processing, the scaled output features and the corresponding weight coefficients are obtained.

[0030]

[0031]

[0032] Among them, F1 and F2 are input feature maps, F3 is the output feature map, and M c 、M s It is channel attention and spatial attention.

[0033] In the above scheme, in the loss function of the improved YOLOv8 network model, the positioning loss using Inner-CIoU as the loss function of the improved model is specifically:

[0034] In the YOLOv8 model loss function, the CIOU loss function is used for bounding box regression. The expression of the CIOU loss function is as follows:

[0035]

[0036]

[0037]

[0038]

[0039] In the above formula, IOU is the intersection area of ​​the predicted box and the real box divided by the union area of ​​the two, b is the center point of the predicted box, and b gt is the center point of the real frame, and p is the center point of b and b gt The Euclidean distance between them, c is the diagonal length of the smallest bounding rectangle that contains both the predicted box and the true box; α is the weight coefficient, and v is the similarity of the aspect ratio. gt is the width of the real frame; h gt is the height of the real box; w is the width of the predicted box detected by the model; h is the height of the predicted box detected by the model.

[0040] The expression of the introduced Inner-CIoU is as follows:

[0041] L Inner-CIoU =L CIoU +IOU-IOU inner

[0042]

[0043]

[0044] union=(w gt ×h gt )×(ratio) 2 +w×h×(ratio) 2 -inter

[0045]

[0046]

[0047]

[0048] In the above formula: ratio is the proportional factor; They are the center points of the ground truth box and the internal GT box respectively; x c ,y c are the center points of the anchor box and the internal anchor box respectively; w, h, w gt 、h gt are the width and height of the anchor box and GT box respectively; They are the left, upper, right and lower boundaries of the GT box respectively; b l , b t , b r , b b They are the left, upper, right and lower boundaries of the anchor box respectively; inter is the intersection area of ​​the GT box and the anchor box; union is the joint area of ​​the GT box and the anchor box.

[0049] A system for implementing the deep learning-based PCB surface defect detection algorithm is characterized by comprising the following modules: an image acquisition module, a defect annotation module, a detection model training module, a defect detection module, and a defect data management module.

[0050] The image acquisition module is used to collect PCB surface defect images and obtain PCB original data sets.

[0051] The image preprocessing module is used to expand the data set by performing basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity on the collected original defective images of the PCB board.

[0052] The defect annotation module refers to using the labelimg image annotation tool to select different defects on the surface of the PCB board and annotate the defects according to their categories. The system will automatically divide the data set and annotation file into image training set and image test set in a 9:1 ratio.

[0053] The training module of the detection model is used to construct a PCB board defect detection network model. The PCB board image data set is input into the constructed target detection network model for training and testing. After 100 iterations, the best PCB board defect detection network model is obtained.

[0054] The defect detection module places the best PCB board defect detection network model in an online defect detection device, and performs online detection on the incoming PCB board. If defects are found on the surface of the produced PCB board, the defect detection device will mark the detected defects in the form of a rectangular frame and save the detected defect data to the defect data management module.

[0055] The defect data management module is used to store the location information, image information, category information and number information of the defects in the defective PCB board. The user can find out all defects and categories in the defective products by comparing the actual number of the PCB board with the defect data.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] According to one embodiment of the present invention, in the actual industrial defect detection process of PCB boards, due to limited computing resources, the detection and processing speed of the equipment for images often cannot meet the needs of industrial production. The C2f module used multiple times in the improved model is replaced by a small convolution structure PConv to reduce the deployment difficulties and slow detection speed of the model in the actual process.

[0058] According to one method of the present invention, the current PCB board surface defect detection has problems such as large error in target defect feature extraction and defect omission. The feature extraction network performs multiple downsampling operations in the trunk part, and the feature map may lose most of the location information and semantic information of small target defects after continuous reduction, resulting in unsatisfactory detection results or even misjudgment. An improvement plan is proposed to solve this problem. The GAM attention module is introduced in the last layer of the trunk part to construct a residual network. In this way, the information loss of the feature map after passing through the network layer can be reduced, which is beneficial to the information retention of small target defect information in the detection head. That is, the detection performance is stably improved and the operation that causes information reduction can be eliminated.

[0059] According to one method of the present invention, in view of the current problem of small target positioning difficulty and low average accuracy in PCB board surface defect detection, starting from the positioning loss function, an auxiliary boundary is introduced to calculate the IoU loss, the scale of the auxiliary boundary box is controlled by the scale factor, and the regression process of the boundary box is accelerated by distinguishing different regression samples and using scale-independent auxiliary boundary boxes to calculate the loss, thereby improving the accuracy of the positioning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is an overall design block diagram of an implementation mode of the present invention.

[0061] Figure 2 It is a schematic diagram of the PConv convolution structure of one embodiment of the present invention.

[0062] Figure 3 It is a schematic diagram of the GAM module structure according to one embodiment of the present invention.

[0063] Figure 4It is an overall schematic diagram of an improved PCB defect detection network structure according to an embodiment of the present invention.

[0064] Figure 5 This is a loss reduction graph of this example according to one embodiment of the present invention.

[0065] Figure 6 PR curve diagram of this example of one embodiment of the present invention.

[0066] Figure 7 It is a detection effect diagram of this example of one implementation mode of the present invention. DETAILED DESCRIPTION

[0067] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0068] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0069] Figure 1 A preferred implementation of the PCB board surface defect detection method based on deep learning is shown, and the PCB board surface defect detection method based on deep learning includes the following steps:

[0070] Step S1, defect image acquisition: on the built PCB inspection platform, an industrial camera is used to collect defect images generated during the PCB bare board manufacturing process to obtain a PCB bare board original image defect data set.

[0071] Step S2, image preprocessing: expanding the data set by performing basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity on the original data set in step S1;

[0072] Step S3, annotate and divide the data set: use labelimg to annotate the defect locations and categories of the data set obtained after the image preprocessing in step S2, and divide the data set and the annotated file into an image training set and an image test set at a ratio of 9:1;

[0073] Step S4, improvement, optimization and training of the target detection network model: using the improved YOLOv8 target detection network model as the network model for PCB board surface defect detection, inputting the battery image data set prepared in step S3 into the network model for training and testing, and obtaining a trained PCB board surface defect detection network model;

[0074] Step S5, online detection of PCB board surface defects: the PCB board surface defect detection network model with the highest average accuracy obtained after training in step S4 is placed in an online defect detection device. If any defect is detected on the surface of the PCB board input into the device, the defect detection device marks the detected defect in the annotation style in step S3, and saves the defect data. The defective products can be found by comparing the product number with the defect detection data.

[0075] Preferably, the step S3 of adding annotations and dividing the data set refers to using the labelImg image annotation tool to select different defects on the surface of the PCB board and label them with defect names, and the defect names are the same as the label names, namely, mouse_bite, spur, missing_hole, short, open_circuit, and spurious_copper.

[0076] Preferably, the improvement of the YOLOv8 target detection network in step S4 includes:

[0077] Introduce lightweight convolution to reduce the number of network parameters.

[0078] According to the actual inspection situation of PCB, the GAM attention mechanism is added to make the network model pay more attention to small target defects.

[0079] The loss function of the original network model is improved according to the detected position loss data, and Inner-CIoU is used as the positioning loss of the improved model;

[0080] In the above scheme, the lightweight convolution is introduced into the network structure to reduce the number of network parameters:

[0081] The C2f module in YOLOv8 uses a multi-branch design with multiple Bottleneck modules in series. Although it improves the model's ability to extract features of different scales, it will generate too much channel redundant information. By replacing the Bottleneck module with the PConv Block module, the lightweight improvement of the network structure can be achieved; the PConv Block module consists of two 3×3 PConv modules in series. The first PConv compresses the feature map, and the second PConv performs high-dimensional regression on the feature map and removes residual connections to reduce excessive redundant information, thereby reducing the model's parameters and improving the model's detection speed. PConv parameter quantity P pc , calculation amount F pc 、Memory access amount B pc The specific calculation formula is:

[0082]

[0083]

[0084]

[0085] Among them, k represents the size of the convolution kernel, w and h represent the width and height of the feature layer respectively, and c p Indicates the number of channels for convolution operation.

[0086] The process of adding the GAM attention mechanism is specifically as follows: the GAM attention module is introduced after the SPPF module in the YOLOv8 network structure. The GAM module first passes the input features through the channel attention module, which maintains the three-dimensional channel information of the input features. The channel attention module performs an arrangement operation to convert the input feature map from the C×W×H format to the W×H×C format, and then generates a new feature map through the inverse arrangement and activation function. Next, the output feature map is input into the spatial attention module. This module consists of two 7x7 convolutional layers and an activation function for processing feature maps. After processing, the scaled output features and the corresponding weight coefficients are obtained.

[0087]

[0088]

[0089] Among them, F1 and F2 are input feature maps, F3 is the output feature map, and M c 、M s It is channel attention and spatial attention.

[0090] The loss function of the improved original network model, using Inner-CIoU as the positioning loss of the improved model is specifically:

[0091] In the YOLOv8 model loss function, the CIOU loss function is used for bounding box regression. The expression of the CIOU loss function is:

[0092]

[0093]

[0094]

[0095]

[0096] In the above formula, IOU is the intersection area of ​​the predicted box and the real box divided by the union area of ​​the two, b is the center point of the predicted box, and b gt is the center point of the real frame, and р is the center point of b and b gt The Euclidean distance between them, c is the diagonal length of the smallest bounding rectangle that contains both the predicted box and the true box; α is the weight coefficient, and v is the similarity of the aspect ratio. gt is the width of the real frame; h gt is the height of the real box; w is the width of the predicted box detected by the model; h is the height of the predicted box detected by the model.

[0097] The expression of Inner-CIoU is as follows:

[0098] L Inner-CIoU =L CIoU +IOU-IOU inner

[0099]

[0100]

[0101] union=(w gt ×h gt )×(ratio) 2 +w×h×(ratio) 2 -inter

[0102]

[0103]

[0104]

[0105] In the above formula: ratio is the proportional factor; They are the center points of the ground truth box and the internal GT box respectively; x c ,y care the center points of the anchor box and the internal anchor box respectively; w, h, w gt 、h gt are the width and height of the anchor box and GT box respectively; They are the left, upper, right and lower boundaries of the GT box respectively; b l , b t , b r , b b They are the left, upper, right and lower boundaries of the anchor box respectively; inter is the intersection area of ​​the GT box and the anchor box; union is the joint area of ​​the GT box and the anchor box.

[0106] A system for implementing the deep learning-based battery defect detection method includes an image acquisition module, a defect annotation module, a detection model training module, a defect detection module, and a defect data management module.

[0107] The image acquisition module is used to collect PCB images and obtain PCB original data sets.

[0108] The image preprocessing module is used to perform basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity on the collected original PCB board image to expand the data set.

[0109] The defect annotation module refers to using the labelImg image annotation tool to select different defects on the surface of the PCB board and annotate the defects according to the defect category. The system will automatically divide the data set and annotation file into an image training set and an image test set at a ratio of 9:1;

[0110] The training module of the detection model is used to build a PCB board defect detection network model, and the PCB board image data set is input into the built target detection network model for training and testing. After 100 iterations, the best PCB board defect detection network model is obtained;

[0111] The defect detection module places the best PCB board defect detection network model in an online defect detection device, and performs online detection on the incoming PCB board. If defects are found on the surface of the produced PCB board, the defect detection device marks the detected defects in the form of a rectangular frame and saves the detected defect data to the defect data management module;

[0112] The defect data management module is used to store the location information, image information, category information and number information of the defect in the defective PCB board. The user can find out all the defects and categories of the defective products by comparing the actual number of the PCB board with the defect data; Specific embodiment:

[0114] According to this embodiment, preferably, the model training parameters are set as follows: the hyperparameter setting batch size is 32, the training cycle is 100, the initial learning rate is 0.01, and the size of the input image is 640×640 pixels.

[0115] The overall flow chart of the deep learning battery defect detection method of the present invention is as follows: Figure 1 As shown, it includes the following steps:

[0116] Step S101: Image acquisition: A complete image acquisition device is constructed using a high-resolution camera, a ring-shaped three-color light source, a camera controller, and a light source controller. Preferably, the resolution and focal length of the camera are adjusted to ensure that the acquired image is clear. After acquisition, the image is optimized and processed, including correcting geometric distortion, adjusting contrast, and cutting redundant parts. Finally, the acquired image data is saved as the original data set of the PCB board.

[0117] Step S102: Image preprocessing: The original data set collected in step S101 is expanded by performing basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity.

[0118] Step S103: Image data set preparation: Use labelImg annotation tool to annotate the defect location and category of the data set obtained after image preprocessing in step S102. In this embodiment, the image data set annotation method is to select the corresponding defects in a rectangle and specify the label name according to the defect category, which are mouse_bite, spur, missing_hole, short, open_circuit, spurious_copper. After the annotation is completed, the defect information is stored in a .txt suffix file. Finally, the annotation file set and the data set are divided into an image training set and an image test set according to the quantity ratio of 9:1;

[0119] Step S104: Construction and training of network model: First, the problems of low small target detection accuracy and large network parameters in the original YOLOv8 network model that are not conducive to deployment on resource-constrained devices are improved. Through lightweight improvements, channel attention and spatial attention mechanisms are introduced to enable it to be deployed on mobile devices and improve detection accuracy, so that it can adapt to the complex production conditions in industrial production. The PCB board defect data set annotated in step S103 is input into the constructed target detection network model for training and testing. After 100 iterations, the best PCB board defect detection network model is obtained;

[0120] The lightweight improvements are specifically as follows:

[0121] PConv is a lightweight and fast convolutional structure. Some convolutional structures such as Figure 2 As shown: it only processes part of the input channel c p Perform convolution operation on the remaining channels cc p (c is the channel) maintains the same mapping. The parameter P of PConv pc , calculation amount F pc 、Memory access amount B pc The specific calculation formula is:

[0122]

[0123]

[0124]

[0125] The specific mechanism of adding attention is as follows:

[0126] The GAM attention module is introduced after the SPPF module in the YOLOv8 network structure, which contains channel attention and spatial attention. Figure 3 As shown in the figure. GAM can not only reduce the diffusion of information, but also expand global interactivity, thereby optimizing network performance. The channel attention submodule mainly stores dimensional information. Its storage principle relies on 3D sequences, and then uses a two-layer encoder-decoder structure (MLP) to amplify cross-dimensional channel-space dependencies. The spatial attention submodule contains two convolutional layers, which can focus more spatial information.

[0127] The GAM module first passes the input features through the channel attention module, which maintains the three-dimensional channel information of the input features. The channel attention module performs a permutation operation to convert the input feature map from the C×W×H format to the W×H×C format, and then generates a new feature map through inverse permutation and activation function. Next, the output feature map is input into the spatial attention module. This module consists of two 7×7 convolutional layers and activation functions to process the feature map. After processing, the scaled output features and the corresponding weight coefficients are obtained.

[0128]

[0129]

[0130] Among them, F1 and F2 are input feature maps, F3 is the output feature map, and M c 、M s It is channel attention and spatial attention.

[0131] In this embodiment, the model training in step S104 uses a 5-fold cross validation method, dividing the data set prepared in step S103 into 5 parts, taking one of them as the test set each time without repetition, and using the other four parts as the training set to train the model. The initial learning rate is 1×10 -3 , the momentum coefficient is 0.949, the number of iterations is 100, and the model training is completed when 1oss tends to be stable. The loss decline graph of this embodiment is as follows Figure 5 As shown, Figure 5 (a) Figure 5 (b) Figure 5 (c) represents the bounding box loss, confidence loss and category loss of the training set respectively; Figure 5 (d) and Figure 5 (e) Two graphs represent the accuracy and recall of the model: Figure 5 (f) Figure 5 (g) and Figure 5 (h) The three figures show the bounding box loss, confidence loss and category loss of the validation set respectively. Figure 5 (i) and Figure 5 (j) The two figures represent the mAP value when the confidence threshold is set to 0.5 and the average precision of all categories with confidence thresholds ranging from 0.5 to 0.95. The PR curve when the IoU threshold is set to 0.5, as shown in Figure 6 As shown, Figure 6 In the example, the recall rate is set as the horizontal axis and the precision rate is set as the vertical axis. The better the performance of the model, the larger the area under the PR curve. Figure 6 It can be seen that the model of the present invention has excellent performance.

[0132] Step S105: Online detection of PCB board surface defects: The trained model is deployed in an industrial computer, and the image acquisition device in step S101 takes pictures of the PCB bare boards produced in the assembly line, and the pictures are transferred to the defect detection system in the industrial computer; the model automatically detects the images, and some detection effect diagrams are shown in FIG. Figure 7 As shown, it can be seen that this method can select the location of the defect and distinguish the category of the defect, thereby achieving the purpose of defect detection.

[0133] Therefore, through the above steps, the average precision mAP@0.5 of PCB defect detection using the improved YOLOv8 target detection model is 99.2%, as shown in Figure 5 (i) As shown; the parameter amount and calculation amount of the original model and the improved model during detection are shown in Table 1. The parameter amount and calculation amount of the improved model are reduced by 23% and 33% respectively.

[0134] Table 1

[0135]

[0136] Preferably, the hardware and software configuration is as shown in Table 2.

[0137] Table 2

[0138]

[0139] The defect detection method of PCB board based on deep learning described in the present invention can improve the ability of PCB defect detection under the premise of ensuring better deployment in resource-constrained equipment, and at the same time meet the requirements of speed and accuracy of detection in industrial production. The present invention improves the YOLOv8 target detection network, and introduces channel attention and spatial attention mechanisms to the original target detection network to increase the detection ability of PCB defects in view of the problems such as high false detection rate and slow speed of PCB detection on the current industrial production line; optimizes the positioning loss, and further improves the detection accuracy of the target network by introducing Inner-CIoU; then uses a lightweight convolution model to replace the original convolution structure to solve the deployment problem of the network model on resource-constrained equipment. Finally, after testing, the average accuracy mAP@0.5 of PCB defect detection of the present invention is 99.2%. The present invention can significantly solve the problem that deep learning networks are difficult to deploy in the current PCB industrial production, and meet the speed and accuracy requirements of detection in industrial production.

[0140] Finally, it should be noted that the above examples are only some specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments, and there are many variations. All variations that can be directly derived or associated with the content disclosed by a person skilled in the art should be considered as the protection scope of the present invention.

Claims

1. A PCB board surface defect detection algorithm based on deep learning, characterized by: The method is implemented by the following steps: Step S1, image acquisition: using an industrial camera to collect defect images generated during the production process of a bare PCB on a built PCB inspection platform, and obtaining a raw image data set of the bare PCB; Step S2, image preprocessing: expanding the data set by performing basic enhancement methods such as random rotation, random cropping, random adjustment of contrast or brightness, and random transformation of chromaticity on the original data set in step S1; Step S3, annotating and dividing the data set: using labelImg to annotate the defect locations and categories of the data set obtained after the image preprocessing in step S2, and dividing the data set and the annotated file into an image training set and an image test set at a ratio of 9:1; Step S4, improvement, optimization and training of the target detection network model: using the improved YOLOv8 target detection network model as the network model for PCB board surface defect detection, inputting the battery image data set prepared in step S3 into the network model for training and testing, and obtaining a trained PCB board surface defect detection network model; Step S5, online detection of PCB board surface defects: the PCB board surface defect detection network model with the highest average accuracy obtained after training in step S4 is placed in an online defect detection device. If any defect is detected on the surface of the PCB board transmitted to the device, the defect detection device will mark the detected defect in the annotation style in step S3, and save the defect data. The defective products can be found by comparing the product number with the defect detection data.

2. According to a deep learning-based PCB board surface defect detection algorithm according to claim 1, it is characterized in that: Adding annotations to the data set in step S3 refers to using the labelimg image annotation tool to select different defects on the surface of the PCB board and label them with the defect names. The defect names are the same as the label names, namely mouse_bite, spur, missing_hole, short, open_circuit, and spurious_copper.

3. According to the deep learning-based PCB surface defect detection algorithm of claim 1, it is characterized in that: The PCB board surface defect detection network model described in step S4 is an improved YOLOv8 target detection network model; the improvement of the YOLOv8 target detection network includes: Introduce lightweight convolution to reduce the number of network parameters; Introduce the GAM attention mechanism in the banckbone part of the network structure; Improve the loss function of the original network model and use Inner-CIoU as the positioning loss of the improved model.

4. According to the deep learning-based PCB surface defect detection algorithm of claim 3, it is characterized in that: Introducing lightweight convolution to reduce the number of network parameters is as follows: The C2f module in YOLOv8 adopts a multi-branch design of connecting multiple Bottleneck modules in series. Although it improves the model's ability to extract features of different scales, it will generate too much channel redundant information. By replacing the Bottleneck module with the PConv Block module, a lightweight improvement of the network structure can be achieved. The PConv module consists of two 3×3 PConv modules in series. The first PConv compresses the feature map, and the second PConv performs high-dimensional regression on the feature map and removes the residual connection to reduce excessive redundant information.

5. The PCB surface defect detection algorithm based on deep learning according to claim 3 is characterized in that: The process of introducing the GAM attention mechanism into the banckbone part of the network structure is specifically as follows: The GAM attention module is introduced after the SPPF module in the YOLOv8 network structure. The GAM module first passes the input feature map through the channel attention module, which maintains the three-dimensional channel information of the input feature; the channel attention module performs a permutation operation to convert the input feature map from the C×W×H format to the W×H×C format, and then generates a new feature map through inverse permutation and activation function; next, the output feature map is input into the spatial attention module; This module consists of two 7×7 convolutional layers and an activation function to process the feature map; after processing, the scaled output features and corresponding weight coefficients are obtained.

6. A PCB surface defect detection algorithm based on deep learning according to claim 3, characterized in that: The loss function of the improved YOLOv8 network model uses Inner-CIoU as the positioning loss in the loss function of the improved model, specifically: In the loss function of the YOLOv8 target detection network model, CIoU is used as the loss function for bounding box regression. Although the CIoU loss function simultaneously considers the center point distance, overlapping area and aspect ratio between the predicted box and the true box, effectively improving the accuracy of regression positioning, it does not consider the directional mismatch between the predicted box and the true box and the rationality of the IoU loss itself, which affects the PCB surface defect detection performance to a certain extent; the Inner-CIoU loss function uses an auxiliary anchor frame to reduce the overlapping area between the predicted box and the true box by a certain ratio and then calculates CIoU.

7. A system for implementing the deep learning-based PCB surface defect detection algorithm according to any one of claims 1 to 6, characterized in that: It includes the following modules: image acquisition module, defect annotation module, detection model training module, defect detection module, and defect data management module; The image acquisition module is used to collect PCB surface defect images and obtain PCB original data sets; The image preprocessing module is used to perform random rotation, random cropping, random adjustment of contrast or brightness, random transformation of chromaticity and other basic enhancement methods on the collected original PCB board image to expand the data set; The defect annotation module refers to using the labelimg image annotation tool to select different defects on the surface of the PCB board and annotate the defects according to the defect category. The system will automatically divide the data set and annotation file into an image training set and an image test set at a ratio of 9:1; The training module of the detection model is used to build a PCB board defect detection network model, and the PCB board image data set is input into the built target detection network model for training and testing. After 100 iterations, the best PCB board defect detection network model is obtained; The defect detection module places the best PCB board defect detection network model in an online defect detection device, and performs online detection on the incoming PCB board. If defects are found on the surface of the produced PCB board, the defect detection device marks the detected defects in the form of a rectangular frame and saves the detected defect data to the defect data management module; The defect data management module is used to store the location information, image information, category information and number information of the defects in the defective PCB board. The user can find out all defects and categories in the defective products by comparing the actual number of the PCB board with the defect data.

Images