A lightweight PCB defect detection method and system based on high-order spatial interaction

Abstract

The present invention relates to the technical field of PCB defect detection, and particularly to a lightweight PCB defect detection method and system based on high-order spatial interaction, which solves the technical problems in the background art. The method mainly introduces the HorFPN module, the CARAFE upsampling operator, and the lightweight convolution GhostConv on the basis of the YOLOv5 model. The system includes a data acquisition module, a parameter adjustment module, a PCB defect detection model, and a result display module. The present invention improves the high-order spatial interaction ability, feature expression ability, and multi-scale perception ability of the PCB defect detection model, effectively restores the detailed information of the feature map and improves the accuracy of target detection, while reducing the computational amount and parameter amount of the PCB defect detection model. It can intuitively display the detection results and accuracy, make the detection more rapid and efficient, improve the operability of PCB defect detection, and improve work efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of PCB defect detection, and in particular, to a lightweight PCB defect detection method and system based on high-order spatial interaction. Background Art

[0002] Printed circuit boards (PCBs) are an important part of electronic products, and their quality directly affects the performance and reliability of electronic products. However, various defects may occur during the manufacturing process of PCBs, such as missing holes, rat bites, open circuits, short circuits, burrs, and residual copper. Therefore, effective defect detection of PCBs is a key step in improving product quality and reducing costs.

[0003] Early research on PCB defect detection was mainly based on traditional detection techniques and machine learning techniques. However, traditional PCB defect detection techniques and machine learning-based techniques are no longer suitable for the current development trend. They have problems such as low efficiency, low precision, being easily affected by subjective factors, and being difficult to handle complex and variable defect scenarios. The progress of deep learning technology has provided new ideas and methods for PCB defect detection. Object detection networks based on deep learning can quickly, accurately, and stably detect various defects on PCBs. Currently, there are mainly two mainstream object detection schemes, namely Two-stage and One-stage detection. Two-stage is a detection algorithm based on candidate boxes, which is divided into two stages: the first stage is to generate regions that may contain the target, and the second stage is to classify the targets in these regions. RCNN and Faster R-CNN are typical Two-stage algorithms. Although Two-stage algorithms have high precision, they still perform poorly in terms of detection speed and small target detection problems. One-stage algorithms have only one stage. Instead of generating candidate regions and classifying them, they directly output the class probability and position coordinate values of the object, and can obtain the final detection result in one detection. It has the advantage of fast detection speed. The YOLO series, SSD, etc. are all relatively representative algorithms. Currently, methods based on deep learning have made some progress in PCB defect detection, but there are still some challenges, such as the small size of the dataset, the interference of small targets and complex backgrounds, and the demand for real-time performance. Therefore, there is still great room for development and potential in the industrial application of PCB defect detection. Summary of the Invention

[0004] To overcome the technical defects that existing algorithms still perform poorly in terms of detection speed and small target detection problems; the algorithm dataset is too small and is easily interfered by small targets and complex backgrounds, the present invention provides a lightweight PCB defect detection method and system based on high-order spatial interaction.

[0005] On the one hand, the present invention discloses a lightweight PCB defect detection method based on high-order spatial interaction, and the steps are as follows:

[0006] S1. Obtain a PCB defect data set, perform data augmentation operations on the data set, complete data preprocessing, and divide the training set, validation set, and test set according to a ratio;

[0007] S2. Construct a lightweight PCB defect detection model based on high-order spatial interaction, namely the YOLOv5-HorL model. The YOLOv5-HorL model introduces the HorFPN module, lightweight CARAFE upsampling operator, and lightweight convolution GhostConv on the basis of the YOLOv5 model;

[0008] S3. Train the YOLOv5-HorL model through the training set, adjust the hyperparameters, calculate the mean average precision of the adjusted YOLOv5-HorL model on the validation set, and judge whether the mAP of the current model is the best. If so, save the current model as the best PCB defect detection model;

[0009] S4. Input the test set into the best PCB defect detection model to obtain the defect position, defect type, and corresponding detection accuracy of the PCB image to be detected.

[0010] In the described lightweight PCB defect detection method based on high-order spatial interaction, the YOLOv5-HorL model is applied to the PCB image to generate defect detection results. In step S1, data preprocessing uses a series of data augmentation operations to expand the PCB defect data set to avoid the overfitting problem caused by insufficient data volume. The HorFPN module can achieve efficient high-order spatial interaction and improve the performance of the object detection task; the lightweight CARAFE upsampling operator uses a small convolutional network to predict the upsampling weights at each position, and then uses these weights to weighted-combine the surrounding features to achieve the upsampling of the feature map; the lightweight convolutional GhostConv decomposes a standard convolutional kernel into two parts: a main convolutional kernel and some simple linear transformations. The main convolutional kernel is responsible for extracting the main information of the input features, while the linear transformation is used to generate more feature maps to enhance the expressive ability of the model. In this way, a lightweight convolutional GhostConv can achieve the same or similar effect as a standard convolution with less computational resources. By introducing the HorFPN module, the lightweight CARAFE upsampling operator, and the lightweight convolutional GhostConv into the network of the YOLOv5-HorL model, the high-order spatial interaction ability, the feature expressive ability, and the multi-scale perception ability of the model are improved, the detailed information of the feature map is effectively restored, and the accuracy of object detection is improved. At the same time, the computational amount and the number of parameters of the model are reduced. Thereby, the detection cost and the false detection rate can be reduced, and the accuracy of PCB defect detection can be improved.

[0011] Preferably, in step S1, the data augmentation operations include random flipping, random rotation, and HSV adjustment. Random flipping randomly selects a direction on the original image and then flips it along this direction as a new image, while adjusting the direction and position of the corresponding annotation box. This increases the model's detection ability for objects with different symmetries and perspectives. Random rotation randomly selects an angle on the original image and then rotates it by a certain angle as a new image, while adjusting the angle and position of the corresponding annotation box. This increases the model's detection ability for objects with different directions and poses. HSV adjustment randomly selects some parameters on the original image and then performs some color space transformations as a new image. This increases the model's detection ability for objects under different lighting conditions and background colors.

[0012] Preferably, in step S1, the PCB defect data set contains six types of defects: missing holes, mouse bites, open circuits, short circuits, burrs, and excess copper. There are about 1-5 types of defects on each picture, and the number of each type of defect is balanced. And the PCB defect data set contains PCBs of various sizes, with the largest being 12.5 cm × 12 cm and the smallest being 5.3 cm × 4.8 cm.

[0013] Preferably, in step S2, the steps for constructing the YOLOv5-HorL model are as follows:

[0014] S201. Construct a PCB defect detection model based on the YOLOv5 algorithm;

[0015] S202. Introduce the HorFPN module to replace the FPN module in the YOLOv5 algorithm, introduce the CARAFE upsampling operator to replace the default upsampling method in the YOLOv5 algorithm, and introduce the lightweight convolution GhostConv to replace the ordinary convolution in the YOLOv5 algorithm. The HorFPN module is used instead of the FPN module for spatial convolution of feature fusion to achieve efficient high-order spatial interaction and improve the performance of the object detection task.

[0016] Preferably, in step S2, the HorFPN module is improved from the recursive gated convolution g n Conv; the recursive gated convolution g n Conv uses gated convolution to achieve input adaption and high-order spatial interaction, and at the same time uses a large convolution kernel or a global filter to achieve large-range spatial interaction. The operations of the recursive gated convolution g n Conv include the following steps:

[0017] a) For the input feature x, g n Conv projects it into n + 1 subspaces p0 and q k (k = 0, 1,..., n - 1);

[0018] b) Recursively execute gated convolution to obtain p k+1 (k = 0, 1,..., n - 1);

[0019] c) Concatenate all p k (k = 0, 1,..., n) and obtain the output y through a linear projection;

[0020] where p and q are projection features, and the subscripts are subspaces.

[0021] Preferably, in step S2, the CARAFE upsampling operator uses a convolutional network to predict the upsampling weights at each position, and then uses the obtained upsampling weights to weighted-combine the surrounding features to achieve upsampling of the feature map. The operation process of the CARAFE upsampling operator is as follows:

[0022] W l′ = Ψ(N(X l , k encoder )) (1)

[0023] X' l′ = φ(N(X l , kup ),W l′ ) (2)

[0024] In Formulas (1) and (2), Ψ is the upsampling kernel prediction module, φ is the content-aware recombination module, the kernel W l′ is the upsampling kernel, X l ' ′ is the output feature map after content recombination, N(X l , k) represents the neighborhood of X l in the feature map, k encoder is the size of the encoder convolution kernel, and k up is the size of the upsampling kernel.

[0025] Preferably, in step S2, the lightweight convolution GhostConv uses a group of low-rank convolution kernels as a complete convolution kernel to accelerate the convolution operation. The operation process of the lightweight convolution GhostConv is as follows:

[0026] Y = [F m *X, F c *(F m *X)] (3)

[0027] In Formula (3), X is the input feature map, F m is the filter of the main convolution layer, F c is the filter of the linear transformation, * represents the convolution operation, and [·, ·] represents concatenating two feature maps along the channel dimension.

[0028] Preferably, in step S3, the SGD optimizer is used in the training process. The loss function includes a classification loss function, a regression loss function, and a confidence loss function. The classification loss function and the confidence loss function are calculated using the cross-entropy loss function, and the regression loss function is calculated using the GIOU loss function. During actual training, the weight decay is 0.0005, the initial learning rate is set to 0.01, the minimum learning rate is set to 0.00008, and the model is saved after convergence.

[0029] Preferably, in step S4, the evaluation metrics used in the accuracy detection process include precision, recall, mean average precision (mAP), computational complexity, and number of parameters. The precision and recall are metrics that measure the classifier's ability to identify positive samples from different perspectives. Precision represents the proportion of true positive samples among the data predicted as positive samples, i.e., the proportion of "correctly found". Recall represents the proportion of true positive samples that are predicted as positive samples, i.e., the proportion of "fully found". The precision and recall respectively reflect the performance of the network from two aspects. Combining these two metrics can more effectively evaluate the accuracy of the network. Generally, average precision is used to combine these two metrics. Mean average precision is the average of the APs for all classes. The larger the mAP, the better the detection algorithm's ability to identify objects of different classes. mAP is usually the main metric for comparing object detection algorithms. The computational complexity and number of parameters are used to measure the time complexity and space complexity of the model respectively. Computational complexity refers to the number of floating-point operations per second, and the number of parameters refers to the total number of parameters that need to be trained in the network model.

[0030] On the other hand, the present invention also discloses a lightweight PCB defect detection system based on high-order spatial interaction, including a data acquisition module, a parameter adjustment module, a PCB defect detection model, and a result display module; the data acquisition module is used to acquire PCB image data; the parameter adjustment module adjusts the non-maximum suppression IOU and confidence level to meet different detection requirements of users; the PCB defect detection model is constructed based on the YOLOv5-HorL model in the lightweight PCB defect detection method based on high-order spatial interaction of the present invention, and the PCB image is input into the PCB defect detection model for defect detection; the result display module is used to display the type, accuracy, size, and location indicators of the PCB defects and generate the detection results for display to the user, and finally automatically save the detection results. The system operates in real-time and automatically generates detection results in real-time.

[0031] The technical solution provided by the present invention has the following advantages compared with the prior art: The lightweight PCB defect detection method based on high-order spatial interaction of the present invention introduces the HorFPN module, lightweight CARAFE upsampling operator, and lightweight convolutional GhostConv in the YOLOv5-HorL network, improving the high-order spatial interaction ability, feature expression ability, and multi-scale perception ability of the PCB defect detection model, effectively restoring the detailed information of the feature map and improving the accuracy of object detection, while reducing the computational complexity and number of parameters of the PCB defect detection model. Thereby, it can reduce the detection cost and false detection rate, and thus improve the accuracy of PCB defect detection. The lightweight PCB defect detection method and system based on high-order spatial interaction can intuitively display the detection results and accuracy, making the detection faster and more efficient, greatly improving the operability of PCB defect detection and improving the work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present invention and, together with the specification, are used to explain the principles of the present invention.

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0034] Figure 1 It is the overall flowchart of a lightweight PCB defect detection method based on high-order spatial interaction provided in the embodiments of the present invention;

[0035] Figure 2 It is the network structure diagram of the YOLOv5-HorL model in the specific implementation manner of a lightweight PCB defect detection method based on high-order spatial interaction provided in the embodiments of the present invention;

[0036] Figure 3 It is the g in the specific implementation manner of a lightweight PCB defect detection method based on high-order spatial interaction provided in the embodiments of the present invention n The structure diagram of the Conv recursive gated convolution;

[0037] Figure 4 It is the structure diagram of the CARAFE upsampling operator in the specific implementation manner of a lightweight PCB defect detection method based on high-order spatial interaction provided in the embodiments of the present invention;

[0038] Figure 5 It is the structure diagram of the GhostConv lightweight convolution in the specific implementation manner of a lightweight PCB defect detection method based on high-order spatial interaction provided in the embodiments of the present invention;

[0039] Figure 6 It is the module schematic diagram of a lightweight PCB defect detection system based on high-order spatial interaction provided in the embodiments of the present invention;

[0040] Figure 7 It is the detailed module diagram of a lightweight PCB defect detection system based on high-order spatial interaction provided in the embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] In order to better understand the above objects, features and advantages of the present invention, the following will further describe the solutions of the present invention. It should be noted that, without conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other.

[0042] In the description, it should be noted that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. It should be noted that unless otherwise clearly specified and defined, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0043] In the following description, many specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways different from those described herein; obviously, the embodiments in the specification are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0044] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0045] In one embodiment, as Figures 1-7 shown, a lightweight PCB defect detection method based on high-order spatial interaction, as Figure 1 shown, the steps are as follows:

[0046] S1. Obtain a PCB defect data set, perform data augmentation operations on the data set, complete data preprocessing, and divide the training set, validation set, and test set according to a ratio.

[0047] S2. Construct a lightweight PCB defect detection model based on high-order spatial interaction, namely the YOLOv5-HorL model. The YOLOv5-HorL model introduces the HorFPN module, lightweight CARAFE upsampling operator, and lightweight convolutional GhostConv on the basis of the YOLOv5 model.

[0048] S3. Train the YOLOv5-HorL model with the training set and adjust the hyperparameters. Calculate the mean average precision of the adjusted YOLOv5-HorL model on the validation set, and determine whether the mAP of the current model is optimal. If so, save the current model as the best PCB defect detection model.

[0049] S4. Input the test set into the best PCB defect detection model to obtain the defect location, defect type, and corresponding detection accuracy of the PCB image to be detected.

[0050] In the lightweight PCB defect detection method based on high-order spatial interaction, the YOLOv5-HorL model is applied to the PCB image to generate defect detection results. In a specific embodiment, in step S1, 693 images suitable for the detection task are selected, and a series of data augmentation operations are used in data preprocessing to expand the PCB defect data set to avoid the overfitting problem caused by insufficient data volume. The HorFPN module can achieve efficient high-order spatial interaction and improve the performance of the object detection task; the lightweight CARAFE upsampling operator uses a small convolutional network to predict the upsampling weights at each position, and then uses these weights to weighted-combine the surrounding features to achieve the upsampling of the feature map; the lightweight convolutional GhostConv decomposes a standard convolutional kernel into two parts: a main convolutional kernel and some simple linear transformations. The main convolutional kernel is responsible for extracting the main information of the input features, while the linear transformation is used to generate more feature maps to enhance the expression ability of the model. In this way, a lightweight convolutional GhostConv can achieve the same or similar effect as a standard convolution with fewer computing resources. By introducing the HorFPN module, the lightweight CARAFE upsampling operator, and the lightweight convolutional GhostConv into the network of the YOLOv5-HorL model, the high-order spatial interaction ability, the feature expression ability, and the multi-scale perception ability of the model are improved, the detailed information of the feature map is effectively restored, and the accuracy of object detection is improved. At the same time, the computational amount and the number of parameters of the model are reduced. Thus, the detection cost and the false detection rate can be reduced, and the accuracy of PCB defect detection can be improved.

[0051] In this embodiment, the basic operation principle of the YOLOv5 model is to divide the input image into small grids, and each grid is responsible for predicting a certain number of bounding boxes and class probabilities. Then, a convolutional neural network (CNN) is used to extract features and classify the predictions of each grid. Finally, the final detection results are selected through non-maximum suppression (NMS). The structure of the YOLOv5 model is mainly divided into the following three parts:

[0052] I. The backbone network (Backbone) uses CSPDarknet, which is mainly composed of the Focus network structure and the CSPNet network structure. First, the Focus network extracts features from the input image and adjusts and expands the number of channels. Then, three effective feature layers of different scales are output through four residual structures composed of CSPNet. Finally, the SPP structure is added to improve the receptive field of the network;

[0053] II. The Neck network adopts the combination of FPN (Feature Pyramid Network) and PAN (Path Aggregation Network) to form a bidirectional feature pyramid network, enabling each scale of features to obtain richer and more balanced information;

[0054] III. The Prediction network adopts a design similar to YOLOv3 and YOLOv4. It uses a convolutional layer to receive feature maps from different scales and outputs a four-dimensional tensor representing the prediction results of each grid cell.

[0055] In a specific embodiment, the training process of the PCB defect detection model in step S3 of the method includes the following steps:

[0056] S301. Randomly divide the PCB defect dataset into a training set, a validation set, and a test set according to a ratio of 8:1:1, ensuring that the defect types and quantities in each dataset are evenly distributed;

[0057] S302. Load the YOLOv5-HorL model and initialize the hyperparameters, where the hyperparameters include: weight decay coefficient, learning rate, training batch size, number of iteration rounds, and coefficient values of each part of the loss function;

[0058] S303. After loading the network model, perform feature extraction, defect localization, and classification on the input training set images;

[0059] S304. In each training round, the network calculates the loss function and then uses the SGD optimizer to optimize the parameters;

[0060] S305. Calculate the mean Average-Precision (mAP) of the network model on the validation set and determine whether the mAP of this model is the best. If so, save the model.

[0061] Based on the above embodiments, in a preferred embodiment, in step S1, the data augmentation operations include random flipping, random rotation, and HSV adjustment. Random flipping randomly selects a direction on the original image and then flips it along that direction as a new image, while adjusting the direction and position of the corresponding annotation box. This increases the model's detection ability for objects with different symmetries and perspectives. Random rotation randomly selects an angle on the original image and then rotates it by a certain angle as a new image, while adjusting the angle and position of the corresponding annotation box. This increases the model's detection ability for objects with different directions and postures. HSV adjustment randomly selects some parameters on the original image and then performs some color space transformations as a new image. This increases the model's detection ability for objects under different lighting conditions and background colors.

[0062] Based on the above embodiments, in a preferred embodiment, in step S1, the PCB defect dataset contains six types of defects: missing holes, rat bites, open circuits, short circuits, burrs, and excess copper. There are approximately 1 - 5 types of defects on each picture, and the quantity of each type of defect is balanced. And the PCB defect dataset contains PCBs of various sizes, with the largest being 12.5 cm × 12 cm and the smallest being 5.3 cm × 4.8 cm.

[0063] Based on the above embodiments, in a preferred embodiment, in step S2, the structure of the YOLOv5 - HorL model is as Figure 2 shown, and the construction steps of the YOLOv5 - HorL model are as follows:

[0064] S201. Construct a PCB defect detection model based on the YOLOv5 algorithm;

[0065] S202. Introduce the HorFPN module to replace the FPN module in the YOLOv5 algorithm, introduce the CARAFE upsampling operator to replace the default upsampling method in the YOLOv5 algorithm, and introduce the lightweight convolution GhostConv to replace the ordinary convolution in the YOLOv5 algorithm. The HorFPN module is used for spatial convolution of feature fusion instead of the FPN module to achieve efficient high - order spatial interaction and improve the performance of the object detection task.

[0066] Based on the above embodiments, in a preferred embodiment, in step S2, as Figure 3 shown, the HorFPN module is improved based on the recursive gated convolution g n Conv proposed in HorNet, replacing the spatial convolution used for feature fusion in FPN to achieve efficient high - order spatial interaction and improve the performance of the object detection task; the recursive gated convolution g nConv uses gated convolution to achieve input adaption and high-order spatial interaction, and uses large convolutional kernels or global filters to achieve large-range spatial interaction. Recursive gated convolution g n The operations of Conv include the following steps:

[0067] a) For the input feature x, g n Conv projects it into n + 1 subspaces p0 and q k (k = 0, 1,..., n - 1);

[0068] b) Recursively execute gated convolution to obtain p k+1 (k = 0, 1,..., n - 1);

[0069] c) Concatenate all p k (k = 0, 1,..., n) and obtain the output y through a linear projection;

[0070] Among them, p and q are projected features, and the subscripts are subspaces.

[0071] Based on the above embodiments, in a preferred embodiment, in step S2, as Figure 4 shown, the CARAFE upsampling operator uses a convolutional network to predict the upsampling weights at each position, and then uses the obtained upsampling weights to weighted-combine the surrounding features to achieve the upsampling of the feature map. The operation process of the CARAFE upsampling operator is as follows:

[0072] W l′ = Ψ(N(X l , k encoder )) (1)

[0073] X' l′ = φ(N(X l , k up ), W l′ ) (2)

[0074] In formulas (1) and (2), Ψ is the upsampling kernel prediction module, φ is the content-aware recombination module, the kernel W l′ is the upsampling kernel, X' l′ is the output feature map after content recombination, N(X l , k) represents the neighborhood of X l in the feature map, k encoder is the size of the encoder convolutional kernel, and k up is the size of the upsampling kernel.

[0075] Based on the above embodiments, in a preferred embodiment, in step S2, as Figure 5As shown, the lightweight convolutional GhostConv decomposes a standard convolutional kernel into two parts: a primary convolutional kernel and some simple linear transformations. The primary convolutional kernel is responsible for extracting the main information of the input features, while the linear transformations are used to generate more feature maps, thereby enhancing the expressive power of the model. In this way, a GhostConv can achieve the same or similar effect as a standard convolution with fewer computational resources. The lightweight convolutional GhostConv uses a set of low-rank convolutional kernels as a complete convolutional kernel to accelerate the convolution operation. The operation process of the lightweight convolutional GhostConv is as follows:

[0076] Y = [F m * X, F c *(F m * X)] (3)

[0077] In formula (3), X is the input feature map, F m is the filter of the primary convolutional layer, F c is the filter of the linear transformation, * represents the convolution operation, and [·,·] represents concatenating two feature maps along the channel dimension.

[0078] Based on the above embodiments, in a preferred embodiment, in step S3, the SGD optimizer is used during the training process. The loss function includes a classification loss function, a regression loss function, and a confidence loss function. The classification loss function and the confidence loss function are calculated using the cross-entropy loss function, and the regression loss function is calculated using the GIOU loss function. During actual training, the weight decay is 0.0005, the initial learning rate is set to 0.01, the minimum learning rate is set to 0.00008, and the model is saved after convergence.

[0079] Based on the above embodiments, in a preferred embodiment, in step S4, the evaluation metrics used in the accuracy detection process include Precision, Recall, mean Average-Precision (mAP), FLOPs (Floating Point Operations), and Params (parameters).

[0080] Among them, the Precision and Recall are metrics that measure the classifier's ability to identify positive samples from different perspectives. Precision represents the proportion of true positive samples among the data predicted as positive samples, that is, the proportion of "correctly found"; Recall represents the proportion of true positive samples that are predicted as positive samples, that is, the proportion of "fully found". The calculation formulas for Precision and Recall are as follows:

[0081]

[0082]

[0083] In formulas (4) and (5), TP represents true positive, that is, predicted as positive sample and correctly recognized; TN represents true negative, that is, predicted as negative sample and correctly recognized; FP represents false positive, that is, predicted as positive sample but incorrectly recognized; FN represents false negative, that is, predicted as negative sample but incorrectly recognized. Precision and recall respectively reflect the performance of the network from two aspects. Combining these two indicators can more effectively evaluate the accuracy of the network. Generally, the average precision (AP) is used to combine these two indicators.

[0084] The mean average precision is the average of the APs of all classes. The larger the mAP, the better the detection algorithm's ability to recognize objects of different classes. mAP is usually used as the main indicator for comparing object detection algorithms. The calculation formula for the mean average precision is:

[0085]

[0086] In formula (6), Ap(j) represents the average precision of the j-th class target, and M represents the total number of classes. The average precision (AP) represents the average of the precisions at different recall rates for a certain class, and the mean average precision (mAP) is the average of the average precisions (APs) of all classes.

[0087] The above-mentioned computational complexity and number of parameters are respectively used to measure the time complexity and space complexity of the model. The computational complexity refers to the number of floating-point operations per second, and the number of parameters refers to the total number of parameters that need to be trained in the network model.

[0088] Table 1 Comparison of indicators of different object detection algorithms on the PCB dataset

[0089]

[0090]

[0091] As shown in Table 1, to verify the superiority of the method described in the present invention, the mAP, Precision, Recall, computational complexity, and number of parameters of different algorithms were compared under the same experimental environment configuration. The results show that the mAP of YOLOv5-HorL of the present invention is as high as 96.26%, which is increased by 43.36%, 5.95%, 14.99%, 1.95%, and 4.99% compared with Faster R-CNN, SSD, YOLOv3, YOLOv4, and YOLOv5s respectively. At the same time, Precision and Recall are 97.90% and 91.84% respectively, which are greatly improved compared with Faster R-CNN, SSD, YOLOv3, YOLOv4, and YOLOv5s. The computational complexity and number of parameters of YOLOv5-HorL of the present invention have also been reduced to a certain extent. It is not only further lightweight on the basis of YOLOv5s but also achieves higher detection accuracy. The experimental results prove the effectiveness of the present invention and have more practical value.

[0092] In a certain embodiment, the present invention also discloses a lightweight PCB defect detection system based on high-order spatial interaction, as Figure 6 and Figure 7 shown, including a data acquisition module, a parameter adjustment module, a PCB defect detection model, and a result display module; the data acquisition module is used to acquire PCB image data; the parameter adjustment module adjusts the non-maximum suppression IOU and confidence level to meet different detection requirements of users; the PCB defect detection model is constructed based on the YOLOv5-HorL model in a lightweight PCB defect detection method described in the present invention, and the PCB image is input into the PCB defect detection model for defect detection; the result display module is used to display the type, accuracy, size, and position indicators of PCB defects and generate a detection result to show to the user, and finally automatically save the detection result. The system runs in real time and automatically generates detection results in real time. The system is based on a processor, a memory, and a computer program, and the lightweight PCB defect detection system described in the present invention is created through the computer program. In the lightweight PCB defect detection system, the YOLOv5-HorL model is used to perform real-time detection on the PCB image and give a detection result.

[0093] After the data acquisition module obtains the PCB image data, it also performs data preprocessing on the image. The preprocessing includes data augmentation processing on the input image to reduce the overfitting phenomenon, and normalization processing on the input image. The PCB image is generated by using an image collector to photograph the PCB, and can be a real-time photographed image or video, or an image stored in a local server or cloud, and is obtained by means of reading or network transmission. The parameter adjustment module is used to adjust the non-maximum suppression IOU and confidence to meet different detection requirements of users. In some embodiments, the adjustment ranges of the non-maximum suppression IOU and confidence are from 0 to 1. The result display module is used to receive the detection result, automatically determine qualitative and quantitative indicators such as the type, accuracy, size and position of the PCB defect and display them to the user, and finally automatically save the detection result. The type is the detected PCB defect, including missing holes, rat bites, open circuits, short circuits, burrs, and residual copper. In some embodiments, the automatically saved detection result is a text display of the above quantitative and qualitative data.

[0094] For a lightweight PCB defect detection method and system based on high-order spatial interaction according to the present invention, first, the PCB image is subjected to data preprocessing to avoid the overfitting problem caused by insufficient data volume, then the processed PCB image is input into the PCB defect detection model for defect detection, and finally the detection result is automatically obtained; effectively improving the accuracy and efficiency of PCB defect detection and the intelligent level of defect detection.

[0095] The above are only specific embodiments of the present invention, enabling those skilled in the art to understand or implement the present invention. Although the foregoing embodiments have been described in detail, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments, and they should all be covered by the protection scope of the claims.

Claims

1. A lightweight PCB defect detection method based on high-order spatial interaction, characterized in that, The steps are as follows: S1. Obtain the PCB defect dataset, perform data augmentation operations on the dataset, complete data preprocessing, and divide the training set, validation set, and test set according to a ratio; S2. Construct a lightweight PCB defect detection model based on high-order spatial interaction, namely the YOLOv5-HorL model. The YOLOv5-HorL model introduces the HorFPN module, lightweight CARAFE upsampling operator, and lightweight convolution GhostConv on the basis of the YOLOv5 model; Among them, the construction steps of the YOLOv5-HorL model are as follows: S201. Construct a PCB defect detection model based on the YOLOv5 algorithm; S202. Introduce the HorFPN module to replace the FPN module in the YOLOv5 algorithm, introduce the CARAFE upsampling operator to replace the default upsampling method in the YOLOv5 algorithm, and introduce the lightweight convolution GhostConv to replace the ordinary convolution in the YOLOv5 algorithm; The HorFPN module is improved based on the recursive gated convolution g n Conv proposed in HorNet; the recursive gated convolution g n Conv uses gated convolution to achieve input adaptation and high-order spatial interaction, and at the same time uses large convolution kernels or global filters to achieve large-range spatial interaction. The operations of the recursive gated convolution g n Conv include the following steps: a) For the input feature x, g n Conv projects it onto n + 1 subspaces p0 and q k , where k = 0, 1,..., n - 1; b) Perform gated convolution iteratively to obtain p k+1 , where k = 0, 1, ..., n - 1; c) Concatenate all \(p\) k , \(k = 0, 1, \ldots, n\) and obtain the output \(y\) through a linear projection; where \(p\) and \(q\) are projection features and the subscripts are subspaces. The CARAFE upsampling operator uses a convolutional network to predict the upsampling weights at each position, and then uses the obtained upsampling weights to weighted combine the surrounding features to achieve the upsampling of the feature map. The operation process of the CARAFE upsampling operator is as follows: W l′ = Ψ(N(X l , k encoder )) (1) X' l′ = φ(N(X l , k up ), W l′ ) (2) In Formulas (1) and (2), Ψ is the upsampling kernel prediction module, φ is the content-aware recombination module, the kernel W l′ is the upsampling kernel, X' l′ is the output feature map after content recombination, N(X l , k) represents the neighborhood of X l in the feature map, k encoder is the size of the encoder convolution kernel, k up is the size of the upsampling kernel; S3. Train the YOLOv5-HorL model with the training set, adjust the hyperparameters, calculate the mean average precision of the adjusted YOLOv5-HorL model on the validation set, and judge whether the mAP of the current model is the best. If so, save the current model as the best PCB defect detection model; S4. Input the test set into the best PCB defect detection model to obtain the defect location, defect type, and corresponding detection accuracy of the PCB image to be detected.

2. The lightweight PCB defect detection method based on high-order spatial interaction according to claim 1, wherein In step S1, the data augmentation operations include random flipping, random rotation, and HSV adjustment.

3. The lightweight PCB defect detection method based on high-order spatial interaction according to claim 2, characterized in that In step S1, the PCB defect dataset includes six types of defects: missing holes, rat bites, open circuits, short circuits, burrs, and surplus copper.

4. The lightweight PCB defect detection method based on high-order spatial interaction according to claim 3, characterized in that, In step S2, the lightweight convolution GhostConv uses a group of low-rank convolution kernels as a complete convolution kernel to accelerate the convolution operation. The operation process of the lightweight convolution GhostConv is as follows: Y = [F m *X, F c *(F m *X)] (3) In formula (3), X is the input feature map, F m is the filter of the main convolutional layer, F c is the filter of the linear transformation, * represents the convolution operation, and [·,·] represents concatenating two feature maps along the channel dimension.

5. The lightweight PCB defect detection method based on high-order spatial interaction according to claim 4, characterized in that, In step S3, the SGD optimizer is used in the training process. The loss function includes a classification loss function, a regression loss function, and a confidence loss function. The classification loss function and the confidence loss function are calculated using the cross-entropy loss function, and the regression loss function is calculated using the GIOU loss function.

6. The lightweight PCB defect detection method based on high-order spatial interaction according to claim 5, characterized in that In step S4, the evaluation metrics used in the accuracy detection process include precision, recall, mean average precision, computational complexity, and number of parameters.

7. A lightweight PCB defect detection system based on high-order spatial interaction, characterized in that, It includes a data acquisition module, a parameter adjustment module, a PCB defect detection model, and a result display module; the data acquisition module is used to acquire PCB image data; the parameter adjustment module adjusts the non-maximum suppression IOU and confidence to meet different detection requirements of users; The PCB defect detection model is constructed based on the YOLOv5-HorL model in a lightweight PCB defect detection method based on high-order spatial interaction described in claim 4. The PCB image is input into the PCB defect detection model for defect detection; The result display module is used to display the type, accuracy, size, and location indicators of PCB defects, generate the detection results and present them to the user, and finally automatically save the detection results.

Images