Circuit board defect detection method, device, equipment and storage medium

Abstract

The present invention relates to the field of image recognition technology, and in particular to a circuit board defect detection method, apparatus, device and storage medium. The method comprises: acquiring an image to be detected and preprocessing it to obtain a preprocessed image; inputting the preprocessed image into a pre-built defect detection model, performing feature extraction processing on the preprocessed image based on pre-confirmed convolution weights, and fusing feature maps of multiple scales based on pre-confirmed fusion weights, and then using a detection head to perform defect detection on the fused feature map to obtain a defect detection result. The defect detection method disclosed in the present application aims to solve the problems of high computational complexity, poor real-time performance and insensitivity to small target detection in existing PCB defect detection methods. By introducing a dynamic convolution selection mechanism to adjust the convolution weights, it is ensured that the model has high precision when processing tiny defects. By introducing a weight adaptive mechanism to adjust the fusion weights, the calculation of redundant information can be reduced, thereby greatly improving detection efficiency.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a circuit board defect detection method, device, equipment and storage medium. Background Art

[0002] Printed circuit boards (PCBs) are the core of electronic devices, and their quality directly determines the performance and lifespan of the devices. Defects on PCBs, including short circuits, open circuits, and soldering issues, can cause circuit failures, thereby affecting the stable operation and safety of the devices.

[0003] Traditionally, PCB defect detection relies on manual inspection or rule-based image processing technology. These methods suffer from low detection efficiency and unstable detection quality, especially when dealing with complex PCB structures and tiny defects, making it difficult to ensure detection accuracy and efficiency. Although traditional image processing-based methods have achieved a certain degree of automation, their stability is insufficient, especially when facing diverse and complex PCB designs. They are prone to misjudgment or omission, which reduces the accuracy of defect detection.

[0004] With the rapid development of deep learning technology, object detection methods based on convolutional neural networks (CNNs) have begun to emerge in the field of PCB defect detection. In particular, the YOLO series of models have achieved remarkable success in the field of object detection with their end-to-end rapid detection capabilities. The YOLO model can predict the location and category of multiple objects in a single forward propagation. Compared with traditional sliding window or region-based detection methods, the YOLO model has a significant advantage in detection efficiency.

[0005] However, the YOLO model, especially the newer YOLOv8 version, although it has improved detection accuracy and speed, has a complex model structure and many parameters. In other words, the model requires a lot of hardware resources, resulting in expensive computational costs. This makes them difficult to popularize in embedded devices or industrial applications with extremely high real-time requirements, which limits their application in resource-constrained environments. In addition, the existing YOLO model is not sensitive enough to the detection of small targets and subtle defects. Especially in complex backgrounds or occlusions, the model is often prone to insufficient robustness, which reduces detection accuracy.

[0006] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the Invention

[0007] In order to overcome the shortcomings of the existing technology, the purpose of the present invention is to provide a circuit board defect detection method, which aims to solve the problems of high computational complexity, poor real-time performance, and insensitivity to small target detection in the existing PCB defect detection methods.

[0008] The first aspect of the present invention provides a circuit board defect detection method, comprising: acquiring an image to be detected and preprocessing it to obtain a preprocessed image; inputting the preprocessed image into a pre-built defect detection model, wherein the defect detection model comprises an extraction network, a feature fusion network and a detection head connected in sequence; confirming the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, using the extraction network to perform feature extraction processing on the preprocessed image to obtain feature maps of multiple scales; confirming the fusion weight corresponding to the scale, and based on the fusion weights, using the feature fusion network to perform fusion processing on the feature maps of multiple scales to obtain a fused feature map; and using the detection head to perform defect detection on the fused feature map to obtain a defect detection result.

[0009] Optionally, in a first implementation method of the first aspect of the present invention, the acquiring of the image to be detected and preprocessing thereof to obtain a preprocessed image includes: acquiring the image to be detected, denoising the image to be detected using a median filtering method, and performing grayscale conversion on the denoised image to be detected to obtain a grayscale image; performing contrast enhancement processing on the grayscale image using a sharpening filter to obtain an enhanced image; using an edge detector to identify the edge contour of the circuit board in the enhanced image, and segmenting the circuit board and the background using a threshold segmentation method to obtain a preprocessed image.

[0010] Optionally, in a second implementation of the first aspect of the present invention, the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the size of the convolution kernel of each convolution layer is different, and each convolution layer corresponds to a convolution weight.

[0011] Optionally, in a third implementation method of the first aspect of the present invention, the convolution weights of the convolution modules included in the confirmation extraction network are confirmed, and based on the convolution weights, the extraction network is used to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales, including: performing a global average pooling operation on the input preprocessed image to obtain first global context information; in the first fully connected layer, according to the first global context information, a Sigmoid activation function is used to generate multiple convolution weights corresponding to multiple convolution layers respectively; in the convolution module, each convolution layer performs a convolution operation on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map; and integrating the feature maps output by multiple convolution modules to obtain multiple feature maps of different scales.

[0012] Optionally, in a fourth implementation manner of the first aspect of the present invention, the fusion weight corresponding to the scale is confirmed, and based on the fusion weight, a feature fusion network is used to fuse feature maps of multiple scales to obtain a fused feature map, including: in the feature fusion network, global average pooling operations are performed on feature maps of different scales to obtain second global context information; the second fully connected layer included in the feature fusion network is used to obtain the fusion weight corresponding to the scale based on the second global context information and the ReLU activation function; based on the fusion weight corresponding to the scale, weighted summation is performed on feature maps of multiple scales to obtain a fused feature map.

[0013] Optionally, in a fifth implementation of the first aspect of the present invention, the detection head is used to perform defect detection on the fused feature map to obtain a defect detection result, including: obtaining pruning information, the pruning information including multiple pruning ratios, and the multiple pruning ratios correspond one-to-one to each layer included in the detection head; the detection head is used to perform defect detection on the fused feature, and during the defect detection process, the layers included in the detection head are pruned according to the pruning information; and the defect detection result output by the detection head is obtained.

[0014] Optionally, in a sixth implementation manner of the first aspect of the present invention, the inputting of the preprocessed image into the pre-built defect detection model includes: when constructing the defect detection model, selecting a YOLOv8 model that can be used for circuit board defect detection as a teacher model, and obtaining soft labels fed back by the teacher model; obtaining hard labels included in the data set used to train the defect detection model; setting the distillation loss function of the defect detection model based on the soft labels, and setting the task loss function of the defect detection model based on the hard labels.

[0015] The second aspect of the present invention provides a circuit board defect detection device, comprising: a processing module, used to acquire an image to be detected and preprocess it to obtain a preprocessed image; an input module, used to input the preprocessed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network and a detection head connected in sequence; a feature extraction module, used to confirm the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction processing on the preprocessed image to obtain feature maps of multiple scales; a fusion module, used to confirm the fusion weight corresponding to the scale, and based on the fusion weights, use the feature fusion network to perform fusion processing on the feature maps of multiple scales to obtain a fused feature map; a detection module, used to use the detection head to perform defect detection on the fused feature map to obtain a defect detection result.

[0016] The third aspect of the present invention provides a circuit board defect detection device, which includes: a memory and at least one processor, wherein the memory stores instructions; at least one processor calls the instructions in the memory so that the circuit board defect detection device performs each step of the circuit board defect detection method described above.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, implement the various steps of any of the above-mentioned circuit board defect detection methods.

[0018] The technical solution of the present invention aims to address the problems of high computational complexity, poor real-time performance, and insensitivity to small target detection in existing PCB defect detection methods. By introducing a dynamic convolution selection mechanism to adjust the convolution weights of the extraction network, the defect detection model ensures high detection accuracy when processing tiny PCB defects. By introducing a weight adaptation mechanism to adjust the fusion weight corresponding to the scale, the calculation of redundant information can be reduced, significantly improving the defect detection efficiency of the defect detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A first flow chart of a circuit board defect detection method provided by an embodiment of the present invention;

[0020] Figure 2 A schematic structural diagram of a circuit board defect detection device provided by an embodiment of the present invention;

[0021] Figure 3 A schematic structural diagram of a circuit board defect detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention provides a circuit board defect detection method, apparatus, device and storage medium. In the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of a circuit board defect detection method according to an embodiment of the present invention includes:

[0024] 101. Acquire an image to be detected and perform preprocessing to obtain a preprocessed image;

[0025] 102. Input the preprocessed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence;

[0026] 103. Confirm the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction on the preprocessed image to obtain multiple feature maps of different scales;

[0027] 104. Determine a fusion weight corresponding to the scale, and fuse the feature maps of multiple scales using a feature fusion network based on the fusion weight to obtain a fused feature map;

[0028] 105. Use the detection head to perform defect detection on the fused feature map to obtain the defect detection result.

[0029] This application discloses a circuit board defect detection method, which aims to solve the problems of high computational complexity, poor real-time performance, and insensitivity to small target detection in existing PCB defect detection methods; by introducing a dynamic convolution selection mechanism to adjust the convolution weights of the extraction network, it ensures that the defect detection model has high detection accuracy when processing tiny defects in PCBs; by introducing a weight adaptation mechanism to adjust the fusion weight corresponding to the scale, it can reduce the calculation of redundant information, significantly reduce the number of model parameters and calculation amount, and greatly improve the defect detection efficiency of the defect detection model.

[0030] In this embodiment, the pre-built defect detection model is improved based on the YOLOv8 model structure.

[0031] The YOLOv8 model consists of three main components: a feature extraction network (Backbone), a feature fusion network (Neck), and a detection head (Head). The convolution kernel of the first convolutional layer in the Backbone was changed from 6×6 to 3×3. All C3 modules were replaced with C2f modules, adding more skip-layer connections and additional split operations. The Neck of the model adopts the PANet architecture, and the Backbone finally adds the SPPF module to achieve 32x downsampling. The Backbone uses the 8x, 16x, and 32x downsampled outputs as input to the PANet architecture. After upsampling and channel fusion, the three PANet output branches are finally fed into the Detection Head for loss calculation or result resolution. Unlike FPN, PANet is a bidirectional network with a bottom-up path, making it easier for bottom-level information to be transmitted to the top level. The Head of the model directly replaces the coupling head with a decoupling head structure similar to Yolox, separating the regression and prediction branches. The regression branch uses the integral form representation proposed in the Distributed Focal Loss strategy.

[0032] Compared with the existing YOLOv8 model, the defect detection model disclosed in this application first designs an enhanced multi-scale mixed convolution (EMSMC) in the backbone part of the model to replace the original standard convolution layer for feature extraction. EMSMC combines convolution kernels of different sizes (such as 3x3, 5x5, and 7x7) to simultaneously extract multi-scale features. It also adaptively selects the optimal convolution kernel through a dynamic convolution selection mechanism, reducing redundant calculations and improving the detection capability of targets of different scales. It performs better in processing multi-scale and diverse targets such as PCB defects. Secondly, in the neck part of the model, an adaptive feature fusion network (AFFN) is designed to fuse multi-scale features. AFFN dynamically assigns weights to feature maps of different scales by introducing global pooling and adaptive weight adjustment mechanisms. Compared with the traditional fixed weighted fusion method, AFFN can automatically adjust weights according to the importance of the input feature map, improve fusion accuracy, reduce invalid information, and ensure accurate capture of important features in PCB defect detection.

[0033] A second embodiment of the circuit board defect detection method according to the present invention includes:

[0034] 201. Acquire an image to be detected, perform denoising on the image to be detected using a median filter method, and perform grayscale conversion on the denoised image to be detected to obtain a grayscale image;

[0035] In this embodiment, the image to be detected is denoised using the median filtering method. Median filtering is a nonlinear signal processing technique that removes noise by replacing the value of each pixel in the image with the median of the pixel values ​​in the neighborhood. It can well preserve the edge information of the image while removing salt and pepper noise. The color image is then converted into a grayscale image. The grayscale image only contains brightness information, not color information, which greatly simplifies the image data and reduces the complexity of processing. At the same time, it preserves the structural characteristics of the image, providing convenience for subsequent image analysis.

[0036] 202. Performing contrast enhancement processing on the grayscale image using a sharpening filter to obtain an enhanced image;

[0037] In this embodiment, a sharpening filter is used to perform contrast enhancement processing on the grayscale image; the sharpening filter makes the image details clearer by enhancing the contrast of the edges of objects in the image, thereby improving the image recognizability. The clear edge contours help to more accurately identify the characteristics of the circuit board.

[0038] 203. Using an edge detector to identify the edge contour of the circuit board in the enhanced image, and using a threshold segmentation method to segment the circuit board and the background to obtain a preprocessed image;

[0039] In this embodiment, an edge detector is used to identify the edge contour of the circuit board in the enhanced image. The edge detector can detect areas in the image where brightness changes dramatically, which generally correspond to the edges of objects. By accurately identifying the edge of the circuit board, a foundation is laid for image segmentation. Furthermore, the circuit board is segmented from the background using a threshold segmentation method to obtain a preprocessed image. Threshold segmentation is a segmentation method based on image grayscale values. By setting one or more thresholds, the pixels in the image are divided into foreground and background parts. By separating the circuit board from the background, clear image data is provided for subsequent feature extraction and recognition.

[0040] In this embodiment, the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the convolution kernel size of each convolution layer is different, and each convolution layer corresponds to a convolution weight; the third embodiment of the circuit board defect detection method includes:

[0041] 301. Perform a global average pooling operation on the input preprocessed image to obtain first global context information;

[0042] 302. In the first fully connected layer, a Sigmoid activation function is used to generate a plurality of convolution weights corresponding to the plurality of convolution layers according to the first global context information.

[0043] In this embodiment, by introducing a dynamic convolution selection mechanism, the defect detection model can dynamically adjust the usage weight of the convolution kernel according to the different input feature maps, extract features in an optimal manner, reduce unnecessary calculations, and improve the efficiency of feature extraction.

[0044] Specifically, through the global average pooling operation, the adaptive weight α is generated k , so that the defect detection model can selectively use the results of different convolution kernels; first, the input feature map F in Perform a global average pooling operation to obtain the first global context information at the channel level:

[0045] g=GAP(F in )∈RC in (1)

[0046] Where g is a channel vector representing the global average of each channel; next, the first fully connected layer of the two-layer structure is used, and the convolution weights of different convolution layers are generated by the Sigmoid activation function:

[0047] α k =σ(W 2,k ReLU(W 1,k ·g)),k∈{3,5,7} (2)

[0048] Where: W 1,k and W 2,k is the corresponding weight generated for the fully connected layer parameters of each convolutional layer; σ(·) is the Sigmoid activation function, which is used to compress the output to the range of (0, 1); α k is the convolution kernel k×k, that is, the convolution weight corresponding to the convolution layer.

[0049] 303. In the convolution module, each convolution layer performs convolution operations on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map;

[0050] In this embodiment, the EMSMC convolution module includes multiple different convolution layers, and the convolution kernel size of each convolution layer is different (such as 3x3, 5x5, 7x7). Each convolution layer performs convolution operations in parallel to extract features of different scales. Specifically, for the input feature map, that is, the preprocessed image, Among them C in is the number of input channels, H×W is the spatial size of the input feature map, and the multi-scale convolution operation is defined as:

[0051] F k =Conv k (F in ),k∈{3,5,7} (3)

[0052] Among them: Conv k Indicates that the convolution operation is performed using a k×k convolution kernel, F k It is a feature map generated by the k×k convolution kernel.

[0053] 304. Integrate the feature maps output by multiple convolution modules to obtain multiple feature maps of different scales;

[0054] Multiple feature maps of different scales are represented as:

[0055] F3=Conv3(F in ),F5=Conv5(F in ),F7=Conv7(F in ) (4)

[0056] Feature maps of different scales represent different levels of information in the preprocessed image, which can include low-level features: capturing information about smaller targets; mid-level features, capturing information about medium-sized targets, and high-level features, capturing global information and large target features.

[0057] A third embodiment of the circuit board defect detection method according to the present invention includes:

[0058] 401. In the feature fusion network, a global average pooling operation is performed on feature maps of different scales to obtain second global context information;

[0059] 402. Using the second fully connected layer included in the feature fusion network, obtain a fusion weight corresponding to the scale based on the second global context information and the ReLU activation function;

[0060] 403. Based on the fusion weights corresponding to the scales, weighted sums are performed on the feature maps of multiple different scales to obtain a fused feature map.

[0061] In this embodiment, in the feature fusion network, adaptive weight adjustment is performed through a weight learning module. Specifically, for each feature map, a global average pooling operation is first performed to compress the feature map from the spatial dimension into a global feature vector to obtain the second global context information of the channel. The formula is as follows:

[0062]

[0063] Among them, X c (i, j) is the value of position (i, j) on channel c of the feature map, and H×W is the width and height of the feature map. Then, through a second fully connected layer, the feature vector after global average pooling is passed to the activation function to generate the fusion weights corresponding to the feature maps of each scale. The formula is as follows:

[0064] Wc =σ(W2·ReLU(W1·g c )) (6)

[0065] Among them, W1 and W2 are the weight matrices of the fully connected layer, and σ(·) is the Sigmoid activation function used to generate the fusion weight W c , ensuring that the fusion weight is between (0,1); the feature fusion network combines spatial and channel attention mechanisms to dynamically adjust the fusion weights of different channels and spatial positions in feature maps of different scales, ensuring that the important information in each feature map is focused on and irrelevant information is suppressed; finally, the feature maps after weight adjustment are weighted fused:

[0066]

[0067] Among them, F l are feature maps from different scales, W c is the learned fusion weight; the result of feature fusion will be output to the downstream module for use by the detection head.

[0068] A fourth embodiment of the circuit board defect detection method according to the present invention includes:

[0069] 501. Acquire pruning information, where the pruning information includes multiple pruning ratios, and the multiple pruning ratios correspond one-to-one to each layer included in the detection head;

[0070] In this embodiment, the pruning information is pre-set according to a preset defect detection requirement.

[0071] 502. Using a detection head to perform defect detection on the fused features, during the defect detection process, pruning is performed on each layer included in the detection head according to the pruning information;

[0072] 503. Obtain the defect detection result output by the detection head.

[0073] In this embodiment, a hierarchical pruning technology is introduced into the detection head. Specifically, based on the correspondence between the pruning ratio and each layer of the detection head, the detection head can perform targeted pruning processing on each layer of the detection head according to different pruning ratios to reduce unnecessary channels and convolution kernels, ensuring that each detection layer only retains important features. This processing method not only improves the detection efficiency, but also can flexibly adjust the pruning ratio according to the specific requirements of the detection task, thereby further optimizing the defect detection speed and resource consumption while ensuring the detection accuracy.

[0074] A fifth embodiment of the circuit board defect detection method according to the present invention includes:

[0075] 601. When building a defect detection model, select the YOLOv8 model that can be used for circuit board defect detection as the teacher model, and obtain the soft label feedback from the teacher model;

[0076] 602. Obtain hard labels included in a data set for training a defect detection model;

[0077] 603. Setting a distillation loss function of the defect detection model based on the soft label, and setting a task loss function of the defect detection model based on the hard label.

[0078] When pre-building the defect detection model, the knowledge distillation method is introduced to optimize the model's detection accuracy. In the process of knowledge distillation, soft labels and hard labels play a crucial role. The soft labels come from the probability distribution information generated by the output layer of the teacher model. It not only contains the confidence of the target category, but also covers the confidence of other categories. This label containing "soft information" can help the student model, that is, the defect detection model, capture more subtle and rich feature representations, thereby gaining a deeper understanding during the learning process. The hard labels come directly from the original dataset used to train the defect detection model. They are real and clear guiding information for the defect detection task.

[0079] When designing the loss function, the loss function of the student model consists of two parts: task loss and distillation loss. The task loss ensures that the student model can correctly learn the supervision information provided by the hard labels, while the distillation loss uses soft labels to guide the student model to imitate the output of the teacher model. Through this loss function design that combines soft labels and hard labels, the student model, that is, the defect detection model, can not only learn the direct information of the data, but also inherit the advanced features and decision logic of the teacher model, that is, the YOLOv8 model used for circuit board defect detection, so that the detection accuracy is close to or even reaches the level of the teacher model, ensuring that the pruned lightweight defect detection model still has the high-precision feature extraction capability of the teacher model, thereby improving the inference speed and resource utilization efficiency.

[0080] The circuit board defect detection method disclosed in this application has significant advantages in PCB defect detection tasks. Through designs such as the enhanced multi-scale hybrid convolution module (EMSMC) and the adaptive feature fusion network (AFFN), the defect detection model of the present invention significantly reduces the number of model parameters and computational complexity while maintaining high accuracy, and improves the inference speed. This makes the defect detection model of the present invention more suitable for real-time PCB defect detection in industrial production, especially on resource-constrained embedded devices.

[0081] The following experiments illustrate the advantages of the defect detection model disclosed in this application over existing detection models:

[0082] Take a test dataset, which contains 1386 images and covers 6 different types of defects, including missing holes, mouse bites, open circuits, short circuits, strays, and pseudo copper. 693 images suitable for the detection task were selected from the test dataset and randomly divided into training, validation, and test sets in a ratio of 8:1:1. The indicators to be evaluated in the experiment include mAP, precision, recall, and average accuracy. Among them, mAP is a common indicator for measuring model detection performance. It is used to measure the average accuracy of the model on multiple categories and considers the degree of overlap between the predicted bounding box and the true bounding box. Average accuracy is a measure of the average accuracy of all categories. The formulas for precision (P), recall (R), average accuracy (AP), and mAP are as follows:

[0083]

[0084] Among them, TP is the sample that is actually positive and correctly predicted as positive, FP is the sample that is actually negative but wrongly predicted as positive, FN is the sample that is actually positive but wrongly predicted as negative, C is the number of categories, AP is i is the average precision of the i-th class;

[0085] The comparison models introduced include:

[0086] YOLOv8n: As a lightweight model in the YOLOv8 series, it has fewer parameters and faster inference speed;

[0087] YOLOv5n: A lightweight YOLOv5 model, commonly used in environments with low computing resources.

[0088] EfficientDet-D0: An optimized lightweight object detection model, commonly used for efficient detection tasks on low-power devices;

[0089] The experimental results are shown in Table 1 below, where FLOPs is used to measure the computational complexity of the model;

[0090] Table 1 PCB defect detection comparison experiment

[0091]

[0092] The experimental results are analyzed as follows:

[0093] In terms of model parameter volume, the defect detection model of the present invention has only 12.5M parameters, which is the least among all the compared models. This is due to the application of enhanced multi-scale hybrid convolution module (EMSMC) and hierarchical pruning technology, which effectively reduces unnecessary convolution kernels and feature channels. The parameter volumes of the YOLOv8n and EfficientDet-D0 models are 20.1M and 15.9M respectively, which are significantly increased compared to the model of the present invention, resulting in higher computational costs.

[0094] For FLOPs (computational complexity), LightYOLOv8's FLOPs is 2.8G, which is significantly lower than YOLOv8n's 4.5G and EfficientDet-D0's 3.8G. This shows that the defect detection model of the present invention has lower computational complexity and is suitable for resource-constrained embedded devices or low-power environments.

[0095] In terms of detection accuracy (mAP@0.5), the defect detection model of the present invention achieved a detection accuracy of 92.5%, which is better than the 90.7% and 89.3% of YOLOv8n and YOLOv5n, and the 88.6% of EfficientDet-D0; that is, the defect detection model of the present invention has significant advantages in small target detection, which is due to the multi-scale feature extraction capability of the enhanced multi-scale hybrid convolution module (EMSMC) and the weighted fusion strategy of the adaptive feature fusion network (AFFN).

[0096] Regarding inference speed: The defect detection model of the present invention has the fastest inference speed, taking only 12ms per image on average. In comparison, the inference speeds of YOLOv8n and YOLOv5n are 15ms and 18ms respectively, while the inference speed of EfficientDet-D0 is slower, at 25ms. This speed advantage mainly comes from the combination of the EMSMC module and pruning technology, which reduces redundant calculations of the model.

[0097] Experiments show that the defect detection model of the present invention performs well in detecting six types of defects, especially for complex defects such as open circuits, short circuits, and false copper, with significantly improved detection accuracy. In comparison, YOLOv5n and EfficientDet-D0 are not as accurate as the model of the present invention in detecting smaller defects, and have a high missed detection rate in some scenarios.

[0098] The above describes the circuit board defect detection method according to the embodiment of the present invention. The following describes the circuit board defect detection device according to the embodiment of the present invention. Figure 2 , an embodiment of a circuit board defect detection device in an embodiment of the present invention includes:

[0099] The processing module 701 is used to obtain the image to be detected and perform preprocessing to obtain a preprocessed image;

[0100] An input module 702 is used to input the pre-processed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence;

[0101] A feature extraction module 703 is used to determine the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction on the preprocessed image to obtain multiple feature maps of different scales;

[0102] A fusion module 704 is configured to determine a fusion weight corresponding to each scale, and based on the fusion weight, use a feature fusion network to fuse feature maps of multiple scales to obtain a fused feature map.

[0103] Detection module 705 is used to use the detection head to perform defect detection on the fusion feature map to obtain the defect detection result

[0104] Based on the same idea as the method in the above embodiment, the device provided in this application can implement the method in the above embodiment.

[0105] above Figure 2 The circuit board defect detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The circuit board defect detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0106] Figure 3 : is a structural diagram of a circuit board defect detection device provided by an embodiment of the present invention. The circuit board defect detection device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 810 (for example, one or more processors) and a memory 820, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 833 or data 832. Among them, the memory 820 and the storage medium 830 can be temporary storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the circuit board defect detection device 800. Furthermore, the processor 810 can be configured to communicate with the storage medium 830 and execute a series of instruction operations in the storage medium 830 on the circuit board defect detection device 800 to implement the steps of the circuit board defect detection method provided by the above-mentioned method embodiments.

[0107] The circuit board defect detection device 800 may further include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input and output interfaces 860, and / or one or more operating systems 831, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be understood by those skilled in the art that Figure 3 The circuit board defect detection device structure shown does not constitute a limitation on the circuit board defect detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0108] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the circuit board defect detection method.

[0109] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0110] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc., various media that can store program code.

[0111] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A circuit board defect detection method, characterized in that: include: Acquire the image to be detected and perform preprocessing to obtain a preprocessed image; When constructing a defect detection model, a YOLOv8 model that can be used for circuit board defect detection is selected as a teacher model, and soft labels fed back by the teacher model are obtained; hard labels included in a data set used to train the defect detection model are obtained; a distillation loss function of the defect detection model is set based on the soft labels, and a task loss function of the defect detection model is set based on the hard labels; a preprocessed image is input into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence, wherein the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the convolution kernel of each convolution layer has a different size, and each convolution layer corresponds to a convolution weight; Determining the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, using the extraction network to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales. Specifically, performing a global average pooling operation on the input preprocessed image to obtain first global context information; In the first fully connected layer, a sigmoid activation function is used to generate multiple convolution weights corresponding to multiple convolution layers based on the first global context information. In the convolution module, each convolution layer performs a convolution operation on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map. The feature maps output by multiple convolution modules are integrated to obtain multiple feature maps of different scales. Determine a fusion weight corresponding to the scale, and based on the fusion weight, use a feature fusion network to fuse multiple feature maps of different scales to obtain a fused feature map. Specifically, in the feature fusion network, perform a global average pooling operation on the feature maps of different scales to obtain the second global context information; Using the second fully connected layer included in the feature fusion network, obtaining a fusion weight corresponding to the scale based on the second global context information and the ReLU activation function; Based on the fusion weights corresponding to the scales, weighted summation is performed on feature maps of multiple different scales to obtain a fused feature map; The detection head is used to perform defect detection on the fused feature map to obtain the defect detection result.

2. The circuit board defect detection method according to claim 1, wherein: The step of obtaining the image to be detected and preprocessing it to obtain a preprocessed image includes: Acquire an image to be detected, perform denoising on the image to be detected using a median filter method, and perform grayscale conversion on the denoised image to obtain a grayscale image; The sharpening filter is used to perform contrast enhancement on the grayscale image to obtain an enhanced image; An edge detector is used to identify the edge contour of the circuit board in the enhanced image, and the circuit board and background are segmented using a threshold segmentation method to obtain a preprocessed image.

3. The circuit board defect detection method according to claim 1, wherein: The defect detection is performed on the fused feature map using a detection head to obtain a defect detection result, including: Acquire pruning information, the pruning information including a plurality of pruning ratios, the plurality of pruning ratios corresponding one-to-one to each layer included in the detection head; Using a detection head to perform defect detection on the fused features, during the defect detection process, pruning each layer included in the detection head is performed according to the pruning information; Obtain the defect detection results output by the detection head.

4. A circuit board defect detection device, characterized in that: include: A processing module is used to obtain the image to be detected and perform preprocessing to obtain a preprocessed image; An input module is configured to select a YOLOv8 model that can be used for circuit board defect detection as a teacher model when constructing a defect detection model, and obtain soft labels fed back by the teacher model; obtain hard labels included in a data set used to train the defect detection model; set a distillation loss function for the defect detection model based on the soft labels, and set a task loss function for the defect detection model based on the hard labels; and input a preprocessed image into a pre-constructed defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence, wherein the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the convolution kernel of each convolution layer has a different size, and each convolution layer corresponds to a convolution weight; A feature extraction module is used to determine the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales. Specifically, a global average pooling operation is performed on the input preprocessed image to obtain first global context information; In the first fully connected layer, a sigmoid activation function is used to generate multiple convolution weights corresponding to multiple convolution layers based on the first global context information. In the convolution module, each convolution layer performs a convolution operation on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map. The feature maps output by multiple convolution modules are integrated to obtain multiple feature maps of different scales. A fusion module is used to determine a fusion weight corresponding to the scale, and based on the fusion weight, use a feature fusion network to fuse multiple feature maps of different scales to obtain a fused feature map. Specifically, in the feature fusion network, a global average pooling operation is performed on the feature maps of different scales to obtain the second global context information; Using the second fully connected layer included in the feature fusion network, obtaining a fusion weight corresponding to the scale based on the second global context information and the ReLU activation function; Based on the fusion weights corresponding to the scales, weighted summation is performed on feature maps of multiple different scales to obtain a fused feature map; The detection module is used to use the detection head to perform defect detection on the fused feature map to obtain the defect detection result.

5. A circuit board defect detection device, characterized in that: The circuit board defect detection device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors calls the instructions in the memory to enable the circuit board defect detection device to perform each step of the circuit board defect detection method according to any one of claims 1 to 3.

6. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the circuit board defect detection method as described in any one of claims 1 to 3 are implemented.

Images