Circuit board defect detection method, device and equipment and storage medium

Abstract

The invention relates to the technical field of image recognition, in particular to a circuit board defect detection method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-detected image, and carrying out the preprocessing of the to-be-detected image, and obtaining a preprocessed image; inputting the preprocessed image into a pre-constructed defect detection model, performing feature extraction processing on the preprocessed image based on a pre-confirmed convolution weight, performing fusion processing on a plurality of feature maps of different scales based on a pre-confirmed fusion weight, and performing defect detection on the fused feature map by adopting a detection head to obtain a defect detection result; the defect detection method disclosed by the invention aims to solve the problems of high calculation complexity, poor real-time performance and insensitivity to small target detection in the existing PCB defect detection method. The convolution weight is adjusted by introducing a dynamic convolution selection mechanism, and it is ensured that the model has high precision when processing small defects; and the fusion weight is adjusted by introducing a weight adaptive mechanism, so that the calculation of redundant information can be reduced, and the detection efficiency is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a circuit board defect detection method, device, equipment and storage medium. Background Art

[0002] Printed circuit boards (PCBs) are the core of electronic devices, and their quality directly determines the performance and service life of the devices. Defects on PCBs, including short circuits, open circuits, and welding problems, may cause circuit failures, thus affecting the stable operation and safety of the equipment.

[0003] Traditionally, PCB defect detection relies on manual inspection or rule-based image processing technology. These methods have the problems of low detection efficiency and unstable detection quality. Especially when dealing with complex PCB structures and tiny defects, it is difficult to ensure the accuracy and efficiency of detection. Although traditional methods based on image processing have achieved automation to a certain extent, their stability is insufficient. Especially when facing diverse and complex PCB designs, they are prone to misjudgment or omission, which reduces the accuracy of defect detection.

[0004] With the vigorous development of deep learning technology, object detection methods based on convolutional neural networks (CNN) have begun to emerge in the field of PCB defect detection; in particular, the YOLO series of models have achieved remarkable achievements in the field of object detection with their end-to-end fast detection capabilities; the YOLO model can predict the location and category of multiple targets in a single forward propagation. Compared with traditional sliding window or region-based detection methods, the YOLO model has obvious advantages in detection efficiency.

[0005] However, the YOLO model, especially the newer YOLOv8 version, has improved detection accuracy and speed, but its model structure is complex and has many parameters, that is, the model has a large demand for hardware resources, resulting in expensive computational costs, which makes them difficult to popularize in embedded devices or industrial applications with extremely high real-time requirements, that is, limits their application in resource-constrained environments; in addition, the existing YOLO model is not sensitive enough to the detection of small targets and subtle defects, especially in complex backgrounds or occlusions, the model is often prone to insufficient robustness, which reduces the detection accuracy.

[0006] It can be seen that the existing technology still needs to be improved and enhanced. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the purpose of the present invention is to provide a circuit board defect detection method, aiming to solve the problems of high computational complexity, poor real-time performance and insensitivity to small target detection in the existing PCB defect detection method.

[0008] The first aspect of the present invention provides a circuit board defect detection method, comprising: acquiring an image to be detected and preprocessing it to obtain a preprocessed image; inputting the preprocessed image into a pre-constructed defect detection model, wherein the defect detection model comprises an extraction network, a feature fusion network and a detection head connected in sequence; confirming the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, using the extraction network to perform feature extraction processing on the preprocessed image to obtain feature maps of multiple different scales; confirming the fusion weight corresponding to the scale, and based on the fusion weight, using the feature fusion network to perform fusion processing on the feature maps of multiple different scales to obtain a fused feature map; using the detection head to perform defect detection on the fused feature map to obtain a defect detection result.

[0009] Optionally, in a first implementation method of the first aspect of the present invention, the acquiring of the image to be detected and preprocessing to obtain a preprocessed image includes: acquiring the image to be detected, denoising the image to be detected using a median filtering method, and performing grayscale conversion on the denoised image to be detected to obtain a grayscale image; performing contrast enhancement processing on the grayscale image using a sharpening filter to obtain an enhanced image; using an edge detector to identify the edge contour of the circuit board in the enhanced image, and segmenting the circuit board and the background by a threshold segmentation method to obtain a preprocessed image.

[0010] Optionally, in a second implementation of the first aspect of the present invention, the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the convolution kernel of each convolution layer has a different size, and each convolution layer corresponds to a convolution weight.

[0011] Optionally, in a third implementation method of the first aspect of the present invention, the convolution weights of each convolution module included in the confirmation extraction network are determined, and based on the convolution weights, the extraction network is used to perform feature extraction processing on the preprocessed image to obtain feature maps of multiple scales, including: performing a global average pooling operation on the input preprocessed image to obtain first global context information; in the first fully connected layer, according to the first global context information, a Sigmoid activation function is used to generate multiple convolution weights corresponding to multiple convolution layers respectively; in the convolution module, each convolution layer performs a convolution operation on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map; and the feature maps output by multiple convolution modules are integrated to obtain feature maps of multiple scales.

[0012] Optionally, in a fourth implementation manner of the first aspect of the present invention, the fusion weight corresponding to the scale is confirmed, and based on the fusion weight, a feature fusion network is used to fuse the feature maps of multiple scales to obtain a fused feature map, including: in the feature fusion network, global average pooling operations are performed on the feature maps of different scales to obtain second global context information; a second fully connected layer included in the feature fusion network is used to obtain the fusion weight corresponding to the scale based on the second global context information and the ReLU activation function; based on the fusion weight corresponding to the scale, weighted summation is performed on the feature maps of multiple scales to obtain a fused feature map.

[0013] Optionally, in a fifth implementation of the first aspect of the present invention, the detection head is used to perform defect detection on the fused feature map to obtain a defect detection result, including: obtaining pruning information, the pruning information including multiple pruning ratios, and the multiple pruning ratios correspond one-to-one to each layer included in the detection head; the detection head is used to perform defect detection on the fused feature, and during the defect detection process, pruning each layer included in the detection head is performed according to the pruning information; and the defect detection result output by the detection head is obtained.

[0014] Optionally, in a sixth implementation manner of the first aspect of the present invention, the process of inputting the preprocessed image into a prebuilt defect detection model includes: when building the defect detection model, selecting a YOLOv8 model that can be used for circuit board defect detection as a teacher model, and obtaining soft labels fed back by the teacher model; obtaining hard labels included in a data set for training the defect detection model; setting a distillation loss function of the defect detection model based on the soft labels, and setting a task loss function of the defect detection model based on the hard labels.

[0015] The second aspect of the present invention provides a circuit board defect detection device, comprising: a processing module, used to acquire an image to be detected and preprocess it to obtain a preprocessed image; an input module, used to input the preprocessed image into a pre-constructed defect detection model, the defect detection model comprising an extraction network, a feature fusion network and a detection head connected in sequence; a feature extraction module, used to confirm the convolution weights of each convolution module included in the extraction network, based on the convolution weights, using the extraction network to perform feature extraction processing on the preprocessed image to obtain feature maps of multiple different scales; a fusion module, used to confirm the fusion weight corresponding to the scale, based on the fusion weight, using the feature fusion network to perform fusion processing on the feature maps of multiple different scales to obtain a fused feature map; a detection module, used to use the detection head to perform defect detection on the fused feature map to obtain a defect detection result.

[0016] The third aspect of the present invention provides a circuit board defect detection device, which includes: a memory and at least one processor, wherein instructions are stored in the memory; at least one of the processors calls the instructions in the memory so that the circuit board defect detection device performs each step of any one of the above-mentioned circuit board defect detection methods.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium having instructions stored thereon, and when the instructions are executed by a processor, the various steps of any of the above-mentioned circuit board defect detection methods are implemented.

[0018] The technical solution of the present invention aims to solve the problems of high computational complexity, poor real-time performance, and insensitivity to small target detection in existing PCB defect detection methods; by introducing a dynamic convolution selection mechanism to adjust the convolution weights of the extraction network, it is ensured that the defect detection model has high detection accuracy when processing tiny defects of the PCB; by introducing a weight adaptive mechanism to adjust the fusion weight corresponding to the scale, the calculation of redundant information can be reduced, and the defect detection efficiency of the defect detection model can be greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A first flow chart of a circuit board defect detection method provided by an embodiment of the present invention;

[0020] Figure 2 A schematic diagram of the structure of a circuit board defect detection device provided by an embodiment of the present invention;

[0021] Figure 3 A schematic diagram of the structure of a circuit board defect detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention provides a circuit board defect detection method, device, equipment and storage medium. In the present invention, the terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] For ease of understanding, the specific process of the embodiment of the present invention is described below. Figure 1 , an embodiment of the circuit board defect detection method in the embodiment of the present invention includes:

[0024] 101. Acquire an image to be detected and perform preprocessing to obtain a preprocessed image;

[0025] 102. Input the preprocessed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence;

[0026] 103. Confirm the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales;

[0027] 104. Determine a fusion weight corresponding to the scale, and based on the fusion weight, use a feature fusion network to fuse feature maps of multiple scales to obtain a fused feature map;

[0028] 105. Use a detection head to perform defect detection on the fused feature map to obtain a defect detection result.

[0029] The present application discloses a circuit board defect detection method, which aims to solve the problems of high computational complexity, poor real-time performance, and insensitivity to small target detection in existing PCB defect detection methods; by introducing a dynamic convolution selection mechanism to adjust the convolution weights of the extraction network, it is ensured that the defect detection model has high detection accuracy when processing tiny defects of the PCB; by introducing a weight adaptive mechanism to adjust the fusion weights corresponding to the scale, the calculation of redundant information can be reduced, the number of parameters and the amount of calculation of the model can be significantly reduced, and the defect detection efficiency of the defect detection model is greatly improved.

[0030] In this embodiment, the pre-built defect detection model is improved based on the YOLOv8 model structure.

[0031] The YOLOv8 model is mainly composed of three parts: feature extraction network (Backbone), feature fusion network (Neck) and detection head (Head); the convolution kernel of the first convolution layer of the Backbone of the model is changed from 6×6 to 3×3; all C3 modules are changed to C2f modules, with more skip-layer connections and additional Split operations; the Neck part of the model adopts the PANet structure, and the Backbone finally adds the SPPF module, which is used to achieve 32-fold downsampling; Backbone uses the output of 8-fold downsampling, 16-fold downsampling and 32-fold downsampling as the input of the PANet structure, and after upsampling and channel fusion, the three output branches of PANet are finally sent to the Detect head for Loss calculation or result solution; Unlike FPN, PANet is a bidirectional path network, which introduces a bottom-up path, making it easier for the bottom-level information to be transmitted to the top layer; the Head part of the model directly changes the coupling head to a decoupling head structure similar to Yolox, separates the regression branch and the prediction branch, and uses the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch.

[0032] Compared with the existing YOLOv8 model, the defect detection model disclosed in the present application firstly designs an enhanced multi-scale mixed convolution (EMSMC) in the Backbone part of the model to replace the original standard convolution layer for extracting features; EMSMC combines convolution kernels of different sizes (such as 3x3, 5x5, 7x7), can extract multi-scale features at the same time, and adaptively selects the optimal convolution kernel through a dynamic convolution selection mechanism, reduces redundant calculations, and improves the detection capability of targets of different scales, especially when dealing with multi-scale and diversified targets such as PCB defects; secondly, in the Neck part of the model, an adaptive feature fusion network (AFFN) is designed to fuse multi-scale features. AFFN dynamically allocates weights to feature maps of different scales by introducing global pooling and weight adaptive adjustment mechanism; compared with the traditional fixed weighted fusion method, AFFN can automatically adjust the weight according to the importance of the input feature map, improve the fusion accuracy, reduce invalid information, and ensure the accurate capture of important features in PCB defect detection.

[0033] A second embodiment of the circuit board defect detection method in the embodiment of the present invention includes:

[0034] 201. Acquire an image to be detected, perform denoising on the image to be detected by using a median filtering method, and perform grayscale conversion on the denoised image to be detected to obtain a grayscale image;

[0035] In this embodiment, the median filtering method is used to denoise the image to be detected; median filtering is a nonlinear signal processing technology, which removes noise by replacing each pixel value in the image with the median of the pixel values ​​in the neighborhood, and can well preserve the edge information of the image while removing salt and pepper noise; then the color image is converted into a grayscale image; the grayscale image only contains brightness information, no color information, which greatly simplifies the image data, reduces the complexity of processing, and at the same time preserves the structural characteristics of the image, providing convenience for subsequent image analysis.

[0036] 202. Performing contrast enhancement processing on the grayscale image using a sharpening filter to obtain an enhanced image;

[0037] In this embodiment, a sharpening filter is used to perform contrast enhancement processing on the grayscale image; the sharpening filter makes the details of the image clearer by enhancing the contrast of the edges of objects in the image, thereby improving the recognizability of the image, and the clear edge contour helps to more accurately identify the characteristics of the circuit board.

[0038] 203. Using an edge detector to identify the edge contour of the circuit board in the enhanced image, and using a threshold segmentation method to segment the circuit board and the background to obtain a preprocessed image;

[0039] In this embodiment, an edge detector is used to identify the edge contour of the circuit board in the enhanced image; the edge detector can detect areas in the image where the brightness changes dramatically, which usually correspond to the edges of the object; by accurately identifying the edge of the circuit board, a foundation is laid for image segmentation; further, the circuit board is segmented from the background by a threshold segmentation method to obtain a preprocessed image; threshold segmentation is a segmentation method based on image grayscale values, which divides the pixels in the image into foreground and background by setting one or more thresholds; by separating the circuit board from the background, clear image data is provided for subsequent feature extraction and recognition.

[0040] In this embodiment, the extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the size of the convolution kernel of each convolution layer is different, and each convolution layer corresponds to a convolution weight; the third embodiment of the circuit board defect detection method includes:

[0041] 301. Perform a global average pooling operation on the input preprocessed image to obtain first global context information;

[0042] 302. In the first fully connected layer, a Sigmoid activation function is used to generate a plurality of convolution weights corresponding to the plurality of convolution layers respectively according to the first global context information;

[0043] In this embodiment, by introducing a dynamic convolution selection mechanism, the defect detection model can dynamically adjust the usage weight of the convolution kernel according to the different input feature maps, extract features in an optimal manner, reduce unnecessary calculations, and improve the efficiency of feature extraction.

[0044] Specifically, through the global average pooling operation, an adaptive weight α is generated k , so that the defect detection model can selectively use the results of different convolution kernels; first, the input feature map F in Perform a global average pooling operation to obtain the first global context information at the channel level:

[0045] g=GAP(F in )∈RC in (1)

[0046] Where g is a channel vector representing the global average of each channel; Next, the first fully connected layer of the two-layer structure is used, and the convolution weights of different convolutional layers are generated through the Sigmoid activation function:

[0047] α k =σ(W 2,k ReLU(W 1,k ·g)),k∈{3,5,7} (2)

[0048] Where: W 1,k and W 2,k is the corresponding weight generated for the fully connected layer parameters of each convolutional layer; σ(·) is the Sigmoid activation function, which is used to compress the output to the range of (0, 1); α k It is the convolution kernel k×k, that is, the convolution weight corresponding to the convolution layer.

[0049] 303. In the convolution module, each convolution layer performs convolution operation on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map;

[0050] In this embodiment, the EMSMC convolution module includes a plurality of different convolutional layers, and the convolution kernel size of each convolutional layer is different (such as 3x3, 5x5, 7x7). Each convolutional layer performs convolution operations in parallel to extract features of different scales. Specifically, for the input feature map, that is, the preprocessed image, Among them C in is the number of input channels, H×W is the spatial size of the input feature map, and the multi-scale convolution operation is defined as:

[0051] F k =Conv k (F in ),k∈{3,5,7} (3)

[0052] Where: Conv k Indicates that a convolution operation is performed using a k×k convolution kernel, F k It is a feature map generated by the k×k convolution kernel.

[0053] 304. Integrate the feature maps output by multiple convolution modules to obtain multiple feature maps of different scales;

[0054] Multiple feature maps of different scales are represented as:

[0055] F3=Conv3(F in ),F5=Conv5(F in ),F7=Conv7(F in ) (4)

[0056] Feature maps of different scales represent different levels of information in the preprocessed image, which may include low-level features that capture information about smaller objects, mid-level features that capture information about medium-sized objects, and high-level features that capture global information and large object features.

[0057] A third embodiment of the circuit board defect detection method in the embodiment of the present invention includes:

[0058] 401. In the feature fusion network, global average pooling operations are performed on feature maps of different scales to obtain second global context information;

[0059] 402. Using the second fully connected layer included in the feature fusion network, based on the second global context information and the ReLU activation function, obtain a fusion weight corresponding to the scale;

[0060] 403. Based on the fusion weights corresponding to the scales, weighted sums are performed on the feature maps of multiple different scales to obtain a fused feature map.

[0061] In this embodiment, in the feature fusion network, an adaptive weight adjustment is performed through a weight learning module; specifically, for each feature map, the feature map is first compressed from the spatial dimension into a global feature vector through a global average pooling operation to obtain the second global context information of the channel, and the formula is as follows:

[0062]

[0063] Among them, X c (i, j) is the value of position (i, j) on channel c of the feature map, and H×W is the width and height of the feature map. Then, through a second fully connected layer, the feature vector after global average pooling is passed to the activation function to generate the fusion weights corresponding to the feature maps of each scale. The formula is as follows:

[0064] Wc =σ(W2·ReLU(W1·g c )) (6)

[0065] Among them, W1 and W2 are the weight matrices of the fully connected layer, and σ(·) is the Sigmoid activation function used to generate the fusion weight W c , ensuring that the fusion weight is between (0,1); the feature fusion network combines the spatial and channel attention mechanisms to dynamically adjust the fusion weights of different channels and spatial positions in feature maps of different scales, ensuring that the important information in each feature map is focused on and irrelevant information is suppressed; finally, the weighted fusion of the feature maps after weight adjustment is performed:

[0066]

[0067] Among them, F l are feature maps from different scales, W c is the learned fusion weight; the result of feature fusion will be output to the downstream module for use by the detection head.

[0068] A fourth embodiment of the circuit board defect detection method in the embodiment of the present invention includes:

[0069] 501. Acquire pruning information, where the pruning information includes multiple pruning ratios, and the multiple pruning ratios correspond one-to-one to each layer included in the detection head;

[0070] In this embodiment, the pruning information is pre-set according to a preset defect detection requirement.

[0071] 502. Use a detection head to perform defect detection on the fused features. During the defect detection process, perform pruning processing on each layer included in the detection head according to the pruning information;

[0072] 503. Obtain the defect detection result output by the detection head.

[0073] In this embodiment, a hierarchical pruning technology is introduced in the detection head. Specifically, according to the correspondence between the pruning ratio and each layer of the detection head, the detection head can perform targeted pruning processing on each layer of the detection head according to different pruning ratios to reduce unnecessary channels and convolution kernels, ensuring that each detection layer only retains important features. This processing method not only improves the detection efficiency, but also can flexibly adjust the pruning ratio according to the specific needs of the detection task, thereby further optimizing the defect detection speed and resource consumption while ensuring the detection accuracy.

[0074] A fifth embodiment of the circuit board defect detection method in the embodiment of the present invention includes:

[0075] 601. When building a defect detection model, select a YOLOv8 model that can be used for circuit board defect detection as a teacher model, and obtain a soft label fed back by the teacher model;

[0076] 602. Obtain hard labels included in a data set for training a defect detection model;

[0077] 603. Setting a distillation loss function of the defect detection model based on the soft label, and setting a task loss function of the defect detection model based on the hard label.

[0078] When pre-building the defect detection model, the knowledge distillation method is introduced to optimize the detection accuracy of the model; in the process of knowledge distillation, soft labels and hard labels play a crucial role; the soft label comes from the probability distribution information generated by the output layer of the teacher model, which not only includes the confidence of the target category, but also covers the confidence of other categories. This label containing "soft information" can help the student model, that is, the defect detection model, capture more subtle and rich feature representations, thereby gaining a deeper understanding in the learning process; and the hard labels come directly from the original data set used to train the defect detection model, which are real and clear guiding information for defect detection tasks.

[0079] When designing the loss function, the loss function of the student model consists of two parts: task loss and distillation loss; the task loss part ensures that the student model can correctly learn the supervision information provided by the hard labels, while the distillation loss part uses soft labels to guide the student model to imitate the output of the teacher model; through this loss function design that combines soft labels and hard labels, the student model, that is, the defect detection model can not only learn the direct information of the data, but also inherit the teacher model, that is, the advanced features and decision logic of the YOLOv8 model for circuit board defect detection, so that the detection accuracy is close to or even reaches the level of the teacher model, ensuring that the pruned lightweight defect detection model still has the high-precision feature extraction capability of the teacher model, thereby improving the reasoning speed and resource utilization efficiency.

[0080] The circuit board defect detection method disclosed in the present application has significant advantages in PCB defect detection tasks; through the design of enhanced multi-scale hybrid convolution module (EMSMC) and adaptive feature fusion network (AFFN), the defect detection model of the present invention greatly reduces the number of model parameters and calculation complexity while maintaining high accuracy, and improves the inference speed; this makes the defect detection model of the present invention more suitable for real-time PCB defect detection in industrial production, especially on resource-constrained embedded devices.

[0081] The following experiments illustrate the advantages of the defect detection model disclosed in this application over existing detection models:

[0082] Take a test data set, which contains 1386 images, covering 6 different types of defects, including missing holes, mouse bites, open circuits, short circuits, strays and pseudo copper; select 693 images suitable for detection tasks from the test data set and randomly divide them into training set, validation set and test set in a ratio of 8:1:1; the indicators to be evaluated in the experiment include: mAP, usage precision, recall rate and average accuracy, among which mAP is a common indicator for measuring model detection performance, which is used to measure the average accuracy of the model on multiple categories and considers the overlap between the predicted bounding box and the true bounding box; and the average accuracy is a measure of the average accuracy of all categories; the formulas for usage precision (P), recall rate (R), average accuracy (AP) and mAP are as follows:

[0083]

[0084] Among them, TP is the sample that is actually positive and correctly predicted as positive, FP is the sample that is actually negative but wrongly predicted as positive, FN is the sample that is actually positive but wrongly predicted as negative, C is the number of categories, AP is i is the average precision of the i-th class;

[0085] The comparison models introduced include:

[0086] YOLOv8n: As a lightweight model in the YOLOv8 series, it has fewer parameters and faster inference speed;

[0087] YOLOv5n: A lightweight YOLOv5 model, commonly used in environments with low computing resources;

[0088] EfficientDet-D0: An optimized lightweight object detection model, commonly used for efficient detection tasks on low-power devices;

[0089] The experimental results are shown in Table 1 below, where FLOPs is used to measure the computational complexity of the model;

[0090] Table 1 PCB defect detection comparison experiment

[0091]

[0092] The experimental results are analyzed as follows:

[0093] Regarding the number of model parameters, the defect detection model of the present invention has only 12.5M parameters, which is the least among all the compared models. This is due to the application of enhanced multi-scale hybrid convolution module (EMSMC) and hierarchical pruning technology, which effectively reduces unnecessary convolution kernels and feature channels. The parameters of the YOLOv8n and EfficientDet-D0 models are 20.1M and 15.9M, respectively, which are significantly increased compared to the model of the present invention, resulting in higher computational costs.

[0094] For FLOPs (computational complexity), the FLOPs of LightYOLOv8 is 2.8G, which is significantly lower than 4.5G of YOLOv8n and 3.8G of EfficientDet-D0. This shows that the defect detection model of the present invention has lower computational complexity and is suitable for resource-constrained embedded devices or low-power environments.

[0095] For detection accuracy (mAP@0.5), the defect detection model of the present invention achieved a detection accuracy of 92.5%, which is better than 90.7% and 89.3% of YOLOv8n and YOLOv5n, and 88.6% of EfficientDet-D0; that is, the defect detection model of the present invention has significant advantages in small target detection, which is due to the multi-scale feature extraction capability of the enhanced multi-scale hybrid convolution module (EMSMC) and the weighted fusion strategy of the adaptive feature fusion network (AFFN).

[0096] Regarding reasoning speed: the defect detection model of the present invention has the fastest reasoning speed, which takes only 12ms per image on average. In comparison, the reasoning speeds of YOLOv8n and YOLOv5n are 15ms and 18ms respectively, while the reasoning speed of EfficientDet-D0 is slower, which is 25ms. This speed advantage mainly comes from the combination of the EMSMC module and pruning technology, which reduces the redundant calculation of the model.

[0097] Experiments show that the defect detection model of the present invention performs well in the detection tasks of 6 types of defects, especially for the detection tasks of complex defects such as open circuit, short circuit and false copper, its detection accuracy is significantly improved; in contrast, YOLOv5n and EfficientDet-D0 are not as accurate as the model of the present invention in detecting smaller defects, and have a high missed detection rate in some scenarios.

[0098] The circuit board defect detection method in the embodiment of the present invention is described above. The circuit board defect detection device in the embodiment of the present invention is described below. Figure 2 , an embodiment of a circuit board defect detection device in an embodiment of the present invention includes:

[0099] The processing module 701 is used to obtain the image to be detected and perform preprocessing to obtain a preprocessed image;

[0100] An input module 702, used to input the preprocessed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network and a detection head connected in sequence;

[0101] A feature extraction module 703 is used to determine the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales;

[0102] A fusion module 704 is used to determine a fusion weight corresponding to the scale, and based on the fusion weight, use a feature fusion network to fuse multiple feature maps of different scales to obtain a fused feature map;

[0103] The detection module 705 is used to use the detection head to perform defect detection on the fused feature map to obtain the defect detection result.

[0104] Based on the same idea as the method in the above embodiment, the device provided by the present application can implement the method in the above embodiment.

[0105] above Figure 2 The circuit board defect detection device in the embodiment of the present invention is described in detail from the perspective of modular functional entities, and the circuit board defect detection device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0106] Figure 3 1 is a schematic diagram of the structure of a circuit board defect detection device provided by an embodiment of the present invention. The circuit board defect detection device 800 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 810 (for example, one or more processors) and a memory 820, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 833 or data 832. Among them, the memory 820 and the storage medium 830 can be short-term storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the circuit board defect detection device 800. Furthermore, the processor 810 can be configured to communicate with the storage medium 830, and execute a series of instruction operations in the storage medium 830 on the circuit board defect detection device 800 to implement the steps of the circuit board defect detection method provided by the above-mentioned method embodiments.

[0107] The circuit board defect detection device 800 may also include one or more power supplies 840, one or more wired or wireless network interfaces 850, one or more input and output interfaces 860, and / or one or more operating systems 831, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The circuit board defect detection device structure shown does not constitute a limitation on the circuit board defect detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0108] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the circuit board defect detection method.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0110] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0111] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A circuit board defect detection method, characterized in that: include: Acquire the image to be detected and perform preprocessing to obtain a preprocessed image; Inputting the preprocessed image into a prebuilt defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network, and a detection head connected in sequence; Confirming the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, using the extraction network to perform feature extraction processing on the preprocessed image to obtain a plurality of feature maps of different scales; Determine a fusion weight corresponding to the scale, and based on the fusion weight, use a feature fusion network to fuse feature maps of multiple scales to obtain a fused feature map; The detection head is used to perform defect detection on the fused feature map to obtain the defect detection result.

2. The circuit board defect detection method according to claim 1, characterized in that: The step of acquiring the image to be detected and preprocessing it to obtain the preprocessed image includes: Acquire an image to be detected, perform denoising on the image to be detected using a median filter method, and perform grayscale conversion on the denoised image to be detected to obtain a grayscale image; The sharpening filter is used to perform contrast enhancement on the grayscale image to obtain an enhanced image; An edge detector is used to identify the edge contour of the circuit board in the enhanced image, and the circuit board and the background are segmented by a threshold segmentation method to obtain a preprocessed image.

3. The circuit board defect detection method according to claim 1, characterized in that: The extraction network includes multiple convolution modules, each convolution module includes multiple convolution layers and a first fully connected layer, the size of the convolution kernel of each convolution layer is different, and each convolution layer corresponds to a convolution weight.

4. The circuit board defect detection method according to claim 3, characterized in that: The convolution weights of the convolution modules included in the confirmation extraction network are determined, and based on the convolution weights, the extraction network is used to perform feature extraction processing on the preprocessed image to obtain a plurality of feature maps of different scales, including: Performing a global average pooling operation on the input preprocessed image to obtain first global context information; In the first fully connected layer, a Sigmoid activation function is used to generate a plurality of convolution weights corresponding to the plurality of convolution layers respectively according to the first global context information; In the convolution module, each convolution layer performs convolution operations on the preprocessed image in parallel based on its corresponding convolution weight to obtain a feature map; The feature maps output by multiple convolutional modules are integrated to obtain multiple feature maps of different scales.

5. The circuit board defect detection method according to claim 1, characterized in that: The determining of the fusion weight corresponding to the scale, and based on the fusion weight, using a feature fusion network to fuse the feature maps of multiple scales to obtain a fused feature map, include: In the feature fusion network, global average pooling operations are performed on feature maps of different scales to obtain the second global context information; Using the second fully connected layer included in the feature fusion network, obtaining a fusion weight corresponding to the scale based on the second global context information and the ReLU activation function; Based on the fusion weights corresponding to the scales, weighted sums are performed on feature maps of multiple different scales to obtain a fused feature map.

6. The circuit board defect detection method according to claim 1, characterized in that: The defect detection is performed on the fused feature map by using the detection head to obtain the defect detection result, including: Acquire pruning information, wherein the pruning information includes a plurality of pruning ratios, and the plurality of pruning ratios correspond one-to-one to each layer included in the detection head; Using a detection head to perform defect detection on the fused features, during the defect detection process, pruning is performed on each layer included in the detection head according to the pruning information; Get the defect detection results output by the detection head.

7. The circuit board defect detection method according to claim 1, characterized in that: The step of inputting the preprocessed image into the prebuilt defect detection model includes: When building a defect detection model, select the YOLOv8 model that can be used for circuit board defect detection as the teacher model, and obtain the soft label fed back by the teacher model; Obtain hard labels included in a dataset used to train a defect detection model; A distillation loss function of the defect detection model is set based on the soft label, and a task loss function of the defect detection model is set based on the hard label.

8. A circuit board defect detection device, characterized in that: include: A processing module is used to obtain the image to be detected and perform preprocessing to obtain a preprocessed image; An input module, used to input the preprocessed image into a pre-built defect detection model, wherein the defect detection model includes an extraction network, a feature fusion network and a detection head connected in sequence; A feature extraction module, used to determine the convolution weights of each convolution module included in the extraction network, and based on the convolution weights, use the extraction network to perform feature extraction processing on the preprocessed image to obtain multiple feature maps of different scales; A fusion module is used to determine a fusion weight corresponding to a scale, and based on the fusion weight, use a feature fusion network to fuse feature maps of multiple scales to obtain a fused feature map; The detection module is used to use the detection head to perform defect detection on the fused feature map to obtain the defect detection result.

9. A circuit board defect detection device, characterized in that: The circuit board defect detection device comprises: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors calls the instructions in the memory to enable the circuit board defect detection device to perform each step of the circuit board defect detection method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the various steps of the circuit board defect detection method as described in any one of claims 1-7 are implemented.

Images