Circuit board defect detection method and equipment based on improved YOLOv8

Abstract

The invention relates to a circuit board defect detection method and equipment based on improved YOLOv8. The method comprises the following steps: setting a multi-modal dynamic attention framework and deformable convolution into a YOLOv8 model for improvement; training the YOLOv8 improved model by using the training data after the dynamic bounding box regression loss optimization and the adaptive focusing classification loss optimization; inputting a to-be-detected circuit board image into the trained YOLOv8 improved model, extracting defect feature information through deformable convolution, and enhancing the defect feature information by using a multi-modal dynamic attention framework to obtain a multi-scale feature image; and performing feature fusion on the multi-scale feature image by using the neck network, and inputting the generated optimal feature image into the head network for defect detection to obtain a defect detection result. According to the invention, the technical problems of low precision and poor real-time performance and deployment efficiency of a circuit board defect detection technology based on traditional YOLOv8 are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of printed circuit board defect detection, and particularly to a method and device for printed circuit board defect detection based on improved YOLOv8. Background Technique

[0002] With the breakthrough development of convolutional neural networks (CNNs), the printed circuit board (PCB) defect detection technology has gradually adopted the YOLO series of algorithms based on a single-stage detection framework. YOLOv5 / v7 boost the detection speed to 120 FPS by constructing multi-scale prediction layers and a feature pyramid network (FPN). However, their shallow feature extraction ability is insufficient, resulting in a recall rate of only about 70% for small target defects. Especially when dealing with short-circuit defects with a spacing of less than 5 μm on high-density PCB boards, it is vulnerable to background texture interference. YOLOv8 introduces the C3 module to enhance cross-scale feature fusion and adopts the Wise-IoU loss function to dynamically adjust the localization loss weight, enabling it to achieve 87.5% mAP@0.5 on the COCO dataset.

[0003] However, due to the limited shallow feature extraction ability of the traditional YOLOv8 model, the miss rate of tiny defects (such as scratches, short circuits, etc. with an area less than 0.1 mm 2 is extremely high. Especially under complex background textures or non-uniform lighting conditions, its detection performance drops significantly. At the same time, the number of parameters of the YOLOv8 model is as high as 3.2M and the weight file is nearly 13.6MB (such as the YOLOv8n version), which causes the inference speed of the model on industrial embedded devices such as Jetson AGX Xavier to drop sharply to 40 FPS, resulting in a sharp increase in computing costs and thus making it difficult to meet the real-time requirement of 50 FPS on industrial embedded devices. Further research shows that when the YOLOv8 model is deployed to an embedded platform with a computing power lower than 5 TOPS, the latency fluctuation of the YOLOv8 model will exceed 30 ms, which may further trigger mis-triggering of the production line synchronization control system.

[0004] The feature selection mechanism of the traditional YOLOv8 model mostly relies on a single attention module. Its single-modal attention lacks the dynamic coupling ability of channel-space. In complex backgrounds such as dense solder joints and silk screen characters, the fixed weight allocation mechanism cannot effectively distinguish the spectral overlap between defect features and background noise, resulting in difficulty in coordinating channel feature enhancement and spatial noise suppression, and further causing a feature confusion error of up to 23%. Secondly, the fixed sampling grid based on standard convolution in the traditional YOLOv8 model is difficult to match the extension directions of irregular defects such as cracks and burrs, resulting in the loss of key morphological features. Moreover, the pooling operation's annihilation effect on the details of small targets (<10 pixels) further reduces the detection sensitivity of slender cracks and isolated solder dross. In addition, the loss function design of the traditional YOLOv8 model has two limitations: the IoU series of losses are sensitive to the annotation quality of fuzzy and occluded defects, and their coupled gradient calculation will cause an optimization direction deviation due to low-quality samples; the cross-entropy loss is easily dominated by high-frequency simple samples when the defect-background samples are extremely imbalanced, resulting in the gradient signal of rare defects (such as micro-holes) being submerged. Summary of the Invention

[0005] (1) Technical Problems to be Solved

[0006] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a circuit board defect detection method and device based on improved YOLOv8, which solves the technical problems of low accuracy, poor real-time performance and deployment efficiency existing in the circuit board defect detection technology based on traditional YOLOv8.

[0007] (2) Technical Solutions

[0008] To achieve the above object, the main technical solutions adopted by the present invention include:

[0009] In the first aspect, an embodiment of the present invention provides a circuit board defect detection method based on improved YOLOv8, including:

[0010] Setting the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model;

[0011] Performing dynamic bounding box regression loss optimization and adaptive focus classification loss optimization on the obtained training data, and using the optimized training data to train the improved YOLOv8 model, where the training data is image data with all circuit board defect types;

[0012] Inputting the circuit board image to be detected into the trained improved YOLOv8 model, extracting defect feature information in the circuit board image through deformable convolution, and enhancing the defect feature information by using the multi-modal dynamic attention framework to obtain a multi-scale feature image;

[0013] Use the neck network in the trained improved YOLOv8 model to perform feature fusion on the multi-scale feature images, and input the generated optimal feature images into the head network of the trained improved YOLOv8 model for defect feature detection to obtain the defect detection results of the circuit board.

[0014] Optionally, setting the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model includes:

[0015] Obtain a multi-modal dynamic attention framework and a deformable convolution, where the multi-modal dynamic attention framework is constructed by an SE module and a CBAM module with a set dynamic weight coefficient ratio;

[0016] Set the SE module in the backbone network of the preset YOLOv8 model;

[0017] Set the CBAM module in the neck network of the preset YOLOv8 model;

[0018] Call the deformable convolution to replace the convolution of the C2f module in the backbone network of the YOLOv8 model.

[0019] Optionally, before optimizing the dynamic bounding box regression loss and the adaptive focus classification loss for the obtained training data and training the improved YOLOv8 model with the optimized training data, it further includes:

[0020] Obtain image data of several circuit boards with defects through a preset shooting device;

[0021] Traverse all the image data for image augmentation adjustment including rotation, flipping, and brightness to obtain the augmented image data;

[0022] Input the augmented image data into a preset adversarial network to obtain synthetic defect image data;

[0023] Mark the circuit board defects in the augmented image data and the synthetic defect image data through a preset automatic marking tool to obtain the marked image data;

[0024] After normalizing the marked image data, divide the normalized image data into training data including a training set and a validation set according to a set allocation ratio.

[0025] Optionally, optimizing the dynamic bounding box regression loss and the adaptive focus classification loss for the obtained training data includes:

[0026] The calculation formula for optimizing the dynamic bounding box regression loss is:

[0027]

[0028] In the formula, L reg represents the dynamic bounding box regression loss function, r detach represents the gradient truncation term of the classification loss with respect to the regression parameters, γ represents the IoU outlier degree, IoU represents the intersection over union of the ground truth box and the predicted box, IoU ∈ [0, 1], μ I represents the mean of the IoU for the current batch, σ I represents the standard deviation of the IoU for the current batch;

[0029] The calculation formula for the adaptive focus classification loss optimization is:

[0030]

[0031] In the formula, L reg represents the adaptive focus classification loss function, α t represents the class weight, α t ∈ [0, 1], p t represents the predicted probability of the model for the target class, p t ∈ [0, 1], λ t represents the focus coefficient dynamically adjusted according to the prediction difficulty.

[0032] Optionally, the circuit board image to be detected is input into the trained improved YOLOv8 model. After extracting the defect feature information in the circuit board image through deformable convolution and enhancing the defect feature information using the multi-modal dynamic attention framework, a multi-scale feature image is obtained, including:

[0033] The calculation formula for deformable convolution is:

[0034]

[0035] In the formula, F out (p) represents the output feature map, K represents the total number of sampling points of the convolution kernel, ω k represents the original weight of the convolution kernel, F in represents the input feature map, p represents the current position of the output feature map, p k represents the fixed offset of the k-th sampling point of the convolution kernel, Δp k represents the learnable dynamic offset, Δm k represents the modulation scalar, which adjusts the contribution weight of the sampling points, in the range [0, 1].

[0036] The calculation formula for the multi-modal dynamic attention framework is:

[0037] F out = α · SE(F in ) + β · CBAM spatial(F in );

[0038] Wherein, F out represents the output feature map, CBAM spatial (·) represents the channel-spatial hybrid attention module, SE(·) represents the channel attention module, α and β represent dynamic weight coefficients, where α, β = Softmax(W θ *[GAP(F in )), GMP(F in )]), satisfying α + β = 1, W θ represents the learnable dynamic weight matrix, Softmax represents the activation function, GAP(·) represents global average pooling, and GMP(·) represents global maximum pooling.

[0039] Optionally, use the neck network in the trained improved YOLOv8 model to perform feature fusion on the multi-scale feature images, and input the generated optimal feature images into the head network of the trained improved YOLOv8 model for defect feature detection. The obtained defect detection results of the circuit board include:

[0040] Obtain the multi-scale feature images output by the backbone network of the trained improved YOLOv8 model;

[0041] Use the neck network of the trained improved YOLOv8 model to perform multi-level feature fusion on the multi-scale feature images to generate the fused optimal feature images;

[0042] Input the optimal feature images into the head network of the trained improved YOLOv8 model for defect feature detection to obtain the defect detection results of the circuit board.

[0043] Optionally, use the neck network of the trained improved YOLOv8 model to perform multi-level feature fusion on the multi-scale feature images to generate the fused optimal feature images, including:

[0044] Perform channel dimension concatenation and spatial dimension alignment on the multi-scale feature images to obtain the fused feature images;

[0045] Use the spatial branch of the CBAM module in the multi-modal dynamic attention framework to dynamically adjust the weight distribution of each scale feature in the multi-scale feature images;

[0046] Based on the attention mechanism of the CBAM module, perform joint optimization of the channel and spatial dimensions of the fused feature images to generate the fused optimal feature images.

[0047] Optionally, input the optimal feature images into the head network of the trained improved YOLOv8 model for defect feature detection. The obtained defect detection results of the circuit board include:

[0048] Divide the optimal feature image into multi-scale prediction grid cells, and each cell independently predicts a preset number of bounding boxes;

[0049] Through the dynamic anchor box matching mechanism, calculate the intersection over union (IoU) of each bounding box and the true defect box, and filter the optimal anchor box as the prediction benchmark;

[0050] Use a convolutional neural network to regress and predict the coordinate offset, width and height scaling ratio, and defect category probability of the bounding box, and generate a preliminary defect detection result;

[0051] According to the preset circuit board design rule library, perform semantic verification and false detection filtering on the defect information in the preliminary defect detection result to obtain a visual defect detection result including a defect distribution heat map and three-dimensional space positioning information.

[0052] In a second aspect, an embodiment of the present invention provides a circuit board defect detection device, including:

[0053] An industrial camera;

[0054] An embedded computing platform, connected to the industrial camera, for executing the above-mentioned circuit board defect detection method based on the improved YOLOv8 to perform circuit board defect detection on the circuit board image captured by the industrial camera.

[0055] Optionally, the embedded computing platform includes:

[0056] A YOLOv8 model improvement module for setting the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model;

[0057] A YOLOv8 improved model training module for optimizing the dynamic bounding box regression loss and the adaptive focusing classification loss of the obtained training data, and training the YOLOv8 improved model with the optimized training data, where the training data is image data with all circuit board defect types;

[0058] A multi-scale feature image acquisition module for inputting the circuit board image to be detected into the trained YOLOv8 improved model, extracting defect feature information in the circuit board image through deformable convolution, and enhancing the defect feature information using the multi-modal dynamic attention framework to obtain a multi-scale feature image;

[0059] A circuit board defect detection module for performing feature fusion on the multi-scale feature image using the neck network in the trained YOLOv8 improved model, and inputting the generated optimal feature image into the head network of the trained YOLOv8 improved model for defect feature detection to obtain the defect detection result of the circuit board.

[0060] (III) Beneficial Effects

[0061] The beneficial effects of the present invention are as follows: The present invention proposes a circuit board defect detection method based on improved YOLOv8. First, by constructing a multi-modal dynamic attention framework that realizes cross-level feature interaction, the representation ability of shallow defect features of the circuit board is enhanced, breaking through the detection limit of traditional YOLOv8 models for defects of 0.1mm 2 or less. Secondly, by introducing deformable convolutions, a collaborative compression strategy of deploying depthwise separable convolutions and dynamic channel pruning can be implemented, reducing the model parameter quantity by 40% while maintaining the detection accuracy. Additionally, a dual dynamic optimization mechanism of dynamic bounding box regression loss optimization and adaptive focus classification loss optimization is adopted, reducing the negative impact of low-quality samples on the training process and improving the recall rate of rare defects, further realizing the efficient joint optimization of the regression task and the classification task. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 is a schematic flow chart of a circuit board defect detection method based on improved YOLOv8 provided by an embodiment of the present invention;

[0063] Figure 2 is a network structure diagram of a traditional YOLOv8 model provided by an embodiment of the present invention;

[0064] Figure 3 is a network structure diagram of an improved YOLOv8 model provided by an embodiment of the present invention;

[0065] Figure 4 is a dataset after data augmentation of circuit board images provided by an embodiment of the present invention;

[0066] Figure 5 is a display diagram of circuit board defect detection results provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] In order to better explain the present invention for easy understanding, the present invention will be described in detail below with reference to the accompanying drawings and through specific embodiments.

[0068] Refer to Figures 1 to 5As shown in the figure, a circuit board defect detection method based on improved YOLOv8 proposed by an embodiment of the present invention includes: setting the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model; performing dynamic bounding box regression loss optimization and adaptive focus classification loss optimization on the obtained training data, and using the optimized training data to train the improved YOLOv8 model, where the training data is image data with all circuit board defect types; inputting the circuit board image to be detected into the trained improved YOLOv8 model, extracting defect feature information in the circuit board image through deformable convolution, and enhancing the defect feature information using the multi-modal dynamic attention framework to obtain a multi-scale feature image; using the neck network in the trained improved YOLOv8 model to perform feature fusion on the multi-scale feature image, and inputting the generated optimal feature image into the head network of the trained improved YOLOv8 model for defect feature detection to obtain the defect detection result of the circuit board.

[0069] In this embodiment, first, by constructing a multi-modal dynamic attention framework that realizes cross-level feature interaction, the representation ability of shallow circuit board defect features is enhanced, breaking through the detection limit of traditional YOLOv8 models for defects smaller than 0.1mm. 2 Secondly, by introducing deformable convolution, a collaborative compression strategy of deploying depthwise separable convolution and dynamic channel pruning can be realized, reducing the model parameter quantity by 40% while maintaining the detection accuracy. Additionally, a dual dynamic optimization mechanism of dynamic bounding box regression loss optimization and adaptive focus classification loss optimization is adopted, reducing the negative impact of low-quality samples on the training process and improving the recall rate of rare defects, further realizing the efficient joint optimization of the regression task and the classification task.

[0070] To better understand the above technical solution, the exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more clear and thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0071] Specifically, referring to Figure 1 As shown in the figure, an embodiment of the present invention proposes a circuit board defect detection method based on improved YOLOv8, which includes:

[0072] S100. Set the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model.

[0073] In this embodiment, the multi-modal dynamic attention framework is constructed by an SE (Squeeze-and-Excitation, channel attention fusion) module and a CBAM (Convolutional Block Attention Module, channel-spatial hybrid attention) module with a set dynamic weight coefficient ratio. The multi-modal dynamic attention mechanism of the multi-modal dynamic attention framework can achieve cross-level feature interaction, adaptively enhance the channel response of defect features, and at the same time suppress the noise interference of complex backgrounds (such as solder joints and silk screens).

[0074] Refer to Figures 2 to 3 As shown, an SE module is added at the end of the backbone network to enhance the feature expression of key channels, and then dynamic attention fusion and CBAM module integration are carried out. Channel-spatial hybrid attention is introduced in the neck network to suppress background noise and improve the focusing ability of the defect area; furthermore, the attention mechanism of the channel attention SE module can enhance the contribution of important channels by weighting the channel features, which can improve the sensitivity of the model to circuit board defect features.

[0075] In this embodiment, deformable convolution mainly replaces the standard convolution in the C2f module of the backbone network in the traditional YOLOv8 model, replacing the convolution operation with a traditional fixed receptive field. By dynamically adjusting the sampling positions of the convolution kernels, the model can accurately capture the morphological features of irregular defects (such as burrs and cracks) and enhance the local detail expression ability of small targets (<10 pixels). Compared with the standard convolution structure, this optimization significantly improves the detection sensitivity of the model to unstructured defects.

[0076] Furthermore, step S100 may include the following sub-steps S110 to S140:

[0077] S110. Obtain the multi-modal dynamic attention framework and deformable convolution. The multi-modal dynamic attention framework is constructed by an SE module and a CBAM module with a set dynamic weight coefficient ratio.

[0078] Preferably, Deformable Convolution V2 is selected for the deformable convolution. By introducing learnable offsets and modulation scalars, it not only allows the convolution kernels to dynamically adjust the sampling positions on the input feature map, but also can assign different weights to each sampling point. And by sharing the offset prediction layer (such as using 1x1 convolution to generate offsets and modulation scalars), on the premise of increasing a small amount of computational cost (usually <10%), it can significantly improve the model accuracy.

[0079] S120. Set the SE module in the backbone network of the preset YOLOv8 model.

[0080] In this embodiment, during the circuit board defect detection, the SE module enhances the chromaticity difference between the defects and the background through HSV color space (H, S, and V channels, corresponding to hue, saturation, and lightness respectively). For example, the weight of the lightness channel (V channel) of the copper leakage defect in the circuit board image is increased to 0.92.

[0081] S130. Set the CBAM module in the neck network of the preset YOLOv8 model.

[0082] In this embodiment, during the circuit board defect detection, the spatial branch of the CBAM module redistributes the weights in the gradient direction using the Sobel operator. For example, to suppress the response of the silk screen characters in the circuit board image in the vertical / horizontal gradient direction, their weights are reduced to 0.15.

[0083] S140. Call the deformable convolution to replace the convolution of the C2f module in the backbone network of the YOLOv8 model.

[0084] In this embodiment, by introducing the deformable convolution, the model can deploy a depthwise separable convolution and dynamic channel pruning collaborative compression strategy, achieving a 40% reduction in the number of model parameters while maintaining the detection accuracy.

[0085] S200. Optimize the dynamic bounding box regression loss and the adaptive focus classification loss for the obtained training data, and use the optimized training data to train the improved YOLOv8 model. The training data is image data with all circuit board defect types.

[0086] In this embodiment, the dynamic bounding box regression loss optimization and the adaptive focus classification loss optimization are used for joint optimization of the bounding box regression and classification tasks, achieving efficient joint optimization of the regression task and the classification task, and significantly improving the circuit board defect detection accuracy. Dynamic bounding box regression loss optimization: Based on the dynamic gradient allocation strategy of outlier evaluation, reduce the negative impact of low-quality samples (such as blurred and occluded defects) on the training process, and optimize the regression accuracy of small targets. Adaptive focus classification loss optimization: Through parameter adaptive adjustment, focus on difficult example defect samples, alleviate the class imbalance problem, and improve the recall rate of rare defects.

[0087] The calculation formula for the dynamic bounding box regression loss optimization is:

[0088]

[0089] In Equation (2), L reg represents the dynamic bounding box regression loss function, and r detachrepresents the gradient truncation term of the classification loss with respect to the regression parameters, γ represents the IoU outlier degree, IoU represents the intersection over union of the ground truth box and the predicted box, IoU ∈ [0, 1], μ I represents the mean of the IoU of the current batch, σ I represents the standard deviation of the IoU of the current batch.

[0090] For example, for the blurred cracks covered by the silk screen on the circuit board (annotation box deviation ±3 pixels), when the IoU outlier degree γ ≥ 2σ I the gradient weight automatically decays to 24%, avoiding misleading the model with low-quality annotations. For the solder dregs less than 15 pixels in the circuit board, through the decoupling term r detach separate the center point and the width-height optimization, so that the positioning error of the 0.2 mm solder ball is compressed from 4.7 pixels to 1.8 pixels, and the edge fitting degree is increased by 3.1 times.

[0091] The calculation formula for the adaptive focus classification loss optimization is:

[0092]

[0093] In formula (2), L reg represents the adaptive focus classification loss function, α t represents the class weight, α t ∈ [0, 1], p t represents the predicted probability of the model for the target class, p t ∈ [0, 1], λ t represents the focus coefficient dynamically adjusted according to the prediction difficulty.

[0094] For example, for the hole deviation defect with a panel occupancy of 0.008% on the circuit board, set the class weight α t = 0.8, which increases the defect recall rate from 31.4% to 67.8%. At the same time, through the dynamic threshold λ t distinguish the oxidation spots (λ t = 3) from the real defects, reducing the false detection rate to 4.3%. For the highly similar defects on the circuit board (such as cold solder joints and normal solder joints), when the predicted probability p t < 0.2, its loss weight is amplified by 8 times, forcing the model to learn the subtle texture differences.

[0095] In this embodiment, dynamic bounding box regression loss optimization and adaptive focus classification loss optimization are used for joint optimization of the bounding box regression and classification tasks, which improves the AP50 (average precision when the IoU threshold is 0.5) of circuit board solder joint inspection from 54.2% to 82.7%, stabilizes the regression error of silk screen covered cracks within 1.8 pixels, and simultaneously compresses the class imbalance miss detection rate from 19.3% to 5.1%. The dual dynamic loss optimization mechanism monitors through KL divergence (DKL < 0.03) to ensure the consistency of dual-task optimization, and realizes the accurate detection of tiny defects (10 pixels) and occluded targets while maintaining the real-time performance of 83 FPS (Frames Per Second).

[0096] Further, before step S200, the following steps F100 to F500 are also included:

[0097] F100. Obtain image data of several circuit boards with defects through a preset shooting device.

[0098] For example, industrial cameras, X-ray imaging devices, etc.

[0099] F200. Traverse all the image data for image augmentation adjustment including rotation, flipping, and brightness to obtain augmented image data.

[0100] F300. Input the augmented image data into a preset adversarial network to obtain synthetic defect image data.

[0101] As Figure 4 shown, after collecting the circuit board image data to be detected, first use techniques such as rotation, flipping, and brightness adjustment to augment the dataset, and introduce a GAN adversarial network to synthesize rare defect samples to alleviate the problem of unbalanced defect data.

[0102] F400. Mark the circuit board defects in the augmented image data and the synthetic defect image data through a preset automated marking tool to obtain marked image data.

[0103] F500. After normalizing the marked image data, divide the normalized image data into training data including a training set and a validation set according to a set allocation ratio.

[0104] In this embodiment, the LabelImg tool is used to accurately label the defects in the circuit board images, and the image data is standardized to accelerate model convergence.

[0105] S300. Input the circuit board image to be detected into the trained improved YOLOv8 model. Extract the defect feature information in the circuit board image through deformable convolution, and use the multi-modal dynamic attention framework to enhance the defect feature information to obtain a multi-scale feature image.

[0106] The calculation formula of deformable convolution is:

[0107]

[0108] In formula (3), F out (p) represents the output feature map, K represents the total number of sampling points of the convolution kernel, ω k represents the original weight of the convolution kernel, F in represents the input feature map, p represents the current position of the output feature map, p k represents the fixed offset of the k-th sampling point of the convolution kernel, Δp k represents the learnable dynamic offset, Δm k represents the modulation scalar, which adjusts the contribution weight of the sampling point, and the range is [0, 1].

[0109] For example, the customized improvement for circuit board defect detection: for irregular defects such as cracks and burrs, the sampling points of the convolution kernel are adaptively offset along the defect extension direction (for example, the spacing is extended from 3 pixels to 8 pixels), so that the 3×3 convolution kernel can effectively cover a 1.2 mm long micro-crack; for solder slag / pinhole defects less than 10 pixels, the edge features are strengthened through the modulation coefficient Δm k (the edge point weight is set to 0.9, and the smooth area is reduced to 0.2), and a sparse sampling strategy (stride = 1) is adopted to avoid detail loss; in the detection of the copper foil layer of the circuit board, the maximum offset is restricted (||Δp k || ≤ 2 pixels) to prevent excessive deformation from damaging the circuit topology.

[0110] The calculation formula of the multi-modal dynamic attention framework is:

[0111] F out = α·SE(F in ) + β·CBAM spatial (F in ) (4)

[0112] In formula (4), F out represents the output feature map, CBAM spatial (·) represents the channel-spatial hybrid attention module, SE(·) represents the channel attention module, α and β represent the dynamic weight coefficients, α, β = Softmax(W θ *[GAP(F in ),GMP(F in )]), satisfying α + β = 1, Wθ represents a learnable dynamic weight matrix, Softmax represents an activation function, GAP(·) represents global average pooling, and GMP(·) represents global maximum pooling.

[0113] For example, in circuit board defect detection, the SE module enhances the chromaticity difference between defects and the background through HSV color space conversion (such as increasing the V-channel weight of the copper leakage defect in the circuit board image to 0.92). The spatial branch of the CBAM module uses the Sobel operator to suppress the vertical / horizontal stroke responses of the silk-screen characters in the gradient direction (the weight is reduced to 0.15). Finally, scene adaptive fusion is achieved through dynamic gating: in the area of dense solder joints, β is increased to 0.8 for priority spatial noise suppression; in the monochromatic copper foil area, α is increased to 0.7 to enhance the channel difference.

[0114] S400. Use the neck network in the trained improved YOLOv8 model to perform feature fusion on the multi-scale feature images, and input the generated optimal feature image into the head network of the trained improved YOLOv8 model for defect feature detection to obtain the defect detection result of the circuit board.

[0115] In this embodiment, step S400 may include the following sub-steps S410 to S430:

[0116] S410. Obtain the multi-scale feature images output by the backbone network of the trained improved YOLOv8 model.

[0117] S420. Use the neck network of the trained improved YOLOv8 model to perform multi-level feature fusion on the multi-scale feature images to generate the optimal fused feature image.

[0118] S430. Input the optimal feature image into the head network of the trained improved YOLOv8 model for defect feature detection to obtain the defect detection result of the circuit board.

[0119] Furthermore, step S420 includes the following sub-steps S421 to S423:

[0120] S421. Perform channel dimension concatenation and spatial dimension alignment on the multi-scale feature images to obtain the fused feature image.

[0121] S422. Use the spatial branch of the CBAM module in the multi-modal dynamic attention framework to dynamically adjust the weight distribution of each scale feature in the multi-scale feature images.

[0122] S423. Based on the attention mechanism of the CBAM module, perform joint optimization of the fused feature image in the channel and spatial dimensions to generate the optimal fused feature image.

[0123] Further, step S430 includes the following sub-steps S431 to S434:

[0124] S431. Divide the optimal feature image into multi-scale prediction grid cells, and each cell independently predicts a preset number of bounding boxes.

[0125] S432. Through the dynamic anchor box matching mechanism, calculate the intersection over union (IoU) between each bounding box and the true defect box, and filter the optimal anchor boxes as the prediction benchmarks.

[0126] S433. Use a convolutional neural network to regress and predict the coordinate offsets, width and height scaling ratios, and defect category probabilities of the bounding boxes to generate a preliminary defect detection result.

[0127] S434. According to the preset circuit board design rule library, perform semantic verification and false detection filtering on the defect information in the preliminary defect detection result to obtain a visual defect detection result including a defect distribution heat map and three-dimensional spatial positioning information.

[0128] In a specific embodiment, the trained improved YOLOv8 model is quantized and deployed through TensorRT, and interface code is written to transmit the circuit board image to be detected into the improved YOLOv8 model. As Figure 3 shown, the image enters the improved YOLOv8 model from the "img" entry, and the information flow is transmitted along the Figure 3 arrow, and finally the improved YOLOv8 model outputs the coordinates of the defects at three "Detect" exits to obtain the final detection result. The detection result is a visual output detection result, which is displayed in the form of a heat map and bounding boxes, clearly marking the defect type, location, and confidence level, as Figure 5 shown.

[0129] In addition, the present invention also proposes a circuit board defect detection device, which includes: an industrial camera; an embedded computing platform connected to the industrial camera, and is used to execute the above-mentioned circuit board defect detection method based on the improved YOLOv8 to perform circuit board defect detection on the circuit board image captured by the industrial camera.

[0130] Further, the embedded computing platform includes:

[0131] A YOLOv8 model improvement module, which is used to set the obtained multi-modal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain an improved YOLOv8 model.

[0132] The YOLOv8 improved model training module is used to optimize the dynamic bounding box regression loss and the adaptive focus classification loss for the obtained training data, and train the YOLOv8 improved model with the optimized training data. The training data is image data with all types of circuit board defects.

[0133] The multi-scale feature image acquisition module is used to input the circuit board image to be detected into the trained YOLOv8 improved model, extract the defect feature information in the circuit board image through deformable convolution, and enhance the defect feature information using the multi-modal dynamic attention framework to obtain a multi-scale feature image.

[0134] The circuit board defect detection module is used to perform feature fusion on the multi-scale feature image using the neck network in the trained YOLOv8 improved model, and input the generated optimal feature image into the head network of the trained YOLOv8 improved model for defect feature detection to obtain the defect detection result of the circuit board.

[0135] In summary, a circuit board defect detection method and device proposed in this embodiment significantly improve the detection accuracy and real-time performance of the circuit board by integrating deformable convolution, multi-modal attention mechanism, and dual dynamic loss optimization. Compared with the prior art, its core advantages are reflected in the following aspects:

[0136] (1) Construct a multi-modal dynamic attention framework through the SE module and the CBAM module. This mechanism can adaptively enhance the channel response of defect features while suppressing the noise interference of complex backgrounds (such as solder joints and silk printing), significantly improving the feature selection ability in high-density defect scenarios and effectively reducing the false detection rate.

[0137] (2) Introduce deformable convolution into the backbone network to replace the traditional convolution operation with a fixed receptive field. By dynamically adjusting the sampling position of the convolution kernel, the model can accurately capture the morphological features of irregular defects (such as burrs and cracks) and enhance the local detail expression ability of small targets (less than 10 pixels). Compared with the standard convolution structure, it significantly improves the detection sensitivity of the model to unstructured defects.

[0138] (3) Adopt a dual dynamic optimization mechanism of dynamic bounding box regression loss optimization and adaptive focus classification loss optimization. The dynamic bounding box regression loss optimization is based on a dynamic gradient allocation strategy based on outlier evaluation, reducing the negative impact of low-quality samples (such as blurred and occluded defects) on the training process and optimizing the regression accuracy of small targets. The adaptive focus classification loss optimization focuses on difficult example defect samples through parameter adaptive adjustment, alleviates the class imbalance problem, and improves the recall rate of rare defects. The two work together to achieve efficient joint optimization of the regression task and the classification task.

[0139] Since the device described in the above embodiments of the present invention is the device adopted for implementing the method of the above embodiments of the present invention, based on the method described in the above embodiments of the present invention, those skilled in the art can understand the specific structure and variations of the device, and thus will not be elaborated herein. Any device, apparatus, or system adopted for the method of the above embodiments of the present invention falls within the scope of protection of the present invention.

[0140] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be implemented in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can also be implemented by computer program instructions.

[0142] It should be noted that in the description of the present invention, the words "a" or "an" before a component do not exclude the existence of a plurality of such components. The present invention can be implemented by means of hardware including several different components and by means of a suitably programmed computer. The use of the words first, second, third, etc. is only for the convenience of expression and does not indicate any order. These words can be understood as part of the component name.

[0143] In addition, it should be noted that in the description of this specification, the description of terms such as "one embodiment", "some embodiments", "embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0144] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications to these embodiments after learning the basic creative concepts.

[0145] Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from the spirit and scope of the present invention.

Claims

1. A circuit board defect detection method based on improved YOLOv8, characterized in that: include: The obtained multimodal dynamic attention framework and deformable convolution are set into the preset YOLOv8 model to obtain the improved YOLOv8 model; Performing dynamic bounding box regression loss optimization and adaptive focus classification loss optimization on the acquired training data, and using the optimized training data to train the YOLOv8 improved model, wherein the training data is image data with all circuit board defect types; The circuit board image to be inspected is input into the trained YOLOv8 improved model, the defect feature information in the circuit board image is extracted through deformable convolution, and the defect feature information is enhanced using the multimodal dynamic attention framework to obtain a multi-scale feature image; The neck network in the trained YOLOv8 improved model is used to perform feature fusion on the multi-scale feature images, and the generated optimal feature image is input into the head network of the trained YOLOv8 improved model for defect feature detection to obtain the defect detection results of the circuit board.

2. The method according to claim 1, characterized in that The obtained multimodal dynamic attention framework and deformable convolution are set to the preset YOLOv8 model to obtain the improved YOLOv8 model including: Obtaining a multimodal dynamic attention framework and a deformable convolution, wherein the multimodal dynamic attention framework is constructed by a SE module and a CBAM module with a set dynamic weight coefficient ratio; Set the SE module in the backbone network of the preset YOLOv8 model; Set the CBAM module in the neck network of the preset YOLOv8 model; Call deformable convolution to replace the convolution of the C2f module of the backbone network in the YOLOv8 model.

3. The method according to claim 1, characterized in that Before performing dynamic bounding box regression loss optimization and adaptive focus classification loss optimization on the acquired training data and using the optimized training data to train the YOLOv8 improved model, it also includes: Acquire image data of a plurality of defective circuit boards through a preset photographing device; Traversing all image data to perform image expansion adjustment including rotation, flipping and brightness to obtain expanded image data; Inputting the expanded image data into a preset adversarial network to obtain synthetic defect image data; By using a preset automatic marking tool, the circuit board defects in the expanded image data and the synthetic defect image data are marked to obtain marked image data; After the labeled image data is normalized, the normalized image data is divided into training data including a training set and a validation set according to a set allocation ratio.

4. The method according to claim 1, characterized in that The dynamic bounding box regression loss optimization and adaptive focus classification loss optimization of the acquired training data include: The calculation formula for dynamic bounding box regression loss optimization is: Where, L reg represents the dynamic bounding box regression loss function, r detach represents the gradient truncation term of the classification loss to the regression parameter, γ represents the IoU outlier, IoU represents the intersection over union ratio of the real box and the predicted box, IoU∈[0,1], μ I represents the mean IoU of the current batch, σ I Represents the standard deviation of IoU of the current batch; The calculation formula for adaptive focus classification loss optimization is: Where, L reg represents the adaptive focused classification loss function, α t represents the category weight, α t ∈[0, 1], p t Represents the model's predicted probability of the target category, p t ∈[0, 1], λ t The representative dynamically adjusts the focus factor according to the prediction difficulty.

5. The method according to claim 1, characterized in that The circuit board image to be inspected is input into the trained YOLOv8 improved model, and the defect feature information in the circuit board image is extracted through deformable convolution. After the defect feature information is enhanced using the multimodal dynamic attention framework, the multi-scale feature image is obtained, including: The calculation formula of deformable convolution is: In the formula, F out (p) represents the output feature map, K represents the total number of sampling points of the convolution kernel, ω k Represents the original weight of the convolution kernel, F in represents the input feature map, p represents the current position of the output feature map, and p k Represents the fixed offset of the kth sampling point of the convolution kernel, Δp k Represents the learnable dynamic offset, Δm k Represents the modulation scalar, which adjusts the contribution weight of the sampling point, and the range is [0,1]. The calculation formula of the multimodal dynamic attention framework is: F out =α·SE(F in )+β·CBAM spatial (F in ); In the formula, F out Represents the output feature map, CBAM spatial (·) represents the channel-spatial hybrid attention module, SE(·) represents the channel attention module, α, β represent the dynamic weight coefficients, α, β = Softmax(W θ *[GAP(F in ), GMP(F in )]), satisfying α+β=1, W θ represents a learnable dynamic weight matrix, Softmax represents the activation function, GAP(·) represents global average pooling, and GMP(·) represents global maximum pooling.

6. The method according to claim 1, characterized in that The neck network in the trained YOLOv8 improved model is used to perform feature fusion on the multi-scale feature images, and the generated optimal feature image is input into the head network of the trained YOLOv8 improved model for defect feature detection. The defect detection results of the circuit board include: Get the multi-scale feature image output by the backbone network of the trained YOLOv8 improved model; The neck network of the trained YOLOv8 improved model is used to perform multi-level feature fusion on the multi-scale feature images to generate the optimal fused feature image; The optimal feature image is input into the head network of the trained YOLOv8 improved model to perform defect feature detection and obtain the defect detection results of the circuit board.

7. The method according to claim 6, characterized in that The multi-scale feature images are fused at multiple levels using the trained neck network of the improved YOLOv8 model to generate the optimal fused feature images, including: The multi-scale feature images are spliced ​​in channel dimension and aligned in spatial dimension to obtain a fused feature image; The spatial branch of the CBAM module in the multimodal dynamic attention framework is used to dynamically adjust the weight distribution of each scale feature in the multi-scale feature image; Based on the attention mechanism of the CBAM module, the fused feature image is jointly optimized in terms of channel and spatial dimensions to generate the optimal fused feature image.

8. The method according to claim 6, characterized in that The optimal feature image is input into the head network of the trained YOLOv8 improved model to detect defect features. The defect detection results of the circuit board are as follows: The optimal feature image is divided into multi-scale prediction grid cells, each of which independently predicts a preset number of bounding boxes; Through the dynamic anchor frame matching mechanism, the intersection-over-union ratio of each bounding box and the real defect box is calculated, and the optimal anchor frame is selected as the prediction benchmark; Use convolutional neural network regression to predict the coordinate offset, width and height scaling ratio, and defect category probability of the bounding box to generate preliminary defect detection results; According to the preset circuit board design rule library, the defect information in the preliminary defect detection results is semantically verified and false detection filtered to obtain a visual defect detection result including a defect distribution heat map and three-dimensional spatial positioning information.

9. A circuit board defect detection device, characterized in that: include: Industrial cameras; An embedded computing platform is connected to an industrial camera and is used to execute the circuit board defect detection method based on the improved YOLOv8 as described in any one of claims 1 to 8, so as to perform circuit board defect detection on the circuit board image captured by the industrial camera.

10. The device according to claim 9, characterized in that Embedded computing platforms include: A YOLOv8 model improvement module is used to set the acquired multimodal dynamic attention framework and deformable convolution into a preset YOLOv8 model to obtain a YOLOv8 improved model; A YOLOv8 improved model training module, used to perform dynamic bounding box regression loss optimization and adaptive focus classification loss optimization on the acquired training data, and train the YOLOv8 improved model using the optimized training data, wherein the training data is image data having all circuit board defect types; The multi-scale feature image acquisition module is used to input the circuit board image to be inspected into the trained YOLOv8 improved model, extract the defect feature information in the circuit board image through deformable convolution, and enhance the defect feature information using the multimodal dynamic attention framework to obtain a multi-scale feature image; The circuit board defect detection module is used to use the neck network in the trained YOLOv8 improved model to perform feature fusion on the multi-scale feature image, and input the generated optimal feature image into the head network of the trained YOLOv8 improved model to perform defect feature detection to obtain the defect detection result of the circuit board.

Images