PCB defect detection method based on visual converter combined with conditional diffusion

Abstract

The invention belongs to the technical field of computer vision and deep learning, and particularly relates to a PCB defect detection method based on combination of a visual converter and conditional diffusion. Comprising the following steps: constructing an unlabeled PCB image data set and carrying out data preprocessing and enhancement to obtain a preprocessed image; executing a self-supervised pre-training task on the preprocessed image to obtain a feature extraction network; based on a conditional diffusion model, generating a synthetic defect PCB image and a label thereof by using the features output by the feature extraction network and the defect type control vector; mixing the synthetic defect image with a small number of real defect images to construct a training set; performing training adjustment on the defect detection model by adopting the training set to obtain a trained defect detection model; performing PCB defect detection by using the trained defect detection model; according to the method, the robustness and the cross-domain generalization ability are remarkably improved, the missed detection risk is reduced, and the rapid and stable quality control requirement of the production line is met.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning technology, and is a PCB defect detection method based on visual transformer combined with conditional diffusion. Background Technology

[0002] In manufacturing, automated detection of surface defects has always been a crucial aspect of quality control. Traditional defect detection methods rely on manual visual inspection or classic image processing algorithms, which suffer from low efficiency and insufficient accuracy. With the development of deep learning, object detection technology based on convolutional neural networks has been applied to defect detection tasks, significantly improving detection performance. However, deep learning methods typically require a large amount of labeled defect image data for training. In real-world scenarios, defect samples are often scarce and difficult to obtain, leading to overfitting in detection models trained solely on real data, making it difficult to identify diverse defect patterns.

[0003] In recent years, self-supervised learning and visual transformer architectures have shown great potential in the field of computer vision. Visual transformers, by segmenting images into patch embedding sequences and utilizing self-attention mechanisms to model global information, have achieved excellent performance in tasks such as image classification. Self-supervised pre-training techniques can pre-train visual transformer models on unlabeled data to obtain general feature representations.

[0004] Meanwhile, diffusion models, as a new generation of generative models, have received widespread attention in the field of image synthesis. Diffusion models generate clear and realistic image samples by progressively adding noise to an image and training the model to learn the inverse denoising process. Compared to GANs, diffusion models are more stable in training and generate higher-quality images. In particular, conditional diffusion models can incorporate control signals into the image generation process, achieving controllable guidance of the generated content.

[0005] In summary, existing technologies still have room for improvement in areas such as the scarcity of defect data, utilization of self-supervised features, augmentation of synthetic data, and detection model architecture. There is an urgent need for a new technical solution that can combine the advantages of self-supervised visual transformer pre-training and conditional diffusion models to improve the accuracy and recall of defect detection under small sample conditions. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes a PCB defect detection method based on a vision transformer combined with conditional diffusion. This method includes the following steps:

[0007] S1: Construct an unlabeled PCB image dataset and perform data preprocessing and enhancement to obtain preprocessed images;

[0008] S2: Perform a self-supervised pre-training task on the preprocessed image to obtain a feature extraction network;

[0009] S3: Based on the conditional diffusion model, use the features and defect type control vectors output by the feature extraction network to generate synthetic defect PCB images and their labels;

[0010] S4: Mix the synthesized defect images with a small number of real defect images to construct a training set;

[0011] S5: The detection model based on the feature extraction network is trained and adjusted using the training set to obtain a well-trained defect detection model.

[0012] Preferably, data preprocessing includes denoising, normalization, and size standardization preprocessing of the defect-free PCB image.

[0013] Preferably, the feature extraction network is a visual transformer backbone architecture, consisting of an image segmentation and embedding module, a positional encoding layer, and a multi-layer Transformer encoder; wherein, the Transformer encoder derives feature representations in shallow, middle, and deep layers respectively, and performs channel alignment and spatial reshaping on the derived features to obtain multi-scale feature maps of three resolutions; the multi-scale features Figure 1 On one hand, the input conditional diffusion network is used as the vector for cross-attention, and on the other hand, the input feature pyramid network is used by the anchorless detection head to achieve unified feature-driven generation and detection.

[0014] Preferably, the self-supervised pre-training task is a mask reconstruction task based on a mask autoencoder, including: dividing the preprocessed PCB defect image into several image blocks of the same size; randomly occluding the original image region; reconstructing the occluded content using a visual transformer encoder-decoder; and using the reconstruction mean square error loss.

[0015] Furthermore, the mask ratio of the mask autoencoder is adjusted using a dual adaptive strategy, including: a global baseline mask rate ρ base,k As the training period k gradually increases; when the reconstruction error When the mask rate continuously drops below the preset threshold, the current mask rate is automatically increased again; full masking is maintained for pixels in areas with significant defects, while only about half the mask rate is used for pixels in the background area.

[0016] Preferably, the U-Net structure used in the conditional diffusion model consists of a symmetrical encoder and decoder and a bottleneck layer, wherein: the encoder consists of multiple convolutional modules, each of which extracts multi-scale features of the image through downsampling operations; the decoder consists of upsampling modules of the corresponding number of layers, which fuse the features of the corresponding layers of the encoder to reconstruct image details; the bottleneck layer connects the encoder and decoder and aggregates global semantics.

[0017] Furthermore, after the cross-attention of each level of the U-Net encoder, bottleneck layer, and decoder, a defect-gated residual unit is connected in series. The unit performs global average pooling on the current feature map, passes it through a single fully connected layer with a fixed bias, and then inputs it into the Sigmoid function to obtain gating coefficients. These coefficients are then weighted and cross-attentioned, and the output is injected in the form of residuals to suppress non-defect token noise and enhance the representation of defect regions.

[0018] Furthermore, the image generation process of the conditional diffusion model employs defect prior modulation noise scheduling during the forward diffusion stage, i.e., the noise step size β'(t,x) is expressed as:

[0019] β'(t,x)=β lin (t)·[1+0.2·p(x)]

[0020] Where β'(t,x) represents the pixel-level noise step size during the forward diffusion process, which depends on the time step t and the pixel position x,β lin (t) represents the time-based linear noise step size scheduling function, p(x) represents the defect saliency probability generated by the self-supervised transformer, and 0.2 is the optimal coefficient determined by grid search.

[0021] Preferably, the defect detection model consists of a feature extraction backbone network, a feature pyramid fusion module, and an anchorless detection head. The feature extraction backbone network adopts a visual transformer feature extraction network, freezing shallow weights to retain general features and fine-tuning deep weights to adapt to the defect detection task. The feature pyramid fusion module adopts a feature pyramid network to fuse feature maps of different scales to improve the detection capability of multi-scale defects. The anchorless detection head outputs the confidence of the defect category, the four-way bounding box offset, and the feature vector that enhances category discrimination.

[0022] Furthermore, the anchorless detection head sets up quantized focus loss, efficient IOU loss and class discrimination loss in parallel on the multi-scale feature map after each layer fusion, and sums the three according to specific weights to form the total loss of the detection head.

[0023] The beneficial effects of this invention are as follows:

[0024] This invention first uses mask self-supervision to pre-train the visual transformer backbone to quickly obtain high-quality defect features; then, using these features as conditions, it guides the U-Net diffusion model to generate high-fidelity, multi-type synthetic defect samples from noise to expand the data; the synthetic samples are then mixed with a small number of real samples to train the defect detection model; the trained model exhibits a higher detection rate and more accurate localization in PCB defect detection, and significantly improves robustness and cross-domain generalization ability, reduces the risk of missed detection, and meets the needs of rapid and stable quality control in production lines.

[0025] The average results of three independent experiments on the HRI-PCB dataset show that the present invention achieves mAP50:95 = 0.716 under the strict COCO evaluation criteria. Compared with commonly used YOLOv5-L, Deformable DETR and Swin-TransformerFPN, it improves the overall accuracy by about 17%-22%, with paired t-test p < 0.005, indicating that the difference is statistically significant.

[0026] This invention achieves an average of 60 FPS on an NVIDIA GeForce RTX 3090 (FP32, batch=1), which meets the real-time detection requirement of 25 FPS for a typical 300 mm / s conveyor belt. The above accuracy can be achieved with only about 20% of the manual defect labels required by the original method, thanks to the high-fidelity synthetic defect samples generated by the conditional diffusion model.

[0027] Under six common perturbation conditions, including illumination, contrast, and artificial scratches, the mAP fluctuation of this invention is ≤2.1pp, while the fluctuation of the baseline YOLOv5-L can reach 5.4pp; indicating that the global features provided by the self-supervised visual transformer and the multi-scale FPN-Anchor-Free design have better out-of-domain robustness. Attached Figure Description

[0028] Figure 1 This describes the workflow of the PCB defect detection method of the present invention.

[0029] Figure 2 This invention describes the workflow for pre-training a visual transformer using a self-supervised mask autoencoder.

[0030] Figure 3 This is a diagram of the U-Net backbone structure in the conditional diffusion model of this invention;

[0031] Figure 4 This describes the process of generating images from forward and reverse diffusion in the conditional diffusion model of this invention.

[0032] Figure 5 This is the training process for training a frameless defect detection model using the fusion of the visual transformer backbone and feature pyramid in this invention. Detailed Implementation

[0033] The technical methods of the embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and not for limiting the scope of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0034] This invention proposes a PCB defect detection method based on a vision transformer combined with conditional diffusion, such as... Figure 1 As shown, the method includes the following:

[0035] S1: Construct an unlabeled PCB image dataset and perform data preprocessing and enhancement to obtain preprocessed images.

[0036] A dataset of original unlabeled PCB defect images was constructed. In this embodiment, the HRIPCB Dataset dataset was used, which contains labels for 6 types of defects, as shown in Table 1. The original PCB defect images were preprocessed as follows: First, the original color PCB images were converted to single-channel grayscale. Then, a 3×3 median filter was applied to each grayscale image to remove salt-and-pepper noise. Subsequently, the filtered pixels were linearly mapped to the [0, 1] interval. Finally, the image size was standardized to 256×256 pixels to obtain a preprocessed image with a uniform format.

[0037] Table 1. Sampling statistics of defect classes in the HRI PCB dataset.

[0038]

[0039] S2: Perform a self-supervised pre-training task on the preprocessed image to obtain the feature extraction network.

[0040] Self-supervised pre-training tasks utilize masked autoencoder (MAE) models to perform mask reconstruction tasks, such as... Figure 2 As shown, the mask autoencoder uses a vision transformer as the core backbone of the encoder-decoder.

[0041] The visual transformer backbone architecture consists of an image segmentation and embedding module, a positional encoding layer, a multi-layer Transformer encoder, and a classification head. The multi-layer Transformer encoder comprises alternating stacks of multi-head self-attention layers and feedforward network layers. The Transformer encoder derives feature representations at shallow, mid, and deep layers, and performs channel alignment and spatial reshaping on the derived features to obtain multi-scale feature maps at three resolutions. Figure 1 On one hand, the input conditional diffusion network is used as the vector for cross-attention, and on the other hand, the input feature pyramid network is used by the anchorless detection head to achieve unified feature-driven generation and detection.

[0042] The multi-head self-attention layer uses a query-key-value mechanism and a scaled dot product attention mechanism to calculate the relevance weights between tokens, allowing each token to exchange information with other visible tokens in the sequence and aggregating the global context. The scaling dot product attention calculation formula is expressed as follows:

[0043]

[0044] Where Attention(·) represents the attention function, which generates weighted feature output. These represent the query, key, and value matrices, respectively, derived from the input sequence. Learnable weights Generated by linear transformation, where L represents the sequence length and d represents the feature dimension. This indicates a single-head dimension, where h represents the number of attention heads. This represents the scaled dot product, and softmax(·) represents normalizing the click result to generate attention weights.

[0045] The mask reconstruction task of the mask autoencoder includes: dividing the preprocessed PCB defect image into several image blocks of the same size; randomly occluding the original image area; reconstructing the occluded content using a visual transformer encoder-decoder; and using the reconstruction mean square error loss as the loss.

[0046] In this embodiment, the mask ratio of the mask autoencoder is adjusted using a dual adaptive strategy, and the specific steps are as follows:

[0047] ①Global reference mask rate ρ base,k As the training period k gradually increases, the formula is expressed as:

[0048] ρ base,k =0.40+(0.75-0.40)·[1-exp(-k / 15)]

[0049] The initial mask rate is 0.40, the progressive target mask rate is 0.75, and the decay constant of the training period k is 15.

[0050] ②When reconstruction error If the price continuously drops below a preset threshold, and if the following conditions are met... The current mask rate Δρ = 0.03 will be automatically increased, and the upper limit of the mask rate is 0.80.

[0051] ③ Maintain full mask for pixels in the significant defect region p(x) ≥ 0.5. For background regions where p(x) < 0.5, only about half the mask rate is used.

[0052] The specific process of visual transformer encoder-decoder reconstructing obscured content includes:

[0053] Encoder Feature Extraction: Visible Image Patch Embedding Sequence z i =E·flatten(patch) i )+b, dimension d; add position encoding p i Enter Lenc The layered visual transformer encoder (number of heads h, feedforward dimension typically 4d) outputs feature representations corresponding to visible image patches through the multi-head self-attention mechanism and feedforward network of the visual transformer.

[0054] Decoder reconstructs occluded content: for N mask The location of the masked image patch is inserted with f j Same-dimensional learnable mask label m k ; merge {f j} and {m k}, restore length N; add position encoding to all tokens, input L dec A lightweight visual transformer decoder; the decoder infers missing content and generates a reconstructed image through a multi-head self-attention mechanism.

[0055] After self-supervised pre-training is completed, the decoder module and classification head related to the reconstruction task are discarded, and only the visual transformer encoder part is retained to form the feature extraction network.

[0056] S3: Based on the conditional diffusion model, the feature extraction network outputs feature and defect type control vector to generate synthetic defect PCB images and their labels.

[0057] like Figure 3 As shown, the U-Net structure used in the conditional diffusion model consists of a symmetrical encoder and decoder, as well as a bottleneck layer. The encoder consists of multiple convolutional modules, each of which extracts multi-scale features of the image through downsampling operations. The decoder consists of upsampling modules of the corresponding number of layers, which fuse the features of the corresponding layers of the encoder to reconstruct image details. The bottleneck layer connects the encoder and decoder and aggregates global semantics.

[0058] U-Net incorporates a cross-attention mechanism into its bottleneck and decoding layers to introduce conditional information. In this embodiment, the defect feature vector output by the pre-trained visual transformer... After linear transformation (K = fW) K , After dimensional expansion, the feature maps are input as conditional key-value vectors into the multi-head attention module of U-Net; simultaneously, the feature maps of the current layer of U-Net are... The vector is mapped by a 1×1 convolution and flattened into a query vector. Also classified as Q i Then, cross-attention calculation is performed on each attention head to update the U-Net features according to the defect vector, ensuring that the generated image reflects the specified defect features at all scales of detail.

[0059] The U-Net encoder, bottleneck layer, and decoder are connected in series with Sigmoid-gated residual units after each stage of cross-attention. These gated residual units inject residuals into the cross-attention outputs, as shown below:

[0060] Z out =Z in +g·CA(Z in ,T),

[0061] g=σ(FC(avg(Z in ))-2),

[0062] Among them, Z in This represents the feature map at the current scale, CA(·,T) represents the cross-attention computation with the pre-trained visual transformer token T as the key, g∈(0,1) represents the dynamic gating coefficient used to adjust the feature injection intensity of the visual transformer, and avg(·) represents the feature map at the current scale. in Global average pooling in the spatial dimension, FC(·) represents a single-layer fully connected mapping, σ(·) represents the Sigmoid function, and a fixed bias of -2 is used to push the gating initial value to 0 to suppress early noise.

[0063] like Figure 4 As shown, the image generation process of conditional diffusion includes a forward diffusion stage and a backward diffusion stage. In the forward diffusion stage, defect prior modulation noise scheduling is used, that is, the noise step size β'(t,x) is expressed as:

[0064] β'(t,x)=β lin (t)·[1+0.2·p(x)]

[0065] Where β'(t,x) represents the pixel-level noise step size during the forward diffusion process, which depends on the time step t and the pixel position x,β lin (t) represents the time-based linear noise step size scheduling function, p(x) represents the defect saliency probability generated by the self-supervised transformer, and 0.2 is the optimal coefficient determined by grid search.

[0066] For each step t, calculate the retention factor α. t =1-β t Cumulative coefficient This allows generating any noisy image X from X0. t After t-step diffusion, image X t It approximately follows a standard normal distribution The original image information is almost completely covered by noise, resulting in a noisy image sequence {X1,X2,…,X}. T} and the corresponding true noise {∈1,∈2,…,∈ T}, used for backdiffusion training.

[0067] Back diffusion stage from random noise image Initially, the U-Net model progressively predicts the denoised image and noise components based on the defect feature vector conditions, generating an output that closely approximates the original defect image. The specific steps are as follows:

[0068] ① Initialize the noisy image: Sample the initial noisy image from a standard normal distribution. Input is the defect feature vector C generated by a pre-trained visual transformer;

[0069] ②U-Net noise prediction: At each step t∈{T,T-1,…,1}, the current noisy image X is predicted. t The defect feature vector is input into U-Net, which then fuses conditional information through a cross-attention mechanism to predict the noise component. Indicates from X t To X t-1 The direction of noise reduction;

[0070] ③ Calculate the denoised image: using the predicted noise component and noise scheduler parameter β t The next image is calculated based on the denoising formula, which is expressed as:

[0071]

[0072] in, X represents the denoised image at time step t-1. t This indicates the current noisy image. Indicates the proportion of control signals retained. Indicates control of noise intensity. The standard deviation of the noise component. Represents the noise prediction component. This represents random Gaussian noise;

[0073] ④ Iterative generation of the final image: Iterate from t=T to t=1, gradually updating image X. t-1 This continues until the final defect image X0 is generated.

[0074] U-Net optimizes the noise prediction loss to ensure that the predicted noise closely approximates the actual noise during forward diffusion. The noise prediction loss is expressed as:

[0075]

[0076] in,

[0077]

[0078] in, This represents the noise prediction loss, and x0 represents the defect-free PCB image. Let ε represent random noise, t represent the diffusion time step, c represent the condition vector composed of defect class encoding, probability heatmap, and multi-scale features, and ε represent the multi-scale feature vector. θ x represents the predicted noise value of the U-Net denoising network. t This represents the noisy PCB image at the t-th diffusion time step. Represents the forward diffusion cumulative coefficient. Let ||·||2 represent the expectation, and let L2 norm represent the L2 norm.

[0079] After completing the training of the conditional diffusion model in S3, the quality and diversity of multiple candidate models are evaluated to select the best performing model. The image preprocessed in step S1 is then input into the best model to synthesize the defect image, generating a synthesized defective PCB image and its label.

[0080] S4: Mix the synthesized defect images with a small number of real defect images to construct a training set.

[0081] The synthetic defect images generated by the conditional diffusion model are mixed with a small number of real-world, collected and labeled defect images at a preset ratio to construct a training set.

[0082] S5: Use the training set to train and adjust the defect detection model to obtain a well-trained defect detection model.

[0083] The defect detection model of this invention consists of a feature extraction backbone network, a feature pyramid fusion module, and an anchorless detection head.

[0084] like Figure 5 As shown, the feature extraction backbone network adopts a pre-trained visual transformer feature extraction network, freezes shallow weights to retain general features, and fine-tunes deep weights to adapt to the defect detection task; the feature pyramid fusion module adopts a feature pyramid network to fuse feature maps of different scales to improve the detection capability of multi-scale defects; the aimless detection head outputs the confidence of the defect category, the four-way bounding box offset, and the feature vector that enhances the category discrimination.

[0085] This invention uses a feature pyramid network to generate multi-scale feature maps from the visual transformer backbone. The specific process is as follows:

[0086] ① Multi-scale feature map generation: Take the token sequence output from the 12th layer of the encoder and reshape it to obtain P3 (16×16×256); apply a deconvolution (transposed convolution, kernel size 4×4, stride 2, padding 1) + batch normalization + ReLU activation to P3, and upsample to obtain P2 (32×32×256); perform the same deconvolution + batch normalization + ReLU activation on P2 again, and upsample to obtain P1 (64×64×256);

[0087] ② Feature Pyramid Fusion (FPN): This process fuses the feature maps of the three scales mentioned above in a top-down manner. Bilinear upsampling is applied to P3, adjusting it to 32×32, and then added to P2 to obtain P2' (Formula: P2' = P2 + Upsample(P3)). Bilinear upsampling is then applied again to P2', adjusting it to 64×64, and then added to P1 to obtain P1' (Formula: P1' = P1 + Upsample(P2')). Finally, the fused feature maps P1', P2', and P3' are output, and smoothed using a 3×3 convolution to unify the number of channels C = 256 (Formula: P...). i '=Conv 3×3 (P i '), i = 1, 2, 3).

[0088] On each fused multi-scale feature map, an anchorless detection head is attached to output the confidence of the defect category, the four-way bounding box offset, and the feature vector that enhances the category discrimination. Specifically, the anchorless detection head sets up a classification branch, a regression branch, and a category discrimination branch in parallel on each fused multi-scale feature map, and the three are weighted and summed according to specific weights to form the total loss of the detection head.

[0089] In this embodiment, the original Focal Loss is improved by adopting QualityFocalLoss (QFL) for classification loss. QFL introduces a modulation factor based on the difference between the predicted probability p and the target y on the basis of binary cross-entropy, which is more sensitive to small target defects and improves the recall and precision of detection. The improved classification loss is defined as follows:

[0090] L QFL (p,y)=|yp| β ·[-ylogp-(1-y)log(1-p)],

[0091] Where p = σ(x) represents the prediction confidence (normalized to 0-1 by the sigmoid function), and β represents the focusing factor (β = 2 is optimal in this experiment); QFL adjusts the classification target of positive samples from a fixed value "1" to the LOU quality score between the predicted bounding box and the true bounding box, generating continuous soft labels y∈[0,1]; when the predicted value p is close to the true value y, the modulation factor tends to 0, reducing the loss; for negative samples (y = 0), when the model correctly predicts the background (p≈0), the loss is significantly suppressed.

[0092] In this embodiment, EfficientloU Loss (EloU) is used for the regression loss. By minimizing the difference in width and height between the predicted bounding box and the ground truth bounding box in addition to the traditional loU and center distance, the regression of small-sized and slender boxes is more finely constrained, thus improving positioning accuracy. The improved regression loss is defined as:

[0093]

[0094] The loU loss term is represented as follows: A represents the ground truth bounding box, B represents the predicted bounding box, and ρ 2 (·,·)=(x2-x1) 2 +(y2-y1) 2 b = (x) represents the squared Euclidean distance between two points; b ,y b ) represents the coordinates of the center point of the prediction box, b gt =(x gt ,y gt () represents the coordinates of the predicted points of the truth box, and c represents the diagonal length of the smallest bounding rectangle containing both the predicted and truth boxes. The value represents the center point distance penalty; ω represents the prediction box width, w gt C represents the width of the truth box. ω This represents the width of the smallest bounding rectangle containing the prediction box and the truth box. The value represents the width difference penalty; g represents the height of the prediction box, and h represents the width difference penalty. gt C represents the height of the truth box. h This represents the height of the smallest bounding rectangle containing the prediction box and the truth box. This indicates a penalty for significant differences.

[0095] This invention, through optimization using Quality Focal Loss and Efficient-IoU, reduces the average error in locating minute defects by approximately 31% and improves the F1 score for detecting extremely long and thin defects (aspect ratio > 5) by 14%.

[0096] In this embodiment, for the category discrimination loss, a Softmax loss with margin is adopted, introducing an additive angular margin m in the classification branch to enhance the separability of features of different defect categories; for the feature vector x i and category weight w j Perform L2 regularization and scaling to make Where θ j Let y be the angle, and s be the scaling factor; for the correct category y i Before calculating the softmax, a fixed angle margin m is added, and the loss is defined as:

[0097]

[0098] Where N represents the number of samples in the batch, θ represents the angle between the feature vector of the i-th sample and its true class weight vector; the smaller the angle, the more accurate the classification. j The angle between the feature vector of the i-th sample and the weight vector of the informal class j is represented. The larger the angle, the more obvious the distinction between the feature and the incorrect class.

[0099] This experiment selected m = 0.2 radians and s = 30 to balance the optimization difficulty and the improvement of discriminative power. ArcFace optimized the overfitting problem and improved the generalization ability by optimizing regularized features and weights as well as inter-class intervals.

[0100] The total loss function is expressed as follows:

[0101] L total =0.303L QFL +0.545L EIoU +0.152L ArcFace

[0102] In this embodiment, the weight ratio of the three losses was finally determined to be 1:1.8:0.5 through grid search and ablation experiments to achieve multi-objective joint optimization.

[0103] During training, a phased optimization training strategy is adopted. Specifically, the first 8 layers of the visual transformer are frozen, and only high-level features, FPN, and detection head are trained. The Adam optimizer is used with an initial learning rate of 0.001 for a total of 20 epochs. All layers are unfrozen, the backbone learning rate is reduced to 0.0001, while the FPN and detection head remain at 0.001, and training continues for another 30 epochs. Finally, a well-trained PCB defect detection model is obtained.

[0104] This invention uses NVIDIA GeForce RTX 3090 for deployment training and inference testing. The comprehensive performance comparison results of this invention and typical comparative models are shown in Table 2. When using FP32 precision and batch size=1 for single-card inference, the measured speed is consistent with that shown in the table below.

[0105] Table 2. Comparison of the overall performance of this invention with typical comparative models.

[0106]

[0107] As can be seen from Table 2, the present invention outperforms existing models in terms of detection accuracy indicators such as mAP50:95, AP@0.5, and Recall@10 (%).

[0108] In summary, through the verification of the above embodiments, the present invention can significantly improve the performance of the detection model when the number of defect samples is limited. Self-supervised pre-training and the improved detection network based on a visual transformer jointly enhance the model's ability to identify defects, and the synthetic defect samples generated by the diffusion model effectively expand the training data space, demonstrating promising application prospects.

[0109] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be noted that the present invention is not limited to the above embodiments. Without departing from the inventive concept, those skilled in the art can make various modifications and improvements, which should also be considered within the scope of protection of the present invention.

Claims

1. A PCB defect detection method based on visual transformer combined with conditional diffusion, characterized in that, The method comprises the following steps: S1: Constructing an unlabeled PCB image dataset and performing data preprocessing and enhancement to obtain preprocessed images; S2: Performing a self-supervised pre-training task on the preprocessed images to obtain a feature extraction network; S3: Based on a conditional diffusion model, using the features output by the feature extraction network and a defect type control vector to generate synthetic defect PCB images and their labels; S4: Mixing the synthetic defect images and a small amount of real defect images to construct a training set; S5: Training and adjusting the defect detection model using the training set to obtain a trained defect detection model.

2. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 1, wherein, The data preprocessing includes denoising, normalization and size standardization preprocessing of the defect-free PCB images.

3. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 1, characterized in that, The feature extraction network is a visual transformer backbone architecture, which is composed of an image patching and embedding module, a position encoding layer and a multi-layer Transformer encoder; wherein the Transformer encoder derives feature representations at the shallow, middle and deep layers, respectively, and performs channel alignment and spatial remodeling on the derived features to obtain multi-scale feature maps of three resolutions; the multi-scale feature maps are input into the conditional diffusion network as vectors for cross-attention on the one hand, and into the feature pyramid network for anchor-free detection head on the other hand, realizing unified feature driving in the generation and detection stages.

4. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 1, characterized in that, The self-supervised pre-training task is a mask reconstruction task based on a mask autoencoder, which includes: dividing the preprocessed PCB defect image into a plurality of image blocks of the same size; randomly masking the original image area; reconstructing the masked content using a visual transformer encoder-decoder; and using a reconstruction mean square error loss for loss.

5. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 4, wherein, The mask proportion of the mask self-encoder adopts a double adaptive strategy adjustment, including: a global reference mask rate p base,k gradually increases with the training period k; when the reconstruction error drops below the preset threshold, the current mask rate is automatically adjusted again; the pixels in the defect significant area are kept full mask, and the pixels in the background area are only used about half of the mask rate.

6. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 1, wherein, The U-Net structure of the conditional diffusion model is composed of a symmetrical encoder and decoder and a bottleneck layer, wherein: the encoder is composed of a plurality of convolution modules, each of which extracts multi-scale features of the image through downsampling operations; the decoder is composed of a corresponding number of upsampling modules, which fuse the features of the corresponding layers of the encoder to reconstruct the image details; and the bottleneck layer connects the encoder and the decoder to aggregate global semantics.

7. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 6, wherein, After the cross-attention of the U-Net encoder, the bottleneck layer and the decoder at each level, a defect gate residual unit is connected in series, which performs global average pooling on the current feature map, inputs the Sigmoid after a single-layer full connection and a fixed bias, and obtains a gating coefficient; then the coefficient is weighted and cross-attention output is injected in the form of a residual to suppress non-defect token noise and enhance defect area representation.

8. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 6, wherein, The image generation process of the conditional diffusion model uses defect priori modulation noise scheduling in the forward diffusion stage, that is, the noise step length β'(t,x) is represented as: β'(t,x) = β lin (t) · [1 + 0.2 · p(x)] where β'(t, x) represents the pixel-level noise step size in the forward diffusion process, dependent on time step t and pixel position x, β lin (t) represents the time-based linear noise step size scheduling function, p(x) represents the defect saliency probability generated by the self-supervised transformer, and 0.2 is the optimal coefficient determined by grid search.

9. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 1, wherein, The defect detection model is composed of a feature extraction backbone network, a feature pyramid fusion module and an anchor-free detection head; the feature extraction backbone network adopts a visual transformer feature extraction network, freezes shallow weights to retain general features, and fine-tunes deep weights to adapt to the defect detection task; the feature pyramid fusion module adopts a feature pyramid network, and fuses feature maps of different scales to improve the detection capability of multi-scale defects; the anchor-free detection head outputs the confidence of the defect category, the four-way boundary box offset and the feature vector for enhancing the category discrimination.

10. The PCB defect detection method based on visual transformer combined with conditional diffusion according to claim 9, wherein, The anchor-free detection head is provided with a quantization focal loss, an efficient loU loss and a category discrimination loss in parallel on each layer of the fused multi-scale feature map, and the three are weighted and summed according to specific weights to form a total loss of the detection head.

Images