PCB weak scratch detection method based on attention twin network
By using attention twin networks and data augmentation technology, the problems of feature annihilation and generalization ability in PCB minor scratch detection are solved, achieving high-precision and robust automated detection and reducing deployment costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2025-12-23
- Publication Date
- 2026-05-01
AI Technical Summary
Existing PCB defect detection technologies suffer from problems such as feature annihilation, weak generalization ability, uninterpretable models, and high deployment costs when dealing with minor scratches, making it difficult to achieve high-precision and robust automated detection.
A detection method based on attention twin networks is adopted. Images are acquired through a high-definition camera, and a twin network structure is constructed by combining data augmentation and triple loss function. Convolutional block attention mechanism and spatial attention module are introduced for feature extraction and judgment. Grad-CAM is used to verify the model decision process.
It significantly improves the sensitivity to identify subtle scratches, enhances the model's generalization ability and detection reliability, reduces deployment and maintenance costs, and achieves high-precision automated detection.
Smart Images
Figure CN121962021A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of industrial automation inspection technology, specifically relating to a method for detecting minor scratches on PCBs based on attention twin networks. Background Technology
[0002] In the field of automated defect detection of printed circuit boards (PCBs), convolutional neural network-based methods have become mainstream, but they still face serious challenges in dealing with weak, low-contrast scratch defects.
[0003] Existing technologies mainly focus on several independent directions: for example, improving the detection rate of minute defects by refining target detection architectures such as YOLO, but such methods struggle to overcome the feature annihilation problem in deep networks; or using generative adversarial networks (GANs) for data augmentation to alleviate sample scarcity, but the quality and diversity of generated samples are limited; some studies explore novel architectures such as Transformers or focus on lightweight model design, but the former has high computational complexity, while the latter often encounters accuracy bottlenecks. Furthermore, although Siamese networks provide a mature approach to defect detection with their comparison learning paradigm, and attention mechanisms have been proven to effectively enhance the ability to focus on key features, existing technologies mostly explore these two as independent paths. Few studies systematically integrate the comparison advantages of Siamese networks with the feature enhancement capabilities of attention mechanisms to specifically address the core challenge of insufficient sensitivity caused by feature annihilation in PCB minor scratch detection. This results in a lack of a precise and robust solution for this specific problem in existing technologies.
[0004] The following are the specific problems with existing PCB defect detection methods:
[0005] (1) In PCB defect detection, traditional convolutional neural networks (CNNs) suffer from low-contrast, pixel-level, subtle scratch features due to their inherent layer-by-layer downsampling operation, which leads to the loss of these features in deep networks (i.e., the "feature annihilation" phenomenon). This results in insufficient sensitivity of the model to identify such fine defects, leading to a high false negative rate. Existing methods struggle to effectively enhance and preserve these subtle target signals in complex background textures.
[0006] (2) Existing supervised learning models rely heavily on a large number of labeled defect samples for training, making it difficult to adapt to the actual situation of diverse PCB product models and varied defect morphologies, resulting in weak generalization ability. At the same time, the models are easily affected by environmental interference such as changes in lighting and workpiece displacement on the production line, leading to frequent false detections and missed detections. The reliability and stability of the detection results cannot meet the requirements of high-precision industrial quality inspection.
[0007] (3) Deep learning models are often treated as "black boxes," and their decision-making logic is difficult to interpret. Engineers cannot verify whether the model is truly making judgments based on defect features rather than irrelevant noise, which reduces the credibility of the results. In addition, many high-precision models are computationally complex and require frequent manual parameter adjustments to adapt to different production lines, resulting in high deployment and maintenance costs, making it difficult to apply them on a large scale in industrial online inspection where efficiency and cost are pursued. Summary of the Invention
[0008] The purpose of this invention is to provide a PCB minor scratch detection method based on attention twin network, so as to enhance the model's ability to perceive the features of minor scratches on the PCB surface and achieve high robustness and high accuracy of defect identification.
[0009] This invention is achieved through the following technical solution:
[0010] A method for detecting subtle scratches on PCBs based on attention twin networks includes the following steps:
[0011] S1 Collects raw defect images of PCBs on the production line using a high-definition camera; S2 Annotates and augments the raw defect images collected in step S1 to complete the data preprocessing of the raw defect images; S3 Feature extraction; S4 Model training and deployment.
[0012] The collection of original defect images in step S1 above specifically includes the following steps:
[0013] S1-1 Image Acquisition Steps: Use an industrial camera or line scan camera to take high-resolution pictures of the PCB surface; acquire raw image data under multiple angles and lighting conditions so that even minute defects can be completely recorded;
[0014] The S1-2 sample construction steps include two types of samples:
[0015] Positive samples, as defective image samples: containing thin, low-contrast, faint scratches;
[0016] Negative samples, as defect-free image samples: standard PCB samples with intact surfaces and no scratches;
[0017] S1-3 After manually labeling and screening the samples collected in step S1-2, they are divided into training set, validation set and test set according to the proportion to ensure the generalizability and objectivity of the model.
[0018] The data preprocessing described in step S2 above specifically includes the following steps:
[0019] S2-1 All sample images from both types in steps S1-2 are uniformly scaled to a fixed resolution to meet the input requirements of the deep network, thereby completing image normalization and size standardization processing.
[0020] S2-2 Image Augmentation Strategy: To improve the robustness of the model to different production scenarios, online data augmentation is used for data preprocessing;
[0021] After manually annotating the defect images in S2-3, a series of data augmentation samples and defect feature enhancement samples were obtained using geometric transformation and image processing methods for large model training.
[0022] S2-4 Sample Triple Construction: The system automatically constructs triplet samples before model training;
[0023] A "triple pair" consists of the following three images:
[0024] Anchor sample (Anchor,xa): An arbitrary baseline image;
[0025] Positive sample (xp): An image belonging to the same category as the anchor sample;
[0026] Negative sample (x): An image that belongs to a different category than the anchor sample.
[0027] The feature extraction described in S3 above specifically includes the following steps:
[0028] S3-1 twin network framework:
[0029] A dual-branch Siamese network structure is adopted. The backbone feature extraction network consists of two weight-shared improved ResNet34 branches: convolutional block attention modules are embedded in each residual block, including: a channel attention module, which is used to evaluate the importance weights of different feature channels, thereby enhancing the feature response related to scratches; and a spatial attention module, which is used to analyze the saliency of different spatial locations in the feature map and focus attention on potential defect areas.
[0030] S3-2 Feature Distance Calculation: The model calculates the feature difference between the standard sample and the test sample using Euclidean distance, defined as:
[0031] D = ||f(x) test )-f(x std )||2
[0032] Where f(x) is the feature extraction function, which maps the input samples to the feature space, x test It is the sample to be tested, x std It is a standard sample;
[0033] S3-3 Defect Judgment Rule: Set a distance threshold τ; when D>τ, it is judged as u. n A sample containing "defective samples" is considered a "defect-free sample" if it does not contain any defects.
[0034] S3-4 Triplet Loss Function: To train a feature space with high discriminative power, a triplet loss function is used to constrain model learning.
[0035] L triplet =max(0,||f(x) a )-f(x p )|| 2 -||f(x a )-f(x n )|| 2 +α)
[0036] Where f(x) is the feature vector generated by the feature extraction network for the input image x;
[0037] ||.|| 2 This represents the square of the Euclidean distance between two vectors;
[0038] α is a hyperparameter, called the boundary, which requires that the distance between negative sample pairs must be at least α greater than the distance between positive sample pairs;
[0039] By minimizing L triplet The loss function forces the model to learn a high-quality feature space: in this space, the feature vectors of all “scratched” samples will cluster together to form a tight cluster; while the feature vectors of all “defect-free” samples will cluster in another region, and the two clusters will maintain a safe distance of at least α between them.
[0040] The model training and deployment described in step S4 above specifically includes the following steps:
[0041] S4-1 Training Strategy: Use the AdamW optimizer and CosineAnnealingLR learning rate scheduling strategy to ensure stable convergence; introduce an online hard example mining mechanism to dynamically select the "most difficult to distinguish" sample triples in each batch to improve learning efficiency; the number of training rounds can be set to more than 100 until the validation set performance converges.
[0042] S4-2 Performance Evaluation: After the model training is completed, a comprehensive evaluation is conducted based on accuracy, precision, recall, and F1 score.
[0043] S4-3 Deployment Method: The trained model is exported as a deployable file through a deep learning framework, supporting real-time operation on inspection equipment in industrial production lines to achieve automated scratch detection;
[0044] S4-4 Visual Validation: The interpretability method can generate heatmaps to visually display the model's areas of interest, validating the effectiveness of the attention mechanism in scratch feature localization.
[0045] In the above steps S2-4, when constructing sample triples, if the anchor point is an image with scratches, then the positive sample is another different image with scratches, and the negative sample is an image without defects.
[0046] The online data augmentation methods described in step S2-2 above include: random horizontal or vertical flipping; random rotation at small angles; brightness, contrast, and saturation jittering; and local cropping and noise reduction enhancement. Data augmentation not only increases sample diversity but also effectively prevents model overfitting.
[0047] The interpretability methods for step S4-4 above include using Grad-CAM.
[0048] Compared with the prior art, the present invention has the following advantages and effects:
[0049] (1) This invention introduces a convolutional block attention mechanism (CBAM) into the feature extraction network, which can adaptively enhance the response to low-contrast detail features such as weak scratches, effectively suppress background noise interference, overcome the "feature annihilation" problem in traditional convolutional neural networks where subtle features are easily lost, and thus significantly improve the model's sensitivity to fine defects.
[0050] (2) This invention adopts a twin network structure with two branches and shared weights to realize a defect comparison and judgment method based on feature similarity. This structure does not depend on specific defect category labeling, has good generalization and transferability, and can adapt to the inspection needs of PCB products of different models and batches.
[0051] (3) By introducing a metric learning strategy based on the triplet loss function, this invention establishes a feature space with high discriminative power, enabling the model to effectively distinguish between defective samples and normal samples, reducing false detections and false negatives, and improving the reliability and stability of detection.
[0052] (4) The detection process of the present invention does not require complex parameter adjustment or environmental calibration, and has strong robustness to external interference such as changes in illumination and image noise, making it suitable for automated detection scenarios in industrial production lines.
[0053] (5) This invention utilizes Grad-CAM to visually display the area of concern of the model in the decision-making process, and verifies that the model can focus on the key parts where the scratch is located, and has good interpretability and engineering verifiability. Attached Figure Description
[0054] Figure 1 This is a system flowchart of the present invention.
[0055] Figure 2 This is a schematic diagram of the CBAM of the present invention.
[0056] Figure 3 This is a schematic diagram of the triplet loss function of the present invention. Detailed Implementation
[0057] The present invention will now be described in further detail with reference to specific embodiments.
[0058] like Figure 1-3 As shown, this invention discloses a method for detecting subtle scratches on PCBs based on attention twin networks. This invention is particularly suitable for the accurate detection of subtle scratch defects on printed circuit boards with low contrast and fine features, and can be implemented through the following steps:
[0059] S1 collects raw defect images of PCBs on the production line using a high-definition camera, specifically including the following steps:
[0060] S1-1 Image Acquisition Steps: Use an industrial camera or a line scan camera to take high-resolution pictures of the PCB surface; acquire raw image data under multiple angles and lighting conditions to ensure that minor defects such as scratches can be completely recorded.
[0061] The S1-2 sample construction steps include two types of samples:
[0062] Positive samples, as defective image samples: containing thin, low-contrast, faint scratches;
[0063] Negative samples, as defect-free image samples: standard PCB samples with intact surfaces and no scratches;
[0064] S1-3 After manually labeling and screening the samples collected in step S1-2, they are divided into training set, validation set and test set according to the proportion to ensure the generalizability and objectivity of the model.
[0065] S2 involves labeling and augmenting the original defect images acquired in step S1 to complete the data preprocessing of the original defect images. This includes the following steps:
[0066] S2-1 All sample images from both types in steps S1-2 are uniformly scaled to a fixed resolution to meet the input requirements of the deep network, thereby completing image normalization and size standardization processing.
[0067] S2-2 Image Augmentation Strategy: To improve the model's robustness to different production scenarios, online data augmentation is employed in data preprocessing, including: random horizontal or vertical flipping; random rotation at small angles (within ±10°); brightness, contrast, and saturation jittering; local cropping and noise reduction enhancement. Data augmentation not only increases sample diversity but also effectively prevents model overfitting.
[0068] After the defect images in S2-3 are manually annotated, a series of data augmentation samples and defect feature enhancement samples are obtained by using geometric transformations such as translation, transpose, mirroring, rotation, scaling, and cropping, as well as image processing methods such as noise reduction, blurring, color transformation, erasure, and filling, for use in large model training.
[0069] S2-4 Sample Triple Construction: The system automatically constructs triplet samples before model training.
[0070] A "triple pair" consists of the following three images:
[0071] Anchor sample (Anchor,xa): An arbitrary baseline image;
[0072] Positive sample (xp): An image belonging to the same category as the anchor sample;
[0073] Negative sample (x): An image that belongs to a different category than the anchor sample;
[0074] In our task, if the anchor point is an image with scratches, then the positive sample is another different image with scratches, and the negative sample is an image without defects.
[0075] S3 feature extraction specifically includes the following steps:
[0076] S3-1 twin network framework:
[0077] A dual-branch Siamese network structure is adopted, with the backbone feature extraction network consisting of two weight-sharing improved ResNet34 branches: Convolutional Block Attention (CBAM) modules are embedded in each residual block, including: a channel attention module, which is used to evaluate the importance weights of different feature channels, thereby enhancing the feature response related to scratches; and a spatial attention module, which is used to analyze the saliency of different spatial locations in the feature map and focus attention on potential defect areas. Through the above attention fusion mechanism, the network can adaptively enhance the scratch feature representation and suppress background interference, effectively alleviating the "feature annihilation" phenomenon in traditional CNN models.
[0078] In this structure, each branch is responsible for deep feature extraction of the input image, making the feature vectors of similar samples closer in the feature space, while keeping the feature vectors of different samples at a greater distance, thereby achieving high-precision defect similarity discrimination;
[0079] S3-2 Feature Distance Calculation: The model calculates the feature difference between the standard sample and the test sample using Euclidean distance, defined as:
[0080] D = ||f(x) test )-f(x std)||2
[0081] Where f(x) is the feature extraction function, which maps the input samples to the feature space, x test It is the sample to be tested, x std It is a standard sample;
[0082] S3-3 Defect Judgment Rule: Set a distance threshold τ; when D>τ, it is judged as u. n A sample containing "defective samples" is considered a "defect-free sample" if it does not contain any defects.
[0083] S3-4 Triplet Loss Function: To train a feature space with high discriminative power, a triplet loss function is used to constrain model learning.
[0084] L triplet =max(0,||f(x) a )-f(x p )|| 2 -||f(x a )-f(x n )|| 2 +α)
[0085] Where f(x) is the feature vector generated by the feature extraction network for the input image x;
[0086] ||.|| 2 This represents the square of the Euclidean distance between two vectors;
[0087] α is a hyperparameter called the margin, which requires that the distance between negative sample pairs must be at least α greater than the distance between positive sample pairs;
[0088] By minimizing L triplet The loss function forces the model to learn a high-quality feature space: in this space, the feature vectors of all "scratched" samples will cluster together to form a tight cluster; while the feature vectors of all "defect-free" samples will cluster in another region, and the two clusters will maintain a safe distance of at least α. This provides a reliable mathematical guarantee for determining defects through a simple distance threshold during the inference stage.
[0089] S4 model training and deployment specifically includes the following steps:
[0090] S4-1 Training Strategy: Use the AdamW optimizer and CosineAnnealingLR learning rate scheduling strategy to ensure stable convergence; introduce an online hard example mining mechanism to dynamically select the "most difficult to distinguish" sample triples in each batch to improve learning efficiency; the number of training rounds can be set to more than 100 until the validation set performance converges.
[0091] S4-2 Performance Evaluation: After the model training is completed, a comprehensive evaluation is conducted using accuracy, precision, recall, and F1 score.
[0092] S4-3 Deployment Method: The trained model can be exported as a deployable file through the deep learning framework, supporting real-time operation on the inspection equipment of the industrial production line to achieve automated scratch detection;
[0093] S4-4 Visualization Verification: Heatmaps can be generated using interpretable methods such as Grad-CAM to visually display the model's areas of interest and verify the effectiveness of the attention mechanism in scratch feature localization.
[0094] As described above, the present invention can be implemented well.
[0095] The implementation of the present invention is not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A method for detecting subtle scratches on a PCB based on an attention twin network, characterized in that... Includes the following steps: S1 uses a high-definition camera to collect raw defect images of PCBs on the production line; S2 involves labeling and augmenting the original defect images acquired in step S1 to complete the data preprocessing of the original defect images. S3 feature extraction; S4 model training and deployment.
2. The PCB weak scratch detection method based on attention twin network according to claim 1, characterized in that, Step S1, the collection of the original defect images, specifically includes the following steps: S1-1 Image Acquisition Steps: Use an industrial camera or line scan camera to take high-resolution pictures of the PCB surface; acquire raw image data under multiple angles and lighting conditions so that even minute defects can be completely recorded; The S1-2 sample construction steps include two types of samples: Positive samples, as defective image samples: containing thin, low-contrast, faint scratches; Negative samples, as defect-free image samples: standard PCB samples with intact surfaces and no scratches; S1-3 After labeling and filtering the samples collected in step S1-2, the samples are divided into training set, validation set and test set according to the proportion to ensure the generalizability and objectivity of the model.
3. The PCB weak scratch detection method based on attention twin network according to claim 1, characterized in that, The data preprocessing described in step S2 specifically includes the following steps: S2-1 All sample images from both types in steps S1-2 are uniformly scaled to a fixed resolution to meet the input requirements of the deep network, thereby completing image normalization and size standardization processing. S2-2 Image Augmentation Strategy: To improve the robustness of the model to different production scenarios, online data augmentation is used for data preprocessing; After the defect images are labeled in S2-3, geometric transformations and image processing are used to obtain data to expand samples and enhance defect features for large model training. S2-4 Sample Triple Construction: Before model training, the system automatically constructs triplet samples; a "triple" consists of the following three images: Anchor point sample: Any one of the base images selected by choice; Positive sample: An image belonging to the same category as the anchor sample; Negative sample: An image that belongs to a different category than the anchor sample.
4. The PCB weak scratch detection method based on attention twin network according to claim 1, characterized in that, The feature extraction described in S3 specifically includes the following steps: S3-1 twin network framework: A dual-branch Siamese network structure is adopted. The backbone feature extraction network consists of two weight-sharing improved ResNet34 branches: Convolutional Block Attention (CBAM) modules are embedded in each residual block, including: a channel attention module, which is used to evaluate the importance weights of different feature channels, thereby enhancing the feature response related to scratches; and a spatial attention module, which is used to analyze the saliency of different spatial locations in the feature map and focus attention on potential defect regions. S3-2 Feature Distance Calculation: The model calculates the feature difference between the standard sample and the test sample using Euclidean distance, defined as: D=||f(x test )-f(x std )||2 Where f(x) is the feature extraction function, x test It is the sample to be tested, x std It is a standard sample; S3-3 Defect Judgment Rule: Set a distance threshold τ; when D>τ, it is judged as u. n A sample containing "defective samples" is considered a "defect-free sample" if it does not contain any defects. S3-4 Triple Loss Function: To train a feature space with high discriminative power, a triple loss function is used to constrain model learning. L triplet =max(0,||f(x a )-f(x p )|| 2 -||f(x a )-f(x n )|| 2 +α) Where f(x) is the feature vector generated by the feature extraction network for the input image x; ||.|| 2 This represents the square of the Euclidean distance between two vectors; α is a hyperparameter, called the boundary, which requires that the distance between negative sample pairs must be at least α greater than the distance between positive sample pairs; By minimizing L triplet The loss function forces the model to learn a high-quality feature space: in this space, the feature vectors of all "scratched" samples will cluster together to form a tight cluster; while the feature vectors of all "defect-free" samples will cluster in another region, and the two clusters will maintain a safe distance of at least α between them.
5. The PCB weak scratch detection method based on attention twin network according to claim 1, characterized in that, Step S4, model training and deployment, specifically includes the following steps: S4-1 Training Strategy: Use the AdamW optimizer and CosineAnnealingLR learning rate scheduling strategy to ensure stable convergence; introduce an online hard example mining mechanism to dynamically select the "most difficult to distinguish" sample triples in each batch to improve learning efficiency; the number of training rounds can be set to more than 100 until the validation set performance converges. S4-2 Performance Evaluation: After the model training is completed, a comprehensive evaluation is conducted based on accuracy, precision, recall, and F1 score. S4-3 Deployment Method: The trained model is exported as a deployable file through a deep learning framework, supporting real-time operation on inspection equipment in industrial production lines to achieve automated scratch detection; S4-4 Visual Verification: Interpretable methods can generate heatmaps to visually display the model's areas of interest and verify the effectiveness of the attention mechanism in scratch feature localization.
6. The PCB weak scratch detection method based on attention twin network according to claim 3, characterized in that, In step S2-4, when constructing the sample triplet, if the anchor point is an image with scratches, then the positive sample is another different image with scratches, and the negative sample is an image without defects.
7. The PCB weak scratch detection method based on attention twin network according to claim 3, characterized in that, The online data augmentation methods described in step S2-2 include: random horizontal or vertical flipping; random rotation; brightness, contrast, and saturation jitter; local cropping and noise reduction enhancement.
8. The PCB weak scratch detection method based on attention twin network according to claim 5, characterized in that, Interpretability methods include the use of Grad-CAM.