PCB defect detection method based on improved Faster-RCNN

Abstract

The invention provides a PCB defect data set detection method based on an improved Faster-RCNN. The method comprises the following steps: generating a PCB defect data set; the method comprises the following steps: adding a DAS attention mechanism obtained by improving two aspects of lightweight deformable convolution and a mixed attention mechanism into an RPN module of a Faster-RCNN, constructing a feature pyramid FPN in a feature extraction module, and adding deformation regularization into a loss function of RPN and RoI Heads at the same time, and finally training the improved network to carry out target detection and identification on a real PCB defect data set. According to the method, the detection capability of a small target of PCB defects is effectively improved, the performance is improved, and the calculation cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a PCB defect detection method based on an improved Faster-RCNN. Background Art

[0002] With the rapid development of electronic technology, printed circuit boards (PCBs), as a core component of electronic products, play a vital role in providing electrical connections and supporting components. In modern industrial production, both civilian and military products are inseparable from PCBs, especially in precision industries such as aerospace, communications, and computers. Minor production defects such as open circuits, short circuits, and poor solder joints can affect device stability. To ensure product quality and reliability, PCB defect detection is crucial.

[0003] In recent years, deep learning technology has developed rapidly. More and more scholars have applied deep learning to the field of PCB defect detection to improve the accuracy and real-time performance of detection. The two-stage detection target algorithm represented by Faster-RCNN has been widely used in PCB defect detection.

[0004] The accuracy and real-time performance of PCB defect detection are directly related to the quality and efficiency of the entire production process. Given the complexity of PCB surface characteristics, the diversity of defect types, the fuzzy distinction between background and defects, and the prevalence of minor defects, most current defect detection methods still face challenges such as low detection efficiency and difficulty adapting to the diversity of defects. In PCB (printed circuit board) defect detection, the prevalence of minor defects refers to the extremely small defects that frequently appear during PCB manufacturing. Due to their small physical size and subtle morphology, these defects are easily overlooked during detection. Therefore, further research and development of efficient and accurate PCB defect detection methods is particularly important. Summary of the Invention

[0005] The purpose of the present invention is to provide a PCB defect detection method based on an improved Faster-RCNN, which can solve the problems of high cost and long time consumption in obtaining a large number of real PCB defect images in actual production, enrich the training data set, and also solve the problem of missed detections during network detection due to the large variety and small size of PCB defects, thereby achieving better detection of PCB defects.

[0006] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0007] A PCB defect detection method based on an improved Faster-RCNN, the method comprising the following steps:

[0008] S1, draws a PCB circuit diagram, sets various defects including open circuit and short circuit, and generates a PCB defect dataset based on the PCB circuit diagram;

[0009] S2, introduces the FPN module into the feature extraction module of the Faster-RCNN network to construct a feature pyramid. It fuses multi-scale features from top to bottom to identify pixel-level changes of tiny defects in the PCB circuit diagram and extract the edge and texture features of PCB defects.

[0010] S3: Insert a grouping module and an SE module at the input and output of the DAS module, respectively, to obtain an improved DAS module. The grouping module is used to group input features, with each group sharing an offset. The SE module is used to further enhance PCB defect-related features through channel attention while suppressing irrelevant channel noise. The improved DAS module is added to the RPN module of the Faster-RCNN network to reduce the activation intensity of non-critical areas, including uniform copper foil, suppress background features, and amplify the difference between PCB defects and surrounding areas through the attention mechanism, enhancing contrast features and separating the background from the PCB defects.

[0011] S4, adding deformation regularization to the loss of the RPN module and the RoI detection head to obtain the improved Faster-RCNN network model, and training the Faster-RCNN network model;

[0012] S5, inputting the defective PCB circuit diagram into the trained Faster-RCNN network model to identify the location and category of the PCB defects contained therein.

[0013] Step S1 further comprises:

[0014] Use Altium Designer software to draw and edit the PCB circuit board, set various defects including open circuits and short circuits, and make it into slices, perform image amplification, and generate a PCB defect data set;

[0015] The image-augmented PCB defect dataset is divided into a training set and a validation set, and publicly available defective PCB circuit diagrams are obtained as a test set.

[0016] Furthermore, in step S1, the process of image amplification includes: performing mirror flipping, rotation, brightness and contrast enhancement processing on the PCB images in the PCB defect dataset to obtain an image augmented dataset.

[0017] Step S2 further comprises:

[0018] Select the outputs of the C2, C3, C4, and C5 layers from the Backbone module as the input of the FPN module;

[0019] Starting from the high-level feature map C5, high-resolution feature maps are gradually generated through upsampling;

[0020] The bottom-up feature map is fused with the top-down feature map to generate the final multi-scale feature map.

[0021] Furthermore, in step S3, the RPN module of the improved Faster-RCNN network model includes a feature input layer, a 3×3 convolutional layer, a grouping module, a depthwise separable convolutional layer, an instance normalization layer, a deformable convolutional layer, a layer normalization layer, an activation layer, a fusion layer, a SE module, a classification branch, and a regression branch;

[0022] The 3×3 convolutional layer extracts features from the feature map received by the feature input layer after being processed by the Backbone module to obtain the RPN feature map F∈R H×W×C The grouping module divides the input feature map into several groups along the channel dimension, where the number of channels is an integer multiple of the number of groups. Each group predicts only one set of offsets, and the number of groups matches the offsets so that the same offsets are shared within each group. The RPN feature map F of the shared offset passes through the depthwise separable convolution layer, the instance normalization layer, the deformable convolution layer, the layer normalization layer, and the activation layer in sequence, focusing on six significant areas in the PCB circuit diagram, including missing holes, mouse bites, open circuits, short circuits, strays, and pseudo copper, and calculating pixel-level weights to generate a spatial attention map A∈R H×W ;

[0023] The fusion layer multiplies the spatial attention map A and the RPN feature map F element by element to obtain the weighted feature map F';

[0024] The SE module processes the weighted feature map F' to obtain a channel-attention-weighted output feature map. The SE module adaptively learns the weight of each channel by introducing Squeeze and Excitation operations, focusing on more important feature channels, enhancing high-level features of PCB defects including shape and texture, while suppressing irrelevant channel noise. The output feature map is processed by the classification branch and the regression branch respectively.

[0025] The classification branch performs binary classification on each anchor point to determine whether it contains a PCB defect target, and calculates the probability of the target and the background through the Softmax function;

[0026] The regression branch performs bounding box regression on the anchor point containing the target, adjusts the position and size of the candidate box to make it closer to the real target, and generates the final candidate region based on the classification and regression results.

[0027] Furthermore, in step S4, adding deformation regularization to the loss of the RPN module and the RoI detection head means that,

[0028] Add the L2 constraint of the offset to the loss function: L offset =λ‖Δp‖2, where λ is the balance coefficient.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] First, the PCB defect detection method based on the improved Faster-RCNN of the present invention generates PCB images with different defects through Altium Designer software, which can solve the problem of insufficient data and greatly enrich the training data set.

[0031] Second, the PCB defect detection method based on the improved Faster-RCNN of the present invention uses the DAS attention mechanism, which is improved from the lightweight deformable convolution and hybrid attention mechanism, to dynamically capture important spatial areas in the image, enhance the model's attention to the key area of PCB defects, further reduce the number of model operations and floating-point operations, and improve the speed of PCB defect detection; construct a feature pyramid FPN in the feature extraction module to improve the detection capability of small targets such as PCB defects; at the same time, add deformation regularization to the loss functions of RPN and RoI Heads to reduce the capture of background noise and reduce the false detection rate; ultimately, the effect of identifying and locating real PCB defects is achieved by using a network trained with simulated PCB defect data samples.

[0032] Third, the PCB defect detection method based on the improved Faster-RCNN of the present invention reduces the dependence on real PCB defect data samples, solves the data shortage problem, and achieves the effect of identifying and locating real PCB defects using a network trained with simulated PCB defect data samples. In addition, the improved model enhances the focus on PCB defects, effectively improving the detection capability of small targets such as PCB defects, improving performance and reducing computational costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a flow chart of the PCB defect dataset detection method based on the improved Faster-RCNN of the present invention;

[0034] Figure 2 This is the improved RPN module architecture diagram;

[0035] Figure 3 This is the architecture diagram of the feature extraction module after adding the FPN module;

[0036] Figure 4 Schematic diagram of the experimental results; (a) is the detected short defect result diagram, (b) is the detected missing_hole defect result diagram, (c) is the detected spurious_copper defect result diagram, (d) is the detected open_circuit defect result diagram, (e) is the detected mouse_bite defect result diagram, and (f) is the detected spur defect result diagram. DETAILED DESCRIPTION

[0037] The embodiments of the present invention are described in further detail below with reference to the accompanying drawings.

[0038] The present invention discloses a PCB defect detection method based on an improved Faster-RCNN, the method comprising the following steps:

[0039] S1, draws a PCB circuit diagram, sets various defects including open circuit and short circuit, and generates a PCB defect dataset based on the PCB circuit diagram.

[0040] S2, introduces the FPN module into the feature extraction module of the Faster-RCNN network to construct a feature pyramid. It fuses multi-scale features from top to bottom to identify pixel-level changes of tiny defects in the PCB circuit diagram and extract the edge and texture features of PCB defects.

[0041] S3: Insert a grouping module and an SE module at the input and output of the DAS module, respectively, to obtain an improved DAS module. The grouping module is used to group input features, with each group sharing an offset. The SE module is used to further enhance PCB defect-related features through channel attention while suppressing irrelevant channel noise. The improved DAS module is added to the RPN module of the Faster-RCNN network to reduce the activation intensity of non-critical areas, including uniform copper foil, suppress background features, and amplify the difference between PCB defects and surrounding areas through the attention mechanism, enhancing contrast features and separating the background from the PCB defects.

[0042] S4, adding deformation regularization to the loss of the RPN module and the RoI detection head to obtain the improved Faster-RCNN network model, and training the Faster-RCNN network model;

[0043] S5, inputting the defective PCB circuit diagram into the trained Faster-RCNN network model to identify the location and category of the PCB defects contained therein.

[0044] Figure 1This is a flow chart of the PCB defect dataset detection method based on the improved Faster-RCNN of the present invention. In this embodiment, a circuit board is first drawn using Altium Designer, and various defects such as shorts and opens are edited and set. Slices are then created and image augmentation is performed, including mirror flipping, rotation, brightness and contrast enhancement, etc. The dataset is divided into a training set and a validation set in a 9:1 ratio. Public images of real PCBs containing defects are obtained as a test set. The DAS attention mechanism, which is improved by lightweight deformable convolution and hybrid attention mechanisms, is then added to the RPN module of Faster-RCNN. A feature pyramid FPN is constructed in the feature extraction module, and deformation regularization is added to the loss functions of the RPN and RoIHeads. Finally, the improved network is trained to perform target detection and recognition on a real PCB defect dataset.

[0045] Preferably, in step S1, this embodiment uses Altium Designer to draw a PCB circuit board, and edits it, sets various defects such as open circuits and short circuits, and slices it, performs image amplification, and divides the image-augmented data set into a training set and a validation set, and obtains publicly available real PCB images containing defects as a test set.

[0046] Figure 2 The following is a diagram of the improved RPN module architecture. The improved DAS (Deformable Attention to Capture Salient Information) attention mechanism is added to the RPN module in Faster-RCNN. Its core idea is to generate a spatial attention map through learning, thereby weighting different regions of the image and dynamically capturing important spatial regions in the image. Because traditional deformable convolutions require large offset learning parameters, the DAS module is improved by grouping input features, with each group sharing an offset to reduce computational complexity. Since DAS only focuses on spatial dimensions and ignores channel features, the SE module is inserted after DAS to further enhance PCB defect-related features through channel attention while suppressing irrelevant channel noise.

[0047] The present invention adds an improved DAS attention mechanism module to the RPN module in Faster-RCNN. The improved RPN module processes the feature map in the following steps:

[0048] (1) Extract the RPN feature map obtained by Backbone module processing, assuming that the input feature map of RPN is F∈R H ×W×C ;

[0049] (2) The improved DAS module is inserted after the 3×3 convolution layer in the RPN. The improved DAS module includes a grouping module, a depthwise separable convolution layer, an instance normalization layer, a deformable convolution layer, a layer normalization layer, an activation layer, a fusion layer, and a SE module. The grouping module divides the input feature map into several groups along the channel dimension. The number of channels is an integer multiple of the number of groups. Each group predicts only one set of offsets. The number of groups matches the offsets so that each group shares the same offset. The RPN feature map F with shared offsets passes through the depthwise separable convolution layer, the instance normalization layer, the deformable convolution layer, the layer normalization layer, and the activation layer in sequence. Through the two parts of DSC (depthwise separable convolution) and DC (deformable convolution), the attention of six salient areas in the PCB circuit diagram, including missing holes, mouse bites, open circuits, short circuits, strays, and pseudo copper, is focused and the density attention (pixel-level weight) is calculated to generate a spatial attention map A∈R H×W ;

[0050] (3) Multiply the spatial attention map A and the RPN feature map F element by element to obtain the weighted feature map F';

[0051] (4) Insert the SE module to process the weighted feature map F' and obtain the channel attention weighted output feature map. The SE module adaptively learns the weight of each channel by introducing Squeeze and Excitation operations, focusing on more important feature channels, strengthening the high-level features of PCB defects including shape and texture, while suppressing irrelevant channel noise;

[0052] (5) The output feature map of the channel attention weighted is processed by two 1×1 convolution branches: the classification branch and the regression branch. The classification branch performs binary classification on each anchor point to determine whether it contains the target and calculates the probability of the target and background through the Softmax function. The regression branch performs bounding box regression on the anchor point containing the target and adjusts the position and size of the candidate box to make it closer to the real target.

[0053] (6) Finally, the final candidate region is generated based on the classification and regression results.

[0054] The improved RPN module significantly enhances the feature extraction capability of PCB defects, especially its adaptability to irregular shapes, tiny defects and complex backgrounds. Compared with the fixed grid of standard convolution in traditional methods that is difficult to match the diversity of defects, DAS can improve feature alignment accuracy through local deformation. For defect separation under complex backgrounds, DAS can reduce the activation intensity of non-critical areas (such as uniform copper foil), suppress background features, amplify the difference between defects and surrounding areas through the attention mechanism, enhance contrast features, and thus better achieve separation of background and defects.

[0055] Figure 3This diagram shows the architecture for fusing the FPN feature pyramid with the ResNet50 feature extraction module. By introducing FPN (Feature Pyramid Networks) into the feature extraction module, and by constructing a feature pyramid and fusing multi-scale features from top to bottom, the FPN solves the problem of small defects being lost in deep feature maps in traditional methods. The high-resolution feature map directly captures the pixel-level changes of tiny defects and extracts the edge and texture features of PCB defects, effectively enhancing the detection capability of small targets such as PCB defects, improving performance and reducing computational costs. To address the problem that unconstrained offsets may cause excessive deformation of the convolution kernel and miscapture background noise, deformation regularization is added to the RPN and RoIHeads losses. The improved Faster-RCNN network model is obtained. The network input data is the PCB defect dataset expanded from S1, and the output data consists of two parts: confidence and the horizontal and vertical coordinates of the prediction box, which are used to describe the location and category of the detected PCB defects. The above-mentioned improvement method can enhance the feature extraction capability of PCB defect datasets, better handle multi-scale targets, and significantly improve detection performance, especially the detection capability of small targets in PCB defect datasets. It achieves the effect of identifying and locating real PCB defects using a network trained with simulated PCB defect data samples.

[0056] The present invention introduces a feature pyramid FPN into the feature extraction module in Faster-RCNN, including the following steps:

[0057] (1) Select the output of multiple stages such as C2, C3, C4, and C5 layers from Backbone (ResNet50) as the input of FPN;

[0058] (2) Starting from the high-level feature map C5, high-resolution feature maps are gradually generated through upsampling;

[0059] (3) The bottom-up feature map is fused with the top-down feature map to generate the final multi-scale feature map.

[0060] The present invention adds deformation regularization to the loss of RPN and RoI Heads, including the following steps:

[0061] Add the L2 constraint of the offset to the loss function: L offset =λ‖Δp‖2

[0062] Where λ is the balance coefficient (usually set to 0.01-0.1).

[0063] experiment

[0064] The experiments used an Ubuntu 20.04.5 LTS operating system and an NVIDIA GeForce RTX 2080Ti GPU with 11GB of video memory. CUDA 11.4 and cuDNN 8.2.2 were used to accelerate network training. Faster-RCNN employed SGD as the optimizer, with a momentum value of 0.937, weight decay of 0.0001, a learning rate of 0.01, and a training batch size of 200 epochs. Table 1 summarizes the number of parameters and floating-point operations of the improved Faster-RCNN model before and after the improvement. The improved Faster-RCNN significantly reduces the number of parameters and floating-point operations, reducing the number of parameters by approximately 27.5% and the number of floating-point operations by approximately 15.2% compared to the original. Table 2 summarizes the mAP results for six PCB defect detection types achieved by the improved and unimproved models. The improved model improves the mAP for all six defect types. As can be seen from the above, this method can effectively detect and identify PCB defects, improving detection accuracy.

[0065] Table 1 Statistics of the number of parameters and floating-point operations of the model before and after the improvement of Faster-RCNN

[0066]

[0067] Table 2 mAP experimental results of different model algorithms for 6 types of PCB defect detection

[0068]

[0069] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0070] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A PCB defect detection method based on improved Faster-RCNN, characterized in that: The method comprises the following steps: S1, draws a PCB circuit diagram, sets various defects including open circuit and short circuit, and generates a PCB defect dataset based on the PCB circuit diagram; S2, introduces the FPN module into the feature extraction module of the Faster-RCNN network to construct a feature pyramid. It fuses multi-scale features from top to bottom to identify pixel-level changes of tiny defects in the PCB circuit diagram and extract the edge and texture features of PCB defects. S3: Insert a grouping module and an SE module at the input and output of the DAS module, respectively, to obtain an improved DAS module. The grouping module is used to group input features, with each group sharing an offset. The SE module is used to further enhance PCB defect-related features through channel attention while suppressing irrelevant channel noise. The improved DAS module is added to the RPN module of the Faster-RCNN network to reduce the activation intensity of non-critical areas, including uniform copper foil, suppress background features, and amplify the difference between PCB defects and surrounding areas through the attention mechanism, enhancing contrast features and separating the background from the PCB defects. S4, adding deformation regularization to the loss of the RPN module and the RoI detection head to obtain the improved Faster-RCNN network model, and training the Faster-RCNN network model; S5, inputting the defective PCB circuit diagram into the trained Faster-RCNN network model to identify the location and category of the PCB defects contained therein.

2. The PCB defect detection method based on improved Faster-RCNN according to claim 1, characterized in that: Step S1 further comprises: Use Altium Designer software to draw and edit the PCB circuit board, set various defects including open circuits and short circuits, and make it into slices, perform image amplification, and generate a PCB defect data set; The image-augmented PCB defect dataset is divided into a training set and a validation set, and publicly available defective PCB circuit diagrams are obtained as a test set.

3. The PCB defect detection method based on improved Faster-RCNN according to claim 2, characterized in that: In step S1, the process of image amplification includes: performing mirror flipping, rotation, brightness and contrast enhancement processing on the PCB images in the PCB defect dataset to obtain an image augmented dataset.

4. The PCB defect detection method based on improved Faster-RCNN according to claim 1, characterized in that: Step S2 further comprises: Select the outputs of the C2, C3, C4, and C5 layers from the Backbone module as the input of the FPN module; Starting from the high-level feature map C5, high-resolution feature maps are gradually generated through upsampling; The bottom-up feature map is fused with the top-down feature map to generate the final multi-scale feature map.

5. The PCB defect detection method based on improved Faster-RCNN according to claim 1, characterized in that: In step S3, the RPN module of the improved Faster-RCNN network model includes a feature input layer, a 3×3 convolutional layer, a grouping module, a depth-separable convolutional layer, an instance normalization layer, a deformable convolutional layer, a layer normalization layer, an activation layer, a fusion layer, a SE module, a classification branch, and a regression branch; The 3×3 convolutional layer extracts features from the feature map received by the feature input layer after being processed by the Backbone module to obtain the RPN feature map F∈R H×W×C The grouping module divides the input feature map into several groups along the channel dimension, where the number of channels is an integer multiple of the number of groups. Each group predicts only one set of offsets, and the number of groups matches the offsets so that the same offsets are shared within each group. The RPN feature map F of the shared offset passes through the depthwise separable convolution layer, the instance normalization layer, the deformable convolution layer, the layer normalization layer, and the activation layer in sequence, focusing on six salient areas in the PCB circuit diagram, including missing holes, mouse bites, open circuits, short circuits, strays, and pseudo copper, and calculating pixel-level weights to generate a spatial attention map A∈R H×W ; The fusion layer multiplies the spatial attention map A and the RPN feature map F element by element to obtain the weighted feature map F'; The SE module processes the weighted feature map F' to obtain a channel-attention-weighted output feature map. The SE module adaptively learns the weight of each channel by introducing Squeeze and Excitation operations, focusing on more important feature channels, enhancing high-level features of PCB defects including shape and texture, while suppressing irrelevant channel noise. The output feature map is processed by the classification branch and the regression branch respectively. The classification branch performs binary classification on each anchor point to determine whether it contains a PCB defect target, and calculates the probability of the target and the background through the Softmax function; The regression branch performs bounding box regression on the anchor point containing the target, adjusts the position and size of the candidate box to make it closer to the real target, and generates the final candidate region based on the classification and regression results.

6. The PCB defect detection method based on improved Faster-RCNN according to claim 1, characterized in that: In step S4, adding deformation regularization to the loss of the RPN module and RoI detection head means, Add the L2 constraint of the offset to the loss function: L offset =λ‖Δp‖2, where λ is the balance coefficient.

Images