Printed circuit board defect detection method based on semantic segmentation

Abstract

The present invention relates to the field of intelligent manufacturing technology and discloses a method for printed circuit board (PCB) defect detection based on semantic segmentation. The method comprises the following steps: extracting visual defect features from a PCB query image and a PCB support image, wherein the PCB query image is a PCB image without defect labels, and the PCB support image is a PCB image with defect labels; using a self-supervised inner loop to adjust the distribution of low-level visual defect features; using a deep nonlinear metric to determine defect regions of interest in the PCB query image; and using a segmentation decoder to refine the defect regions of interest in the PCB query image to the size of the original PCB image, thereby obtaining defect segmentation results. The method utilizes a self-supervised scheme to dynamically adjust the marginal distribution of potential defect category features and employs a basic defect feature learner driven by a self-segmentation task, thereby enhancing the feature description of specific defect categories and improving the performance of defect label prediction.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and in particular to a printed circuit board defect detection method based on semantic segmentation. Background Art

[0002] The application of printed circuit boards (PCBs) is becoming more and more extensive with the development of the electronics industry, and the corresponding demand for PCBs is also increasing. In the PCB manufacturing industry, defects and flaws in PCBs may be caused by process accuracy, which affects the use of products. A lot of manpower and material resources are often invested in the defect detection link of PCBs, which increases production costs. With the introduction of deep learning, PCB defect detection has also become intelligent and automated. When the deep learning network has sufficient training samples, it can achieve good detection and segmentation effects, which is of great help to defect detection. However, in the actual production process, in most cases, very few defect samples are obtained, which are not enough to train the deep learning network. If defect data is used to fine-tune the existing semantic segmentation model, underfitting problems may occur, and the detection accuracy is far from meeting the production requirements, which brings great difficulties to the PCB defect detection task. The present invention focuses on the ability to learn defect segmentation (small sample segmentation) under the premise of a small number of PCB defect samples. Summary of the Invention

[0003] In order to solve the above technical problems, the present invention provides a printed circuit board defect detection method based on semantic segmentation.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] A printed circuit board defect detection method based on semantic segmentation obtains defect segmentation results of a PCB query image through a defect segmentation model. The defect detection method includes the following steps:

[0006] Step 1: Extract visual defect features from the PCB query image and PCB support image, where the PCB query image is a PCB image without defect labels, and the PCB support image is a PCB image with defect labels;

[0007] Step 2: Use the self-supervised inner loop to adjust the distribution of low-level visual defect features;

[0008] Step 3: Use deep nonlinear metrics to determine the defect regions of interest in the PCB query image;

[0009] Step 4: Use the segmentation decoder to refine the defect area of ​​interest in the PCB query image to the size of the original PCB image to obtain the defect segmentation result.

[0010] Furthermore, in step 1, when extracting visual defect features of the PCB query image and the PCB support image, the following steps are specifically included:

[0011] S11: Divide the PCB image sample into a PCB query image without defect labels and a PCB support image with defect labels;

[0012] S12: Using the twin network u m Calculate the visual defect features of the defective PCB query image and the PCB support image:

[0013] T q =u m (Y i ; δ m );

[0014] T p =u m (P i ; δ m );

[0015] Where, T q Query the defect features of the PCB image, T p is the defect feature of the PCB support image, δ m For the twin network u m The learnable parameters of Y i 、P i are the i-th category PCB query image and the i-th category PCB support image respectively.

[0016] Furthermore, in step 2, when using the self-supervised inner loop to adjust the distribution of low-level visual defect features, the following steps are specifically included:

[0017] S21: Using deep nonlinear metrics u n Measure the area I where the defect features exist inside the PCB support image p :I p =u n (T p , T p ; δ n );δ n represents the nonlinear measurement parameter;

[0018] S22: Use decoder u d Area I p Refine to the original PCB image size and obtain the PCB defect image mask K p :K p =u d (I p ; δ d );δ d Represents decoder parameters;

[0019] S23: Calculate auxiliary cross entropy loss Loss with the corresponding PCB support image mask p : is the label of the defective pixel in the i-th PCB support image;

[0020] S24: Obtain the gradient map of the back-propagation self-supervised loss calculation of the visual defect features of the PCB support image, and obtain the guidance for enhancing each pixel in the PCB support image defect embedding space

[0021]

[0022] Furthermore, in step 3, when using deep nonlinear metrics to determine the defect area of ​​interest in the PCB query image, the following steps are specifically included:

[0023] S31: Multiply the visual defect feature guidance obtained in the previous step by the defect pixel label of the supporting PCB image, and aggregate the results to obtain the potential defect feature;

[0024] S32: Flatten the defect potential features to the original spatial scale so that each dimension of the visual defect features of the PCB query image is consistent with the representative defect features. Alignment:

[0025]

[0026] Where t represents the tiling operation and pool represents the pooling operation;

[0027] S33: Obtain the defect response region of the PCB query image, i.e., the defect region of interest I in the PCB query image, through the relational network comparator Re:

[0028]

[0029] Furthermore, in step 4, when using the segmentation decoder to refine the defect area of ​​interest in the PCB query image to the size of the original PCB image, the following steps are specifically included:

[0030] S41: Input the defect area of ​​interest into the segmentation decoder, refine and restore it to the size of the original PCB image to obtain the defect segmentation result K and the main cross entropy loss function Loss m :

[0031] K=u d (I; δ d );

[0032]

[0033] S42: According to the main cross entropy loss function Loss obtained previously m And auxiliary cross entropy loss Loss e Optimize and train the defect segmentation model.

[0034] Compared with the prior art, the beneficial technical effects of the present invention are:

[0035] In this method, semantic constraints for specific defect categories are provided by the inner loop of self-supervision and used to adjust the distribution of latent defect features across different events. The resulting auxiliary loss is also introduced into the outer loop of the training process, achieving faster convergence of the defect segmentation model. This method offers the advantage of achieving high detection speed and accuracy even with a small number of defect samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a structural diagram of the defect segmentation model of the present invention;

[0037] Figure 2 Schematic diagram of the self-supervisory inner loop module of the present invention;

[0038] Figure 3 Schematic diagram of the depth nonlinear measurement of the present invention;

[0039] Figure 4 This is a schematic diagram of missing parts defects;

[0040] Figure 5 This is a schematic diagram of the detection results of missing parts defects. DETAILED DESCRIPTION

[0041] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] The present invention proposes a printed circuit board defect detection method based on semantic segmentation. The structure of the defect segmentation model used is shown in Figure 1 The present invention uses a self-supervisory scheme to dynamically adjust the marginal distribution of potential defect category features and a basic defect feature learner driven by a self-segmentation task to enhance the feature description of specific defect categories and improve the performance of defect label prediction.

[0043] A printed circuit board defect detection method based on semantic segmentation obtains defect segmentation results of a PCB query image through a defect segmentation model. The defect detection method includes the following steps:

[0044] Step 1: Extract visual defect features of the PCB query image and PCB support image, where the PCB query image is a PCB image without defect labels, and the PCB support image is a PCB image with defect labels.

[0045] Step 1 specifically includes the following steps:

[0046] S11: Divide the PCB image samples into PCB query images without defect labels and PCB support images with defect labels.

[0047] S12: Using the twin network u m Calculate the potential visual defect features of the defective PCB query image and PCB support image:

[0048] T q =u m (Y i ; δ m );

[0049] T p =u m (P i ; δ m );

[0050] Where, T q Query the defect features of the PCB image, T p is the defect feature of the PCB support image, δ m For the twin network u m The learnable parameters of Y i 、P i are the i-th category PCB query image and the i-th category PCB support image respectively.

[0051] Step 2: Use the self-supervised inner loop (see Figure 2 ) adjusts the distribution of low-level visual defect features.

[0052] Step 2 specifically includes the following steps:

[0053] S21: Using deep nonlinear metrics u n Measure the area I where the defect features exist inside the PCB support image p :I p =u n (T p , T p ; δ n );δ n represents the nonlinear metric parameter.

[0054] S22: Use decoder u d Area I pRefine to the original PCB image size and obtain the PCB defect image mask K p :K p =u d (I p ; δ d );δ d Decoder parameters.

[0055] S23: Calculate auxiliary cross entropy loss Loss with the corresponding PCB support image mask p : is the label of the defective pixel in the i-th PCB support image.

[0056] S24: Obtain the gradient map of the back-propagation self-supervised loss calculation of the visual defect features of the PCB support image to enhance the guidance of each pixel in the PCB support image defect embedding space

[0057]

[0058] Step 3: Use deep nonlinear metrics to determine the defect regions of interest in the PCB query image.

[0059] Step three specifically includes the following steps:

[0060] S31: Multiply the visual defect feature guidance obtained in the previous step with the defect pixel label of the supporting PCB image, and aggregate the results to obtain the potential defect feature.

[0061] S32: Flatten the defect potential features to the original spatial scale so that each dimension of the visual defect features of the PCB query image is consistent with the representative defect features. Alignment:

[0062]

[0063] Where t represents the tiling operation and pool represents the pooling operation.

[0064] S33: Obtain the defect response region of the PCB query image, i.e., the defect region of interest I in the PCB query image, through the relational network comparator Re:

[0065]

[0066] Step 4: Use the segmentation decoder to refine the defect area of ​​interest in the PCB query image to the size of the original PCB image to obtain the defect segmentation result.

[0067] Step 4 specifically includes the following steps:

[0068] S41: Input the defect area of ​​interest into the segmentation decoder, refine and restore it to the size of the original PCB image to obtain the defect segmentation result K and the main cross entropy loss function Loss m :

[0069] K=u d (I; δ d );

[0070]

[0071] Step 5: The present invention uses the main cross entropy loss function Loss obtained previously m And auxiliary cross entropy loss Loss p Optimize and train the defect segmentation model.

[0072] The effectiveness of this method is demonstrated by using a small number of missing part defect samples. Figure 4 As shown in the figure, the detection results of this type of defects are shown in Figure 5 , it can be seen that the model successfully segments the defect, and the brighter irregular area in the figure is the model segmentation result.

[0073] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0074] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A printed circuit board defect detection method based on semantic segmentation, wherein a defect segmentation result of a PCB query image is obtained by a defect segmentation model. The defect detection method comprises the following steps: Step 1: Extract visual defect features from the PCB query image and PCB support image, where the PCB query image is a PCB image without defect labels, and the PCB support image is a PCB image with defect labels; Step 2: Use the self-supervised inner loop to adjust the distribution of low-level visual defect features; Step 3: Use deep nonlinear metrics to determine the defect regions of interest in the PCB query image; Step 4: Use the segmentation decoder to refine the defect area of ​​interest in the PCB query image to the size of the original PCB image to obtain the defect segmentation result; In step 1, extracting visual defect features of the PCB query image and the PCB support image specifically includes the following steps: S11: Divide the PCB image sample into a PCB query image without defect labels and a PCB support image with defect labels; S12: Leveraging Siamese Networks Calculate the visual defect features of the defective PCB query image and the PCB support image: ； ； Where, Query defect features of PCB images, Support defect features of PCB images, Siamese network The learnable parameters of 、 are the query image of the i-th PCB and the support image of the i-th PCB respectively; In step 2, when using the self-supervised inner loop to adjust the distribution of low-level visual defect features, the following steps are specifically included: S21: Using Deep Nonlinear Metrics Measure the area where defect features exist inside the PCB support image : ; represents the nonlinear measurement parameter; S22: Using the decoder The area Refine to the original PCB image size to obtain the image mask that supports PCB defects : ; Represents decoder parameters; S23: Compute auxiliary cross entropy loss with the corresponding PCB support image mask : ; is the label of the defective pixel of the i-th PCB support image, is the cross entropy loss; S24: Obtain the gradient map of the back-propagation self-supervised loss calculation of the visual defect features of the PCB support image, and obtain the guidance for enhancing each pixel in the PCB support image defect embedding space : ； In step 3, the deep nonlinear metric is used to determine the defect area of ​​interest in the PCB query image, specifically including: S31: Multiply the visual defect feature guidance obtained in the previous step by the defect pixel label of the supporting PCB image, and aggregate the results to obtain the potential defect feature; S32: Flatten the defect potential features to the original spatial scale so that each dimension of the visual defect features of the PCB query image is consistent with the representative defect features. Alignment: ； In the formula Represents a tiling operation, Represents pooling operation; S33: Comparator via Relational Network , get the defect response area of ​​the PCB query image, that is, the defect area of ​​interest in the PCB query image : 。 2. The printed circuit board defect detection method based on semantic segmentation according to claim 1, characterized in that: In step 4, when using the segmentation decoder to refine the defect area of ​​interest in the PCB query image to the size of the original PCB image, the following steps are specifically included: S41: Input the defect area of ​​interest into the segmentation decoder, refine and restore it to the size of the original PCB image to obtain the defect segmentation result. And the main cross entropy loss function : ； ； S42: According to the main cross entropy loss function obtained previously and auxiliary cross entropy loss Optimize and train the defect segmentation model.

Images