Two-stage detection method for PCB defects based on deep learning

Abstract

The application discloses a two-stage detection method for PCB defects based on deep learning, which comprises: using a shallow neural network for first-stage detection, including a PCB image acquisition module, a CNN module, an FPN module, a PCB feature fusion module, a feature mask module, a center-ness module, an HCL module, a PCB defect classification module, a defect frame regression module, an NCC difference correction module, an NMS module, a PCB defect library module, etc.; and using a deep neural network for second-stage detection, including a CNN module, a DropMax feature enhancement module, a fusion feature extraction module, a classification module, a classification loss module, a normalization module, a defect type module, a defect grading module, etc., to generate the final result of PCB defect detection of a to-be-detected board. The neural network technology of deep learning is used to fully utilize the standard board information of the PCB for PCB defect detection, and the detection speed is fast and the accuracy is high.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a two-stage PCB defect detection method based on deep learning. Background Technology

[0002] Printed Circuit Boards (PCBs) are essential components of electronic devices such as computers, mobile phones, and refrigerators. The performance of modern electronic devices is influenced not only by the quality and performance of the electronic components themselves, but also, to a large extent, by the quality of the PCB. In particular, the two-dimensional PCB is the foundation of the entire electronic circuit, the first critical link affecting the quality of subsequent electronic products, and a crucial link in the quality of electronic systems and production cycles. With the rapid development of the electronics and information technology field, manufacturers of various equipment have increasingly higher requirements for PCBs, not only pursuing higher performance and efficiency, but also placing stricter demands on PCB yield and production capacity.

[0003] With increasingly stringent requirements for PCB yield and production capacity, the precision and quality demands of PCB manufacturing are also rising. A key challenge is finding a way to detect PCB defects that are often small and sparse, with less noticeable visual features.

[0004] In related technologies, traditional methods for 2D PCB inspection include manual inspection and flying probe inspection. Manual inspection involves hiring workers on the production line to visually inspect various areas of the manufactured circuit boards under lights to identify defects. Manual inspection is unreliable, prone to missed defects, inefficient, and increases labor costs. Flying probe inspection, on the other hand, checks the connectivity between solder joints according to the design drawings; it is a physical inspection method. It uses probes instead of a bed of pins, employing multiple motor-driven, rapidly moving electrical probes to contact the pins of components and perform electrical measurements. Flying probe inspection is time-consuming and can potentially damage the PCB, limiting its accuracy and applicability.

[0005] Using artificial intelligence methods for automated inspection is a current trend in intelligent manufacturing. It can not only improve complex inspection equipment, but also free workers from mechanical labor on the production line, allowing them to use their labor in other more needed areas. Currently, there is a need for a technology that can combine PCB inspection with artificial intelligence. Summary of the Invention

[0006] This application provides a two-stage PCB defect detection method based on deep learning. It makes full use of the information of the standard PCB board and uses deep learning neural network technology to perform two-stage PCB defect detection, and obtains the defect detection results of the PCB board under test. The detection speed is fast and the accuracy is high.

[0007] The first aspect of this application provides a two-stage PCB defect detection method based on deep learning, comprising the following steps: acquiring images of a standard PCB board and a PCB board under test; constructing a shallow neural network for detecting defects in the PCB board under test, referring to the standard PCB board image; performing defect feature extraction, feature fusion, and PCB feature masking on the PCB board under test based on the shallow neural network, and performing preliminary detection, classification, and regression of defects in the PCB board under test; classifying and classifying the defects in the PCB board under test based on the defects in the PCB board under test image using a pre-constructed shallow classification and grading network, and generating preliminary detection results of defects in the PCB board under test; constructing a deep neural network for detecting defects in the PCB board under test, learning and strengthening weak features based on the defects in the PCB board under test image initially detected by the shallow neural network, eliminating overfitting of the model to partial view information, fusing feature extraction, refining the classification and grading of defects in the PCB board under test, and generating the final result of defect detection in the PCB board under test.

[0008] Optionally, in one embodiment of this application, a shallow neural network is constructed to detect defects in the PCB board under test. Based on the shallow neural network, defect feature extraction, feature fusion, and PCB feature masking are performed on the PCB board under test. Preliminary detection, classification, and regression of defects in the PCB board under test are then performed, including: preprocessing the PCB standard board image and the PCB board under test image to obtain processed PCB standard board images and PCB board under test images; pruning and optimizing the CNN network and FPN network; and using the optimized shallow CNN network and shallow FPN network to perform predictions at different levels to obtain the processed PCB standard board image and PCB board under test image. Multi-scale defect detection bounding boxes are generated from PCB images. Based on the processed standard PCB image and the PCB under test image, after network convolution, feature maps of the standard board and the PCB under test are obtained and compared to extract the feature difference information learned by the network. PCB fusion features are obtained by fusing the feature difference information learned by the network with the visual information of the PCB. Channel concatenation is performed between the feature map of the PCB under test and the feature map of the standard board using CGA. The concatenated feature maps are then grouped and fused, followed by grouped multi-layer convolution to output an attention mask. The CGA is interacted with the features before the attention mask, and feature residual information is added to output the features after the mask. The PCB defect annotation system is used to annotate the PCB images. For example, defects are segmented, bounding boxes are selected and collected, and the types and levels of PCB board defects are marked according to international PCB board classification and grading principles to construct a PCB defect feature library; a shallow PCB board defect classification network is constructed using a PCB board feature mask module and a CNN classification branch module; a shallow PCB board defect regression network is constructed using a PCB board feature mask module and a CNN regression branch module; a PCB board feature fusion mask module and an NCC difference correction module are used to normalize the cross-correlation of specific points and regions, calculate the maximum response of the defect features of the PCB board image under test within the template image, and output an anomaly attention mask; the shallow PCB board defect classification network and the... A PCB defect feature library is used to initially identify the types of defects in the images of the PCB board under test. A CNN classification branch module, a Center-ness module, and a High-Level Classification (HCL) module are used to enhance foreground and background region features, improving PCB detection for sparsely distributed defects and indistinct target features, and increasing the network's confidence and discriminative power for foreground and background regions. The shallow PCB defect regression network, the Center-ness module, and the HCL module are used to predict the defect center of the PCB board under test and determine the bounding box and location of the defects. The PCB defect classification module interacts with the HCL module and then merges the information with that from the defect regression bounding box module.By utilizing the interaction between the non-maximum suppression module and the NCC difference correction module, redundant detection boxes for PCB defects are filtered out, and a preliminary judgment of potential defects in the PCB board under test is given, thus determining the defects in the PCB board image detected in the first stage of network inspection.

[0009] Optionally, in one embodiment of this application, a deep neural network for detecting defects in the PCB board under test is constructed. Based on the defects in the PCB board under test image detected by the shallow neural network for detecting defects, weak features are learned and strengthened to eliminate overfitting of the model to partial view information. Feature extraction is fused to refine the classification and grading of defects in the PCB board under test and generate the final result of defect detection in the PCB board under test. This includes: using the deep neural network for detecting defects, fusing the defect features detected by the first-stage network with the features of the PCB standard board and the features of the PCB board under test to extract the defect features of the PCB board under test; using the DropMax module to extract the network's multi-view comprehensive information, learning and strengthening the features of PCB defects, and eliminating overfitting of the model to partial view information; using the fusion feature extraction module and the DropMax module to further extract refined features of PCB defects from the fused PCB features using a fully connected network; and using the fusion feature extraction module to extract refined features of PCB defects. A deep classification network for PCB board defects is constructed using a module and a PCB defect classification module. The deep network and classification loss module are used to train the PCB board defect classification. The fusion feature extraction and normalization modules are used to generate retrieval features from the PCB defect features. The fine classification and defect feature retrieval of PCB defects trained by the deep network are used to finely classify the defects detected on the PCB board under test, and the classification is compared with the PCB defect classification feature library to obtain the defect identification and classification ranking results. The severity of each type of defect on the identified PCB board under test is graded and labeled using the PCB defect grading feature library, and the classification is compared with the PCB defect grading feature library to obtain the different levels of defects detected on the PCB board under test. Finally, the second-stage deep neural network for PCB defect detection is used to finely classify and grade the defects on the PCB board under test, obtaining the type and level of the defects detected on the PCB board under test.

[0010] A second aspect of this application provides a deep learning-based PCB defect detection device, comprising: an acquisition component for acquiring images of a standard PCB board and an image of a PCB board under test; a first detection component for using a pre-built shallow neural network to detect defects in the PCB board under test, providing a preliminary judgment of potential defects in the PCB board under test, and determining defects detected in the PCB board image in the first stage of network detection; and a second detection component for using a pre-built deep neural network and the defects detected in the PCB board under test by the first stage shallow neural network to perform a second stage deep network detection, performing fine defect classification and grading on the defects in the PCB board under test, obtaining the type and level of defects detected in the PCB board under test, and generating the final result of the defect detection of the PCB board under test.

[0011] Optionally, in one embodiment of this application, the acquisition component includes: a PCB standard board acquisition module for acquiring the PCB standard board image; and a PCB test board module for acquiring the PCB test board image.

[0012] Optionally, in one embodiment of this application, the first detection component includes: a preprocessing module, used to preprocess the PCB standard board image and the PCB board under test image through noise filtering, illumination processing, and image registration to obtain processed PCB standard board images and PCB board under test images; a CNN processing module, used to optimize and predict at different levels using shallow networks, and to perform pruning optimization on shallow CNN networks and shallow FPN networks to obtain multi-scale defect detection boxes for the processed PCB standard board images and PCB board under test images; and a network residual module, used to perform preprocessing based on the processed PCB standard board images and PCB board under test images through... After network convolution, a standard board feature map and a test board feature map are obtained. These are compared to extract the network-learned feature difference information. A first PCB board feature fusion module is used to fuse the network-learned feature difference information and the visual information of the PCB board to obtain PCB fusion features. A feature mask module is used to perform channel-by-channel stitching of the test board feature map and the standard board feature map using CGA, grouping and fusing the stitched feature maps, and then performing grouped multi-layer convolution to obtain an attention mask. The CGA is then interacted with the features before the attention mask, and feature residual information is added to obtain the features after the mask. A PCB defect feature library is also included. This system is used to collect various PCB board defects. It utilizes a PCB defect annotation system to segment PCB images, select and collect defects, and label the types and levels of PCB board defects according to international PCB classification and grading principles, thus constructing a PCB defect feature library. A CNN classification branch module is used to construct a shallow PCB board defect classification network. A PCB defect classification module is used to identify the defect types in the PCB board images under test using the shallow PCB board defect classification network and the PCB defect feature library. A Center-ness module is used to predict the defect center of the PCB board under test, locate the position of the defect, and add center-ness information to the classification branch. The network employs several mechanisms: a central constraint to reduce the confidence of predicted bounding boxes in non-object center regions, thereby reducing the output of predicted bounding boxes in background regions; an HCL module, using a loss function to adjust for foreground and background features, enhancing the network's confidence in distinguishing foreground and background regions and improving PCB detection performance; a CNN regression branch module to construct a shallow PCB defect regression network; a defect box regression module to identify the bounding boxes of defects in the tested PCB using the shallow PCB defect regression network and determine the location of the defects; and a difference correction module to perform normalized cross-correlation on a specific point and region, calculating the features in the tested image within the template. Figure 1The maximum response within a certain range is output as an abnormal attention mask; a non-maximum value suppression module is used to filter redundant detection boxes of defects on the PCB board under test; an interaction module is used to filter redundant defect boxes of defects on the PCB board under test, and after difference correction, it gives a suspected prediction of defects on the PCB board under test, and determines the defects of the PCB board image in the first stage of network detection.

[0013] Optionally, in one embodiment of this application, the pre-constructed deep neural network for detecting defects in the PCB under test and the first-stage shallow neural network are used to initially detect defects in the PCB under test. Then, the second-stage deep network detection is performed, including: a deep CNN network module, used to fuse the defect features of the PCB under test initially detected in the first stage with the features of the PCB standard board and the PCB under test to extract the defect features of the PCB under test; a DropMax module, used to extract multi-view comprehensive information from the network, learn and enhance the features of PCB defects, and eliminate overfitting of the model to partial view information; a second PCB board feature fusion module, used to fuse and enhance the refined features of the PCB standard board and the PCB under test to obtain fused features, and to further extract refined features of PCB defects using a fully connected network on the fused PCB features; a classification module, used for... A deep classification network for PCB board defects is constructed. A classification loss module is used for training the classification of PCB board defects. A normalization module is used to normalize the features of the PCB board defects and generate retrieval features. A defect type module is used to finely classify the detected defects on the PCB board using the PCB defect classification layer deep network trained on by the network, and compare it with the PCB defect classification feature library to obtain the PCB board defect identification and classification ranking results. A defect grading module is used to grade and label the severity of each type of defect on the identified PCB board, and compare it with the PCB defect grading feature library to obtain the different levels of the detected defects on the PCB board. The second-stage deep network is used to perform fine defect classification and grading on the defects of the PCB board to obtain the type and level of the detected defects on the PCB board.

[0014] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep learning-based two-stage PCB defect detection method as described in the above embodiments.

[0015] A fourth aspect of this application provides a computer-readable storage medium storing computer instructions for causing the computer to perform the deep learning-based two-stage PCB defect detection method as described in the above embodiments.

[0016] This application fully utilizes information from standard PCB boards and employs deep learning neural network technology to perform a two-stage defect detection process for the PCB board under test, obtaining defect detection results. First, a pre-constructed shallow neural network is used to detect defects in the PCB board under test, providing a preliminary judgment of potential defects and identifying defects detected in the PCB board image during the first stage of network detection. Then, using the pre-constructed deep neural network and the defects detected by the first-stage shallow neural network, a second-stage deep network detection process is performed to perform refined defect classification and grading, obtaining the type and level of defects detected on the PCB board under test, and generating the final defect detection result for the PCB board under test. This method offers fast detection speed and high accuracy.

[0017] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0019] Figure 1 This application provides a two-stage PCB defect detection method based on deep learning, according to an embodiment of the present application.

[0020] Figure 2 This is a schematic diagram of a two-stage detection structure provided according to an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the CGA function provided according to an embodiment of this application;

[0022] Figure 4 This is a feature mask map provided according to an embodiment of this application;

[0023] Figure 5 Here is an example diagram of a deep learning-based PCB defect detection device according to an embodiment of this application;

[0024] Figure 6 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] The following describes an embodiment of the deep learning-based two-stage PCB defect detection method according to the present application, with reference to the accompanying drawings. Addressing the problem mentioned in the background section regarding the lack of a solution combining PCB inspection with artificial intelligence, this application provides a deep learning-based two-stage PCB defect detection method. In this method, deep learning neural network technology is used to perform two-stage PCB defect detection, obtaining classification and grading detection results for defects on the PCB under test. The method offers fast detection speed and high accuracy.

[0027] Specifically, Figure 1 This is a flowchart illustrating a two-stage PCB defect detection method based on deep learning, provided in an embodiment of this application.

[0028] like Figure 1 As shown, the two-stage PCB defect detection method based on deep learning includes the following steps:

[0029] In step S11, images of the PCB standard board and the PCB board under test are obtained.

[0030] In step S12, a pre-constructed shallow neural network is used to perform preliminary defect detection on the PCB board under test, including:

[0031] Step S121: Use a pre-constructed shallow CNN network, shallow FPN network, and network residual module to extract and fuse PCB board features;

[0032] Step S122: Use CGA to mask the PCB board features and output the attention mask and the features after masking.

[0033] Step S123 involves classifying and regressing PCB board defects using a pre-built shallow CNN classification branch network, a shallow CNN regression branch network, a PCB defect classification feature library, a PCB board defect classification module, a Center-ness module, an HCL module, an NMS module, and an NCC module. Optionally, in embodiments of this application, a pre-built shallow neural network for detecting defects in the PCB board under test is used, including: preprocessing PCB standard board images and PCB board under test images to obtain processed PCB standard board images and PCB board under test images; and pruning and optimizing the CNN network and the FPN network. Utilizing optimized shallow CNN and shallow FPN networks, predictions are performed at different levels to obtain processed images of the standard PCB board and the PCB board under test, outputting multi-scale defect detection boxes. Based on the processed images, the standard and under-test board features are obtained through network convolution and compared to extract network-learned feature difference information. Based on the network-learned feature difference information and the visual information of the PCB board, PCB board image fusion features are obtained. Channel Group Attention (CGA) is used to concatenate the feature maps of the under-test board and the standard board, grouping and fusing the concatenated feature maps, followed by grouped multi-layer convolution to generate an attention mask. CGA is interacted with the features before the attention mask, incorporating feature residual information to output the masked features. A PCB board defect annotation system is used to segment the PCB board images, select and collect defects, and label the types and levels of PCB board defects according to international PCB board defect classification and grading principles, constructing a PCB defect feature library. A PCB board defect classification network is designed using the PCB board feature mask module and the CNN classification branch module. A PCB board defect regression network is designed using a PCB board feature masking module and a CNN regression branch module. A PCB board feature fusion masking module and an NCC difference correction module are used to normalize the cross-correlation of specific points and regions, calculate the maximum response of defect features in the template image of the PCB board under test, and output an anomaly attention mask. A PCB board defect classification network and a PCB defect feature library are used to identify the types of defects in the PCB board image under test. A CNN classification branch module, a Center-ness module, and a High-Level Classification (HCL) module are used to enhance foreground and background region features, improve PCB detection with sparse defect distribution and indistinct target features, and enhance the network's confidence and discriminative power for foreground and background regions. A PCB board defect regression network, a Center-ness module, and an HCL module are used to predict the defect center of the PCB board under test and determine the defect bounding box and location. The PCB board defect classification module interacts with the HCL module first, and then the information is fused with the defect regression bounding box module. A non-maximum suppression module is used to filter out redundant defect bounding boxes of the PCB board.By utilizing the interaction of the non-maximum suppression module and the NCC difference correction module, redundant detection boxes of PCB board defects are filtered out, and a suspected prediction of defects in the PCB board under test is given, thus determining the defects in the PCB board image in the first stage of network detection.

[0034] In step S13, a pre-built deep neural network is used to perform fine-grained defect detection on the PCB board under test, including:

[0035] Step S131: Using a pre-built deep CNN network, the defect features detected by the PCB under test in the first stage are fused with the features of the standard PCB board and the PCB under test to extract the defect features of the PCB under test B. The DropMax module is used to fully extract the multi-view comprehensive information of the network, learn and enhance the features of PCB defects, and eliminate the overfitting of the model to partial view information.

[0036] Step S132: Using the fusion feature extraction module, a fully connected network is used to extract refined features of PCB board defects from the fused PCB board defect features.

[0037] Step S133 involves using a pre-built deep neural network for PCB board defect classification, a classification loss module, a normalization module, a PCB defect grading feature library, a PCB board defect type module, and a PCB board defect grading module to perform fine classification and grading of defects on the PCB board under test. Optionally, in embodiments of this application, after using a pre-built deep neural network for detecting defects on the PCB board under test and a first-stage shallow neural network to detect defects, a second-stage deep network detection is performed, further including: using a deep CNN network to fuse the defect features detected by the first-stage PCB board under test with the features of the PCB standard board and the PCB board under test to extract the defect features of the PCB board under test. The DropMax module is used to fully extract the multi-view comprehensive information of the network, learn and enhance the features of PCB defects, and eliminate overfitting of the model to partial view information. The fusion feature extraction module and the DropMax module are used to further extract refined features of PCB board defects from the fused PCB board features using a fully connected network. A deep classification network for PCB board defects is designed using the fusion feature extraction module and the PCB board defect classification module. A deep neural network for PCB defect classification and a classification loss module are used for training to classify PCB defects. A fusion feature extraction module and a normalization module are used to generate retrieval features from the PCB defect features. The deep neural network for PCB defect classification and defect feature retrieval, trained on top of this, performs fine-grained classification of defects detected on the PCB under test. The classification is then compared with a PCB defect classification feature library to output the defect identification and classification ranking results. Using a PCB defect grading feature library, the severity of each type of defect in the identified PCB under test is graded and labeled to obtain different levels of defects detected on the PCB under test. A second-stage deep neural network is used to perform fine-grained defect classification and grading on the PCB under test, obtaining the type and level of defects detected on the PCB under test.

[0038] In step S14, the results of defect detection on the PCB board under test are output using a two-stage detection process.

[0039] With the increasing demand for printed circuit boards (PCBs), the precision and quality requirements for PCB manufacturing are also becoming increasingly stringent. How to fully utilize standard PCB board information to address the issue of PCB defect data, which often has insignificant visual features and is typically small and sparse, is a problem this application aims to solve. This application proposes a two-stage PCB defect detection method based on deep learning neural network technology.

[0040] The first stage of the PCB target detection method, based on reference comparison, uses a shallow network construction algorithm. This stage directly extracts the original image of the entire PCB for detection. However, the original PCB image is large, and the defect distribution is sparse; using a complex network model would consume significant computational resources. Furthermore, directly using the classification and grading of PCB defects for the detection network would lead to overly fine-grained class distinctions in the network output, exacerbating the imbalance between each class and the background, including non-defect parts, which is detrimental to the training of the detection network. Therefore, a shallow neural network is adopted in the first stage, including a shallow CNN network module, a shallow FPN network module, a PCB feature fusion module, a feature mask module, a center-ness module, an HCL (Higher Center-ness Loss) module, a PCB defect classification module, a defect box regression module, an NMS (Non-maximum Suppression) module, an NCC (Normalized Cross-Correlation) difference correction module, and a PCB defect database module, etc. Figure 2 As shown.

[0041] Combination Figure 2 The architecture provides a detailed description of the first stage of testing in this application.

[0042] The input for PCB board defect detection is a standard PCB board image and a PCB board under test image. Both of these images are the original PCB board images after a series of preprocessing steps, such as noise filtering, lighting processing, and image registration.

[0043] like Figure 2 As shown, the PCB standard board image and the PCB under test image are input through the PCB standard board module and the PCB under test module, respectively. The CNN module is connected to the PCB standard board module and the PCB under test module, respectively. The original PCB board image usually has high parsing complexity. In order to reduce computation and accelerate defect detection, a shallow CNN network is adopted.

[0044] The CNN network includes an FPN network module, which is downsampled by the FPN. The FPN still uses a shallow network, and optimization and pruning techniques are applied to the FPN. Defects in PCB boards have different scales, some as small as a few pixels and others as large as hundreds of pixels. Each layer of the FPN represents the features of defects at different scales, and according to the characteristics of convolutional networks, the receptive field of each layer is continuously expanded. The network makes predictions at different layers and outputs multi-scale defect detection boxes.

[0045] The network residual module is connected to the CNN network module and the FPN network module. After the images of the PCB standard board and the board under test are respectively convolved by the network, the features of the standard board and the features of the board under test are compared to extract the difference information of the network learned features.

[0046] The PCB feature fusion module is connected to the CNN network module, the FPN network module, and the network residual module. Utilizing the difference information between the "test board" and the standard board, while also considering the visual information of the PCB itself, features from different levels of the FPN network are fused. Then, normalized cross-correlation, nonlocal resampling, and attention masks are used, combined with anomaly attention and nonlocal feature fusion methods, to obtain the fused features of the PCB.

[0047] The feature mask module includes a CGA module, which is connected to the PCB feature fusion module and the network residual module. CGA is a mechanism for extracting information by grouping and generating attention masks. CGA concatenates the feature maps of the board under test and the standard board, groups and fuses the concatenated feature maps, performs grouped multi-layer convolution, and outputs the attention mask, such as... Figure 3 As shown. Then, the CGA features are interacted with the pre-mask features. Simply using the attention mask features may cause the network training to become dependent on local features of PCB board defects, which is not conducive to learning complete features. Therefore, when using the CGA attention mechanism, the feature residual part is added to output the post-mask features, such as... Figure 4 As shown. CGA preserves the original visual information, enhancing the network's utilization of the standard board reference information while also preventing the network from overly relying on difference information during learning.

[0048] PCB Defect Feature Library. This library collects PCB board defects for various part numbers. Using a PCB board defect annotation system, it segments PCB board images, selects and collects defects, and labels the types and levels of PCB board defects according to international PCB board defect classification and grading principles, thus constructing a PCB defect feature library.

[0049] The CNN classification branch module is connected to the feature mask module to construct a PCB board defect classification network.

[0050] The CNN regression branch module is connected to the feature mask module to construct a PCB board defect regression network.

[0051] The PCB defect classification module is connected to the CNN classification branch module and the PCB defect feature library module to identify the type of defect in the PCB board under test.

[0052] The Center-ness module and the CNN classification branch module predict the center of defects on the PCB board and locate the defect location. A center constraint is added to the classification branch to reduce the confidence of predicted bounding boxes in non-object center regions, thereby reducing the output of predicted bounding boxes in background regions.

[0053] The HCL module connects to the Center-ness module. In PCB defect detection, many defects have indistinct visual features, making it essential to increase the network's ability to distinguish between foreground and background regions. HCL is a module in the network that uses a loss function to adjust the output, making the Center-ness output more discriminative of foreground and background regions, thus improving the detection performance of sparsely distributed PCB defects with indistinct visual features. It also reduces negative samples and lessens the burden of NMS filtering. Furthermore, it enhances the features of foreground and background regions, improving the network's confidence in distinguishing between foreground and background regions and the overall performance of PCB defect detection.

[0054] The defect bounding box regression module is connected to the CNN regression branch module to determine the bounding box and location of defects on the PCB board.

[0055] The NMS module connects to the PCB board defect classification module, HCL module, and defect regression box module. First, the PCB board defect classification module interacts with the HCL module, and then the information is fused with that from the defect regression box module. Non-maximum value suppression is used to filter redundant PCB board defect boxes.

[0056] The NCC difference correction module is connected to the PCB feature fusion mask module. Normalized cross-correlation is performed on a specific point and region to calculate the features in the test image within the template. Figure 1 The maximum response within a defined range is output as an anomaly attention mask with a numerical range of [0,1]. The NMS module interacts with the NCC difference correction module to filter redundant detection boxes of PCB board defects and provide a preliminary judgment of the first stage defects of the PCB board under test.

[0057] Optionally, in the embodiments of this application, based on the deep neural network for detecting defects in the PCB board under test and the defects in the PCB board under test detected by the first-stage shallow neural network, a second-stage deep network detection is performed, including: obtaining refined features of PCB board defects based on the defects in the PCB board under test detected by the first-stage shallow neural network and the deep neural network for detecting defects in the PCB board under test; performing feature fusion and enhancement on the refined features of PCB board defects to obtain fused features; extracting PCB defect features from the fused features using a fully connected network; obtaining defect identification and classification ranking results based on the PCB defect feature library; normalizing the PCB board defect features to generate retrieval features for querying the PCB defect feature library; and classifying and labeling the severity of each type of defect in the PCB board to obtain different levels of defects in the PCB board under test.

[0058] Combination Figure 2 The architecture provides a detailed description of the second phase of testing in this application.

[0059] In the second stage of this application, unlike the shallow network used in the first detection stage, a deep network is employed to refine the defect classification and grading of the PCB board. After the first stage of detection filters out many background areas in the PCB image, the defect results detected in the first stage are used to apply a refined classification and grading network, including a CNN module, a DropMax module, a fusion feature extraction module, a classification module, a classification loss module, a normalization module, a defect type module, and a defect grading module, etc. Figure 2 As shown.

[0060] The deep CNN convolutional module is connected to both the PCB standard board module and the PCB board under test module. In the first stage, the defect results detected on the PCB board under test are filtered and then input into the deep CNN module for refined feature extraction of these defects.

[0061] The DropMax module is connected to the CNN module to address the problem of overfitting to strong features. Previously, multiple views were obtained by stitching together the standard PCB board and the board under test, representing the same object. During feature extraction, overfitting can easily occur, causing the network to focus excessively on partial information in the views, such as only paying attention to the background circuit structure of the standard PCB board that is largely unrelated to defects, resulting in poor generalization. To avoid this problem, the network is enhanced before image fusion to fully extract the comprehensive information provided by the multiple views. During the forward propagation phase of model training, DropMax uses features with probability p to respond. This allows weak features to be learned and strengthened specifically, preventing the network from becoming dependent on strong features, reducing overfitting to partial view information, and ensuring that information from multiple views is fully utilized and learned. Ultimately, this enables the network to cope with feature loss during testing or in complex real-world environments. Then, the features of the standard PCB board and the board under test are fused. Finally, DropMax is used to enhance the features of the fused image.

[0062] The fusion feature extraction module is connected to the DropMax module, and the fused features are further extracted using a fully connected network to extract PCB board defect features.

[0063] The PCB defect classification module is connected to the fusion feature extraction module, forming a classification network for PCB board defects.

[0064] The classification loss module is connected to the PCB defect classification layer network, and the cross-entropy classification loss function is used to train the classification of PCB board defects.

[0065] The normalization module is connected to the fusion feature extraction module to generate retrieval features after normalizing the defect features of the PCB board.

[0066] The PCB defect type module is connected to the classification loss module, the normalization module, and the PCB defect feature library. When identifying PCB defects, the Euclidean distribution of features trained using the cross-entropy classification loss function cannot represent their similarity. Therefore, the features from the previous layer of the classifier are normalized to generate retrieval features, which are then compared with the defect feature library to output the defect identification and classification ranking results.

[0067] The PCB defect grading module connects with the PCB defect type module and the PCB defect feature library to classify and label the severity of each type of defect on the PCB board. Finally, it outputs the different levels of defects detected on the PCB board under test.

[0068] As described above, the first stage of this application is preliminary target detection based on reference comparison, and the second stage is refined defect classification and grading. The first stage can filter out a significant amount of background areas, alleviating the data imbalance problem in the second stage. Simultaneously, such a network system can be extended and compatible with various post-processing workflows in PCB board target detection, such as manually set rule filtering and functional area filtering. The two-stage networks can be maintained and debugged independently, and the independently trained stages are more likely to produce complementary model effects, resulting in better overall PCB board defect detection performance.

[0069] The two-stage PCB defect detection method based on deep learning proposed in this application fully utilizes information from a standard PCB board and employs deep learning neural network technology to perform two-stage defect detection on the PCB board under test, obtaining the defect detection results. First, a pre-constructed shallow neural network is used to detect defects in the PCB board under test, providing a preliminary judgment of potential defects and identifying defects in the PCB board image detected in the first stage. Then, using a pre-constructed deep neural network and the defects initially detected by the first-stage shallow neural network, a second-stage deep network detection is performed to perform refined defect classification and grading, obtaining the type and level of defects detected on the PCB board under test, and generating the final defect detection result for the PCB board under test. This method offers fast detection speed and high accuracy.

[0070] Next, referring to the accompanying drawings, a two-stage PCB defect detection device based on deep learning, according to an embodiment of this application, is described.

[0071] Figure 5 This is an example diagram of a two-stage PCB defect detection device based on deep learning according to an embodiment of this application.

[0072] like Figure 5 As shown, the deep learning-based two-stage PCB defect detection device 10 includes: an acquisition component 100, a first detection component 200, and a second detection component 300.

[0073] The system comprises several components: Component 100, which acquires images of a standard PCB board and the PCB board under test; and Component 200, which uses a pre-built shallow neural network to detect defects in the PCB board under test, providing a preliminary assessment of potential defects and identifying defects detected in the first-stage network detection of the PCB board image; and Component 300, which uses a pre-built deep neural network and the defects detected by the first-stage shallow neural network to perform a second-stage deep network detection, classifying and grading the defects in the PCB board under test to obtain the type and level of the detected defects, and generating the final result of the PCB board defect detection.

[0074] Optionally, in one embodiment of this application, the acquisition component includes: a PCB standard board acquisition module for acquiring PCB standard board images; and a PCB test board module for acquiring PCB test board images.

[0075] Optionally, in one embodiment of this application, the first detection component includes: a preprocessing module, used to preprocess the PCB standard board image and the PCB board under test image through noise filtering, illumination processing, and image registration to obtain processed PCB standard board images and PCB board under test images; a CNN processing module, used to optimize and predict at different levels using shallow networks, and to perform pruning optimization on shallow CNN networks and shallow FPN networks to obtain multi-scale defect detection boxes for the processed PCB standard board images and PCB board under test images; and a network residual module, used to perform preprocessing based on the processed PCB standard board images and PCB board under test images through noise filtering, illumination processing, and image registration to obtain processed PCB standard board images and PCB board under test images. After network convolution, feature maps of the standard board and the board under test are obtained. These are compared to extract the feature difference information learned by the network. The first PCB board feature fusion module fuses the network-learned feature difference information and the visual information of the PCB board to obtain fused PCB board features. The feature mask module uses CGA to concatenate the feature maps of the board under test and the standard board, groups and fuses the concatenated feature maps, and then performs multi-layer convolution to obtain an attention mask. The CGA is then interacted with the features before the attention mask, and residual feature information is added to obtain the features after the mask. A PCB defect feature library is used to collect various P... PCB defect identification utilizes a PCB defect annotation system to segment PCB images, select and collect defects, and label the types and levels of PCB defects according to international PCB defect classification and grading principles to construct a PCB defect feature library. A CNN classification branch module is used to construct a shallow PCB defect classification network. A PCB defect classification module uses this shallow network and feature library to identify defect types in images of the PCB under test. A Center-ness module predicts the defect center of the PCB under test, locates the defect position, and adds center-ness information to the classification branch. The network employs several mechanisms: a core constraint to reduce the confidence of predicted bounding boxes in non-object center regions, thereby reducing the output of predicted bounding boxes in background regions; an HCL module, using a loss function to adjust foreground and background features, enhancing the network's confidence in distinguishing between foreground and background regions and improving PCB detection performance; a CNN regression branch module to construct a shallow PCB defect regression network; a defect box regression module to identify the bounding boxes of defects on the PCB board under test using the shallow PCB defect regression network and determine the location of the defects; and a difference correction module to perform normalized cross-correlation on a specific point and region, calculating the features in the image under test within the template. Figure 1The maximum response within a certain range outputs an anomaly attention mask; the non-maximum value suppression module is used to filter redundant detection boxes of defects on the PCB board under test; the interaction module is used to filter redundant defect boxes of defects on the PCB board under test, and after difference correction, it gives a suspected prediction of defects on the PCB board under test, and determines the defects of the PCB board image in the first stage of network detection.

[0076] Optionally, in one embodiment of this application, a deep neural network for detecting defects in a PCB under test is pre-constructed, along with a first-stage shallow neural network. The defects in the PCB under test are initially detected, and then a second-stage deep network detection is performed. This includes: a deep CNN network module, used to fuse the defect features of the PCB under test initially detected in the first stage with the features of the PCB standard board and the PCB under test, to extract the defect features of the PCB under test; a DropMax module, used to extract multi-view comprehensive information from the network, learn and enhance the features of the PCB board defects, and eliminate overfitting of the model to partial view information; a second PCB board feature fusion module, used to fuse and enhance the refined features of the PCB standard board and the PCB under test, to obtain fused features, and to further extract refined features of PCB defects using a fully connected network on the fused PCB features; and a classification module, used to construct P... The system employs a deep classification network for PCB (PCB under-test) defects. This network includes: a classification loss module for training and classifying PCB defects; a normalization module for normalizing PCB defect features and generating retrieval features; a defect type module for fine-grained classification and feature retrieval of PCB defects using the deep network trained on the PCB defect classification layer, comparing the detected defects with a PCB defect classification feature library to obtain PCB defect identification and classification ranking results; and a defect grading module for grading the severity of each type of defect on the identified PCB, comparing it with a PCB defect grading feature library to obtain different levels of detected defects. Finally, the system utilizes a second-stage deep network to perform fine-grained defect classification and grading of the PCB defects, obtaining the type and level of the detected defects.

[0077] It should be noted that the foregoing explanation of the embodiment of the two-stage PCB defect detection method based on deep learning also applies to the two-stage PCB defect detection device based on deep learning in this embodiment, and will not be repeated here.

[0078] The deep learning-based two-stage PCB defect detection device proposed in this application fully utilizes information from a standard PCB board and employs deep learning neural network technology to perform two-stage PCB defect detection, obtaining defect detection results for the PCB board under test. First, a pre-constructed shallow neural network is used to detect defects in the PCB board under test, providing a preliminary judgment of potential defects and identifying defects in the PCB board image detected in the first-stage network. Then, using a pre-constructed deep neural network and the defects detected by the first-stage shallow neural network, a second-stage deep network detection is performed to perform refined defect classification and grading, obtaining the type and level of defects detected in the PCB board under test, and generating the final defect detection result for the PCB board under test. This method offers fast detection speed and high accuracy.

[0079] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0080] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0081] When the processor 602 executes the program, it implements the two-stage PCB defect detection method based on deep learning provided in the above embodiments.

[0082] Furthermore, electronic devices also include:

[0083] Communication interface 603 is used for communication between memory 601 and processor 602.

[0084] The memory 601 is used to store computer programs that can run on the processor 602.

[0085] The memory 601 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0086] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0087] Optionally, in a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through an internal interface.

[0088] The processor 602 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0089] This embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described two-stage deep learning-based PCB defect detection method.

[0090] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0091] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0092] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0093] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0094] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0095] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0096] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0097] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A two-stage PCB defect detection method based on deep learning, characterized in that, Includes the following steps: Obtain images of the standard PCB board and the PCB board under test; Referring to the PCB standard board image, a shallow neural network for PCB defect detection is constructed to detect defects in the PCB board under test. Based on the shallow neural network for PCB defect detection, defect features of the PCB under test are extracted, fused, and masked. Preliminary detection, classification, and regression of defects on the PCB under test are then performed. This includes: preprocessing the images of the standard PCB board and the PCB under test to obtain processed images; pruning and optimizing the CNN and FPN networks, and using the optimized shallow CNN and FPN networks to perform predictions at different levels to obtain multi-scale defect detection boxes for the processed images of the standard PCB board and the PCB under test; and based on the processed images of the standard PCB board and the PCB under test... After network convolution, the feature maps of the standard board and the board under test are obtained and compared to extract the feature difference information learned by the network. Based on the network-learned feature difference information and the visual information of the PCB board, the PCB fusion feature is obtained. The feature map of the board under test and the feature map of the standard board are concatenated using CGA (Concurrent Gaussian Graphics), and the concatenated feature maps are grouped and fused, then grouped into multiple convolutions to output an attention mask. The CGA is interacted with the features before the attention mask, and feature residual information is added to output the features after the mask. Using a PCB defect annotation system, the PCB image is segmented, defects are selected and collected, and the type and level of PCB board defects are marked according to international PCB board classification and grading principles to construct the PCB fusion feature. A defect feature library is used; a shallow PCB board defect classification network is constructed using a PCB board feature mask module and a CNN classification branch module; a shallow PCB board defect regression network is constructed using a PCB board feature mask module and a CNN regression branch module; a PCB board feature fusion mask module and an NCC difference correction module are used to normalize the cross-correlation of specific points and regions, calculate the maximum response of the defect features of the PCB board under test image within the template image, and output an anomaly attention mask; using the shallow PCB board defect classification network and the PCB defect feature library, the types of defects in the PCB board under test image are initially identified; the foreground is enhanced using a CNN classification branch module, a Center-ness module, and an HCL module. Based on background region features, this method improves PCB detection for sparsely distributed defects and indistinct target features, enhancing the network's confidence and discriminative power between foreground and background regions. Utilizing the shallow PCB board defect regression network, the Center-ness module, and the HCL module, it predicts the defect center of the PCB under test and determines the defect bounding box and location. The PCB defect classification module interacts with the HCL module and then fuses information with the defect regression bounding box module. Finally, the non-maximum suppression module interacts with the NCC difference correction module to filter redundant detection boxes of PCB defects, providing a preliminary judgment of potential defects in the PCB under test and identifying defects detected by the network in the first stage of the PCB under test image. Based on the defects in the PCB board under test image, a pre-constructed shallow classification and grading network is used to classify and grade the defects of the PCB board under test, generating preliminary detection results of the defects of the PCB board under test. A deep neural network for PCB defect detection is constructed to detect defects in the PCB board under test. Based on the defects initially detected by the shallow neural network for PCB defect detection in the image of the PCB board under test, weak features are learned and strengthened to eliminate overfitting of the model to partial view information. Feature extraction is fused to refine the classification and grading of defects in the PCB board under test and generate the final result of the defect detection of the PCB board under test.

2. The method according to claim 1, characterized in that, A deep neural network for PCB defect detection is constructed to detect defects in the PCB board under test. Based on the defects detected by the shallow neural network for PCB defect detection in the PCB board image, weak features are learned and strengthened to eliminate overfitting of the model to partial view information. Feature extraction is fused to refine the classification and grading of defects in the PCB board under test, and the final result of the PCB board defect detection is generated, including: Using the deep neural network for PCB defect detection, the defect features detected by the first-stage network are fused with the features of the PCB standard board and the features of the PCB board under test to extract the defect features of the PCB board under test. The DropMax module is used to extract comprehensive information from multiple views of the network, learn and enhance the characteristics of PCB defects, and eliminate the overfitting of the model to partial view information. By utilizing the fusion feature extraction module and the DropMax module, a fully connected network is used to further extract refined features of PCB defects from the fused PCB features; Using the fusion feature extraction module and the PCB defect classification module, a deep classification network for PCB board defects is constructed. The deep network for PCB defect classification and the classification loss module are used to train the classification of PCB board defects. Using the fusion feature extraction module and the normalization module, retrieval features are generated from PCB defect features; By using a deep network trained on a PCB defect classification layer to perform fine classification and defect feature retrieval of PCB defects, the defects detected on the PCB board under test are finely classified and compared with the PCB defect classification feature library to obtain the defect identification and classification ranking results of the PCB board under test. Using the PCB defect classification feature library, the severity of each type of defect in the identified PCB board under test is classified and labeled, and compared with the PCB defect classification feature library to obtain different levels of the detected defects in the PCB board under test. The second-stage deep neural network for PCB defect detection is used to perform fine-grained defect classification and grading of the defects on the PCB under test, thereby obtaining the type and level of the detected defects on the PCB under test.

3. A two-stage PCB defect detection device based on deep learning, characterized in that, include: The acquisition component is used to acquire images of the standard PCB board and the PCB board under test. The first detection component is used to detect defects in the PCB board under test using a pre-built shallow neural network, provide a suspected prediction of defects in the PCB board under test, and determine the defects detected in the PCB board under test image in the first stage of network detection. The first detection component includes: The preprocessing module is used to preprocess the PCB standard board image and the PCB board under test image through noise filtering, illumination processing and image registration to obtain the processed PCB standard board image and PCB board under test image. The CNN processing module is used to optimize and predict at different levels using shallow networks. It performs pruning optimization on shallow CNN networks and shallow FPN networks to obtain multi-scale defect detection boxes for processed PCB standard board images and PCB test board images. The network residual module is used to obtain the feature map of the standard board and the feature map of the board under test after the processed PCB standard board image and the PCB board under test image are convolved by the network, and to compare the feature map of the standard board and the feature map of the board under test to extract the feature difference information learned by the network. The first PCB board feature fusion module is used to fuse PCB fusion features based on the feature difference information learned by the network and the visual information of the PCB board. The feature mask module is used to use CGA to perform channel splicing of the feature map of the board under test and the feature map of the standard board, group and fuse the spliced ​​feature maps, and then perform grouped multi-layer convolution to obtain an attention mask. The CGA is then interacted with the features before the attention mask, and the feature residual information is added to obtain the features after the mask. The PCB defect feature library is used to collect various PCB board defects. It uses a PCB defect annotation system to segment PCB images, select and collect defects, and mark the types and levels of PCB board defects according to international PCB board defect classification and grading principles to build the PCB defect feature library. The CNN classification branch module is used to construct a shallow PCB board defect classification network; The PCB defect classification module is used to identify the defect type in the PCB board under test image by utilizing the shallow PCB board defect classification network and PCB defect feature library. The Center-ness module is used to predict the defect center of the PCB board under test, locate the position of the defect of the PCB board under test, add center constraints to the classification branch, reduce the confidence of the predicted box in the non-object center area, and thus reduce the output of the predicted box in the background area. The HCL module is used to adjust the loss function, enhance the features of the foreground and background regions, and improve the network's confidence in distinguishing between the foreground and background regions and the performance of PCB inspection. The CNN regression branch module is used to construct a shallow PCB board defect regression network; The defect box regression module is used to identify the boxes of defects in the PCB board under test using the shallow PCB board defect regression network, and to determine the location of the defects in the PCB board under test. The difference correction module is used to perform normalized cross-correlation on a specific point and region, calculate the maximum response of the feature in the test image within a certain range of the template image, and output the anomaly attention mask. The non-maximum value suppression module is used to filter out the redundant detection frames of defects on the PCB board under test. The interaction module is used to filter redundant defect boxes on the PCB under test, and after difference correction, provides a preliminary judgment of the suspected defects of the PCB under test, determining the defects in the PCB under test image during the first stage of network detection; and The second detection component is used to perform a second-stage deep network detection on the PCB under test using a pre-built deep neural network and a first-stage shallow neural network to detect defects. This allows for fine-grained defect classification and grading of the defects on the PCB under test, resulting in the type and level of the detected defects, and generating the final result of the defect detection for the PCB under test.

4. The apparatus according to claim 3, characterized in that, The acquisition component includes: PCB standard board acquisition module, used to acquire the PCB standard board image; The PCB under test module is used to acquire images of the PCB under test.

5. The apparatus according to claim 3, characterized in that, Using the pre-constructed deep neural network for detecting defects in the PCB under test and the first-stage shallow neural network, the defects of the PCB under test are initially detected, and then the second-stage deep network detection is performed, including: A deep CNN network module is used to fuse the defect features of the PCB board under test detected in the first stage with the features of the PCB standard board and the features of the PCB board under test to extract the defect features of the PCB board under test. The DropMax module is used to extract comprehensive information from multiple network views, learn and enhance the features of PCB defects, and eliminate overfitting of the model to partial view information. The second PCB board feature fusion module is used to fuse and enhance the refined features of the PCB standard board and the PCB board under test to obtain fused features. The fused PCB features are further extracted using a fully connected network to refine the PCB defect features. The classification module is used to construct a deep classification network for defects in the PCB board under test. The classification loss module is used for training to classify defects on the PCB board under test. The normalization module is used to normalize the features of defects under test on the PCB and generate retrieval features; The defect type module is used to finely classify PCB defects and retrieve defect features by using the deep network trained by the PCB defect classification layer, finely classify the defects detected on the PCB board under test, and compare them with the PCB defect classification feature library to obtain the defect identification and classification ranking results of the PCB board under test. The defect classification module is used to classify and label the severity of each type of defect in the identified PCB board under test, and compare it with the PCB defect classification feature library to obtain different levels of the detected defects in the PCB board under test. The second-stage deep network is used to perform fine defect classification and grading on the defects in the PCB board under test to obtain the type and level of the detected defects in the PCB board under test.

6. An electronic device, characterized in that, include: The memory, the processor, and the computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the deep learning-based two-stage PCB defect detection method as described in any one of claims 1-2.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the deep learning-based two-stage PCB defect detection method as described in any one of claims 1-2.

Images