Printed circuit board defect detection method and system based on deep learning

Abstract

The application provides a printed circuit board defect detection method and system based on deep learning, wherein the method comprises the following steps: inputting a PCB original image into a deep neural network for target detection reasoning; before the target detection reasoning, performing data preprocessing and background and edge extraction on the PCB original image; wherein the deep learning method aligns the PCB original image with a standard board image; after the alignment of the PCB original image with the standard board image, a deep target detection model is used for defect detection, and the deep target detection model is a two-stage target detection model of a comprehensive single-stage target detection algorithm; based on the deep target detection model, the defect detection of the PCB original image is completed, and a detection result is output. The defect detection scheme and process are optimized in efficiency; while ensuring the recall rate of PCB appearance defect detection, the detection accuracy is improved, and the academic research and application prospect are very broad, and the application can truly contribute to the PCB industry.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method and system for detecting defects in printed circuit boards based on deep learning. Background Technology

[0002] During actual PCB production, various factors such as unstable raw materials and equipment, production environment, and human error can affect the appearance of PCBs, leading to various defects such as scratches, broken lines, oil stains, missing substrate, copper nodules, broken copper, exposed copper, missing components, abnormal wiring, and non-standard contacts. The goal of PCB defect detection is to pinpoint the exact location of all defects on the PCB and identify their type. Based on actual production needs, detection must prioritize a very high recall rate, while simultaneously maximizing accuracy and ensuring real-time performance.

[0003] In the PCB manufacturing process, traditional PCB appearance defect inspection relies on manual inspection, meaning inspectors judge based on visual observation and experience. However, with the continuous improvement of PCB manufacturing processes and functional complexity, PCBs have become more refined and precise, making many defects difficult to see and distinguish visually. Furthermore, factories produce a large volume of PCBs daily; a medium-sized factory might produce tens of thousands of printed circuit boards daily. Such a large output places a significant burden on manual inspection, resulting in substantial labor costs and increasing the likelihood of subjective misjudgments due to prolonged work hours.

[0004] Given the low efficiency and unreliable accuracy of manual inspection, automated methods and equipment for PCB appearance defect detection have been developed to replace manual inspection. Foreign countries started earlier in this area, and technology companies in Europe, America, Japan, South Korea, and Israel have already developed PCB automated optical inspection (AOI) equipment.

[0005] Building upon traditional image processing methods for automated optical inspection of PCBs, advancements in object detection have provided new approaches for detecting PCB surface defects. Deep learning-based object detection algorithms primarily fall into two categories: one-stage and two-stage algorithms. As the name suggests, one-stage algorithms proceed directly to the target. Based on the correspondence between the convolutional pooling feature map and the grid of the original image, they calculate the final layer of features and then directly perform regression calculations on it to obtain the object's category probability and relative coordinates. Two-stage algorithms, on the other hand, perform region proposals before feature regression. This process selects as many regions as possible that might contain the target, and then selects and fits the features corresponding to these regions to determine the target category and relative position.

[0006] Single-stage detectors can densely predict the output class and location of objects across the entire image. While their accuracy is slightly lower than two-stage detectors, their speed advantage is significant. In recent years, a large number of high-accuracy single-stage detection algorithms have emerged. The RetinaNet algorithm classifies a set of predefined sliding anchor boxes and handles foreground-background imbalance by reweighting the loss for each output. The Rep-Point and DenseRepPoint algorithms encode bounding boxes as the contours of a set of points and use the features of this point set for classification. The AlignDet model proposes adding deformable convolutional layers before using features for classification and regression, improving performance. The FCOS and CenterNet models address the issue of multiple anchor points per pixel and classify foreground and background by location. The ATSS and PAA algorithms further improve foreground and background classification by changing the definitions of foreground and background. The BorderDet model proposes merging features along the bounding box, achieving better localization results.

[0007] Two-stage detectors first use a Region Proposal Network (RPN) to generate coarse candidates, then use corresponding region features to classify and refine them. This approach yields better accuracy, but the increased number of proposals also leads to more computation time. The Faster-R-CNN model is the most classic two-stage detection algorithm, using two fully connected layers as the heads of the region feature connections. The Cascade-R-CNN algorithm utilizes Faster-R-CNN to construct three cascaded stages, each with a different threshold, thus the later stages focus more on localization accuracy. The HTC model, based on Cascade-R-CNN, uses additional instances and semantic segmentation annotations to enhance the feature flow between the stages. The recent TSD algorithm proposes decoupling the classification and localization branches of each region of interest, improving overall accuracy.

[0008] The existing technology has the following disadvantages:

[0009] Currently, various AOI equipment and existing printed circuit board appearance defect detection methods mainly adopt transformation and statistical methods, with a detection accuracy of only 10%-20%. This results in a large number of misjudgments and false alarms, making it difficult to cope with the ever-changing development trend of PCB functions and types.

[0010] Using deep learning methods for PCB appearance defect detection presents a dilemma in selecting the detection model:

[0011] 1) While a two-stage object detection algorithm offers an advantage in average accuracy compared to many single-stage algorithms, the manufacturer's priority is to maximize recall (ideally 100%). This poses a challenge to the Region Proposal Network (RPN) in the two-stage algorithm, as the RPN is a relatively weak binary classifier, and the algorithm's accuracy relies heavily on the filtering of regression prediction heads in the second stage. In this case, achieving near-100% recall necessitates setting a very low RPN proposal filtering threshold in the first stage, leading to a sharp increase in overall computation time and failing to meet the manufacturer's efficiency requirements.

[0012] 2) If a single-stage object detection algorithm is chosen, it has a certain advantage in running speed due to its dense object detection characteristics. However, since there are many defect categories to be identified, the parameters of some single-stage object detection models will also increase significantly, making the speed advantage less obvious. A major drawback is that, in this case, if you want to recall as many defects as possible, you need to lower the detection threshold. However, within the lower score range, the classification prediction accuracy of single-stage algorithms is not high enough. This is because single-stage object detection algorithms only use features from positive samples for regression during training, resulting in poor model fitting to negative samples. Consequently, the model may identify some low-scoring background parts as defects, leading to more false detections. It is difficult to guarantee prediction accuracy while maintaining high recall.

[0013] Therefore, improving the accuracy of printed circuit board defect detection is an urgent problem to be solved. Summary of the Invention

[0014] The present invention aims to at least partially solve one of the technical problems in the related art.

[0015] Therefore, the first objective of this invention is to propose a deep learning-based method for detecting defects in printed circuit boards. In situations where defect-free standard PCB images are available in the production process, a method for detecting appearance defects in printed circuit boards is designed based on a deep learning model, which simultaneously meets the requirements of high input image resolution and high model inference efficiency.

[0016] The second objective of this invention is to propose a deep learning-based printed circuit board defect detection system.

[0017] To achieve the above objectives, a first aspect of the present invention proposes a deep learning-based method for detecting defects in printed circuit boards, comprising:

[0018] The original PCB image is input into a deep neural network for target detection inference;

[0019] Before object detection inference, the original PCB image is preprocessed and background and edge extraction is performed; wherein, deep learning methods are used to align the original PCB image with a standard board image.

[0020] After aligning the original PCB image with the standard board image, a depth target detection model is used for defect detection. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm.

[0021] Based on the aforementioned depth target detection model, defect detection of the original PCB image is completed, and the detection results are output.

[0022] In addition, the deep learning-based printed circuit board defect detection method according to the above embodiments of the present invention may also have the following additional technical features:

[0023] Furthermore, in one embodiment of the present invention, aligning the original PCB image with the standard board image using a deep learning method includes:

[0024] The ability to perceive image content is acquired through autonomous learning of masks, where H ab For the original image I a Convert to standard plate image I b The homography matrix has the following overall loss function:

[0025]

[0026] L n The definitions of L are as follows:

[0027]

[0028] L(I a I b )=||F a -F b ||1

[0029] I' a =H ab I a I' b =H ba I b

[0030] Among them, L n Constraints indicate the desire for I a Through the output H ab After the transformation, we get Figure I′. a Features and I b The "-L" part prevents the network training F from being led to all-zero features, and "-L" guides different image inputs to the image feature extraction module to obtain significantly different results. Adding H... ab and H ba Constraints of reciprocal relations, I a with I b The conversion method is the opposite.

[0031] Furthermore, in one embodiment of the present invention, the two-stage target detection model integrating the single-stage target detection algorithm includes: a structural part and a loss design part; wherein,

[0032] The structural component includes: directly replacing the two-stage RPN component with a single-stage detection model, and changing the detected categories from multiple categories to two categories;

[0033] The loss design includes: a training phase in which the classification weights from the first phase are passed to the classification regression in the second phase, and the parameters of the target class and the background class are trained using maximum likelihood estimation; and a testing phase in which the final classification weights are obtained by multiplying them according to conditional probabilities.

[0034] Furthermore, in one embodiment of the present invention, the two-stage RPN portion is directly replaced with a single-stage detection model, and the detected categories are changed from multiple categories to two categories, including:

[0035] The RPN portion of the first stage in the two-stage process is replaced with a binary classifier of preset strength to provide candidate regions. The binary classifier of preset strength is linked with the subsequent prediction regression training in the second stage, and the loss of samples that are difficult to classify is transferred to the binary classifier of preset strength in the first stage for optimization.

[0036] Furthermore, in one embodiment of the present invention, the data preprocessing includes:

[0037] The PCB area image is extracted by denoising the board to be inspected to obtain the PCB image to be inspected;

[0038] The standard board is denoised to extract the PCB area image, and the color of the PCB image to be detected is corrected according to the color information.

[0039] The corrected PCB image is registered, merged, and meshed with the PCB area image extracted by denoising the standard board.

[0040] This invention discloses a deep learning-based printed circuit board (PCB) defect detection method. The method inputs an original PCB image into a deep neural network for target detection inference. Before target detection inference, the original PCB image undergoes data preprocessing and background and edge extraction. A deep learning method is used to align the original PCB image with a standard board image. After alignment, a deep target detection model is used for defect detection. This deep target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm. Based on the deep target detection model, defect detection of the original PCB image is completed, and the detection results are output. This invention optimizes the efficiency of the defect detection scheme and process, and improves detection accuracy while maintaining the recall rate of PCB appearance defect detection. It also has broad prospects for academic research and application, and can truly contribute to the PCB industry.

[0041] To achieve the above objectives, a second aspect of the present invention provides a deep learning-based printed circuit board defect detection system, comprising:

[0042] The input module is used to input the raw PCB image into the deep neural network for target detection inference.

[0043] The alignment module is used to perform data preprocessing and background and edge extraction on the original PCB image before target detection inference; wherein, a deep learning method is used to align the original PCB image with a standard board image.

[0044] The detection module is used to align the original PCB image with the standard board image and then use a depth target detection model to perform defect detection. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm.

[0045] The output module is used to perform defect detection on the original PCB image based on the depth target detection model and output the detection results.

[0046] This invention discloses a deep learning-based printed circuit board (PCB) defect detection system. The system inputs an original PCB image into a deep neural network for target detection inference. Before inference, the original PCB image undergoes data preprocessing and background and edge extraction. A deep learning method is used to align the original PCB image with a standard board image. After alignment, a deep target detection model, a two-stage model combining a single-stage target detection algorithm, is used for defect detection. Based on this model, defect detection of the original PCB image is completed, and the detection results are output. This invention optimizes the efficiency of the defect detection scheme and process, improves detection accuracy while maintaining the recall rate of PCB appearance defect detection, and has broad prospects for academic research and application, truly contributing to the PCB industry.

[0047] The beneficial effects of this invention are as follows:

[0048] 1) Combining the actual PCB production process, based on a large amount of real data and real industrial needs, we designed and implemented a deep residual defect detection algorithm based on anomaly attention mechanism for PCB appearance defect detection.

[0049] 2) Modify and improve the two-stage object detection scheme. Use a single-stage detection method to replace the RPN in the two-stage object detection, make full use of the strong fitting regression ability of the one-step method, enhance the quality of the proposals recalled by the RPN in the two-step method, thereby reducing the number of proposals selected K and improving the efficiency of the two-step method.

[0050] 3) To address the need for dual-input PCB appearance defect detection, a two-stage target detection depth model Loss is designed. The classification loss of the two stages is treated as conditional probability, and gradient linkage is used during training to improve classification performance. This avoids the separation of the two stages in the traditional two-step method and forms an end-to-end solution.

[0051] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0052] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0053] Figure 1 A flowchart of a deep learning-based printed circuit board defect detection method according to an embodiment of the present invention;

[0054] Figure 2 This is a network structure diagram for PCB image defect detection according to an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of a deep residual defect detection algorithm based on anomaly attention mechanism according to an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the data preprocessing process according to an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of a defect detection process according to an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of a deep learning-based printed circuit board defect detection system according to an embodiment of the present invention. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0060] The memory allocation method and system for edge inference in deep learning according to embodiments of the present invention are described below with reference to the accompanying drawings.

[0061] When using traditional deep learning methods for object detection to detect defects in PCB appearance, there is a dilemma in selecting the detection model. To address this, this invention, prioritizing accuracy, employs a two-stage object detection method that ensures detection precision while also focusing on performance improvement.

[0062] Unlike typical deep learning object detection, the PCB appearance defect detection task addressed in this invention has some distinct characteristics. For example, in addition to the board under test, an image of a defect-free standard board is provided. The model design may require incorporating processing and analysis of the standard board's information. Furthermore, the provided images have very high resolution, while there is a practical requirement to minimize model computation time, which presents challenges for deep model design. Therefore, it is necessary to investigate how to design a deep learning model that better aligns with these characteristics to ensure both the accuracy and practicality of defect detection.

[0063] Figure 1 This is a flowchart of a deep learning-based printed circuit board defect detection method provided in an embodiment of the present invention.

[0064] like Figure 1 As shown, the detection method includes the following steps:

[0065] Step S1: Input the original PCB image into the deep neural network for object detection inference.

[0066] Step S2: Before object detection inference, the original PCB image is preprocessed and background and edge extraction is performed; in this step, a deep learning method is used to align the original PCB image with the standard board image.

[0067] It is understandable that, considering the large amount of raw input images from PCB production lines and the performance requirements of PCB appearance defect detection systems, this invention performs targeted data preprocessing and background and edge extraction on the raw images before inputting them into a deep neural network for target detection and inference. The main method for aligning the raw images with standard board images is to use deep learning methods, autonomously learning the mask to gain the ability to perceive image content, where H... ab For the original image I a Convert to standard plate image I b The homography matrix. The overall loss function is:

[0068]

[0069] L n The definitions of L are as follows:

[0070]

[0071] L(I a I b )=||F a -F b ||1

[0072] I' a =H ab I a I' b =H ba I b

[0073] Among them, L n Constraints indicate the desire for I a Through the output H ab After the transformation, we get Figure I′. a Features and I b The L part prevents the network training from leading F to all-zero features, while -L guides different images input to the image feature extraction module to obtain significantly different results. The third part adds H. ab and H ba The constraint of reciprocal relationships also better aligns with the homography principle, i.e., I a with I b The conversion method is the opposite.

[0074] The embodiments of the present invention will be further explained below with reference to the accompanying drawings, but are not limited thereto.

[0075] As an example, such as Figure 4 As shown, the present invention performs data preprocessing and background and edge extraction on the original PCB image, including: denoising the board to be inspected and extracting the PCB area image to obtain the PCB image to be inspected; denoising the standard board and extracting the PCB area image, and performing color correction on the PCB image to be inspected according to the color information; and registering, merging and meshing the corrected PCB image with the PCB area image extracted by denoising the standard board.

[0076] It is understandable that the PCB appearance defect detection task addressed in this invention has some unique characteristics. For example, in addition to the board under test, an additional image of a defect-free standard board is provided. The design model may require processing and analyzing information from the standard board. Furthermore, the provided images have very high resolution, while there is also a practical requirement to minimize model computation time. Starting with PCB images actually captured on the PCB production line, and guided by the goal of high-precision and high-efficiency defect detection, the PCB board under test is processed.

[0077] Step S3: After aligning the original PCB image with the standard board image, a depth target detection model is used to detect defects. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm.

[0078] It is understandable that after aligning the original image with the calibration board image, this invention can use a depth target detection model to detect defects. Through detailed analysis of the two-stage detection method, it can be found that the main reasons for its slightly insufficient performance are:

[0079] First, the RPN in the first stage of the two-stage algorithm is a weak binary classifier, which is not good enough at classifying various candidate regions (proposals). Therefore, it is necessary to select a low candidate region (proposal) screening threshold in order to recall defects as much as possible.

[0080] Secondly, in addition, during the training process of the model, there is no good and reasonable gradient propagation between the RPN and the second-stage classification and regression head. The training of the two is relatively disconnected, which makes it difficult for the RPN to know the shortcomings of its predictions, make it difficult to classify samples (i.e., hard examples), and thus it cannot adjust its parameters based on the loss generated by the wrong examples, and always remains at a weak binary classification level.

[0081] Working backward from the cause to the solution means that a stronger first-stage binary classifier is needed to provide proposals for the two-stage algorithm. This binary classification should then be linked to the subsequent second-stage prediction and regression training, transferring the loss of hard-to-classify samples to the first-stage classifier for targeted adjustments. It can be observed that the strongest predictive classifiers in the current object detection field are precisely those high-accuracy single-stage object detection models. This suggests that designing a two-stage object detection model that integrates single-stage object detection algorithms is entirely feasible.

[0082] As an example, the structural part simply replaces the two-stage RPN with other better-performing single-stage detection models. Of course, the number of categories to be detected changes from multiple categories to two categories, and the reduction in the number of categories better ensures the classification accuracy of the first stage.

[0083] In the loss design, the classification weights from the first stage can be passed to the classification regression in the second stage. From a mathematical point of view, the two classification weights naturally form a conditional probability relationship. The parameters of the target class and the background class can be trained separately using maximum likelihood estimation. In the testing stage, the final classification weights can also be obtained by multiplying them according to the conditional probability.

[0084] As an example, the present invention selects a binary classifier with a preset strength to provide candidate regions, wherein the preset strength is set by those skilled in the art according to actual needs, and the present invention does not impose specific limitations.

[0085] As an example, the present invention uses methods including but not limited to maximum likelihood estimation to train parameters for the target class and the background class respectively, and the present invention does not impose specific limitations.

[0086] Furthermore, Figure 2 This is a network structure diagram for PCB image defect detection according to an embodiment of the present invention, such as... Figure 2 As shown, based on the initial backbone network, the detection network structure diagram includes a one-step detection head, a candidate box set, and a two-step detection head. By autonomously learning the mask, the network gains the ability to perceive image content. Based on this network structure, a deep learning model design that better fits the above characteristics is created to ensure the accuracy and practicality of defect detection.

[0087] As an example, this invention proposes a deep residual defect detection algorithm based on an anomaly attention mechanism for PCB appearance defect detection scenarios, such as... Figure 3 As shown, there are two main advantages to this design:

[0088] First, fully utilize the strong fitting regression ability of the one-step method to enhance the quality of the proposals recalled by RPN in the two-step method, thereby reducing the number of proposals selected K and improving the efficiency of the two-step method.

[0089] Secondly, the classification loss of the two stages is treated as conditional probabilities, and the gradients are linked during training to improve classification performance. This avoids the separation of the two stages in the traditional two-step method and forms an end-to-end approach.

[0090] Based on this, the present invention optimizes the efficiency of the defect detection scheme and process to address the difficulty of large original image input in PCB appearance defect detection; and improves the detection accuracy while ensuring the recall rate of PCB appearance defect detection.

[0091] Step S4: Based on the depth target detection model, complete the defect detection of the original PCB image and output the detection results.

[0092] Understandably, by extracting data from the dataset, determining image data and location information, and then detecting the image data using a detector, the detection results are obtained and output in real time based on the location information. This invention provides a real-time PCB defect detection objective: to detect the specific locations of all defects in the PCB and to identify the defect types. Starting with PCB images actually captured on the PCB production line, and guided by high-precision and high-efficiency defect detection, it studies how to decompose and define each component of the system, designing a reasonable detection process to replace manual inspection at each stage, ensuring the accuracy and practicality of defect detection. For example... Figure 5 As shown.

[0093] This invention discloses a deep learning-based printed circuit board (PCB) defect detection method. The method inputs an original PCB image into a deep neural network for target detection inference. Before target detection inference, the original PCB image undergoes data preprocessing and background and edge extraction. A deep learning method is used to align the original PCB image with a standard board image. After alignment, a deep target detection model is used for defect detection. This deep target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm. Based on the deep target detection model, defect detection of the original PCB image is completed, and the detection results are output. This invention optimizes the efficiency of the defect detection scheme and process, and improves detection accuracy while maintaining the recall rate of PCB appearance defect detection. It also has broad prospects for academic research and application, and can truly contribute to the PCB industry.

[0094] Figure 6 This is a schematic diagram of a deep learning-based printed circuit board defect detection system according to an embodiment of the present invention.

[0095] like Figure 6 As shown, the detection system 10 includes: an input module 100, an alignment module 200, a detection module block 300, and an output module 400.

[0096] The input module 100 is used to input the original PCB image into a deep neural network for target detection inference.

[0097] Alignment module 200 is used to perform data preprocessing and background and edge extraction on the original PCB image before target detection inference; wherein, deep learning method is used to align the original PCB image with a standard board image;

[0098] The detection module 300 is used to perform defect detection using a depth target detection model after aligning the original PCB image with the standard board image. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm.

[0099] Output module 400 is used to perform defect detection on the original PCB image based on the depth target detection model and output the detection results.

[0100] Furthermore, the alignment module 200 described above is also used for:

[0101] The ability to perceive image content is acquired through autonomous learning of masks, where H ab For the original image I a Convert to standard plate image I b The homography matrix has the following overall loss function:

[0102]

[0103] L n The definitions of L are as follows:

[0104]

[0105] L(I a I b )=||F a -F b ||1

[0106] I' a =H ab I a I' b =H ba I b

[0107] Among them, L n Constraints indicate the desire for I a Through the output H ab After the transformation, we get Figure I′. a Features and I b The "-L" part prevents the network training F from being led to all-zero features, and "-L" guides different image inputs to the image feature extraction module to obtain significantly different results. Adding H... ab and H ba Constraints of reciprocal relations, I a with I b The conversion method is the opposite.

[0108] Furthermore, the aforementioned detection module 300 includes: a structural component module and a loss design component module; wherein,

[0109] The structural module is used to directly replace the two-stage RPN part with a single-stage detection model, and the detected categories are changed from multiple categories to two categories.

[0110] The Loss design includes the following modules: a training module, which is used to pass the classification weights from the first stage to the classification regression in the second stage, and uses maximum likelihood estimation to train the parameters of the target class and the background class respectively; and a testing module, which is used to obtain the final classification weights by multiplying them according to the conditional probabilities.

[0111] Furthermore, the aforementioned structural modules are also used for:

[0112] The RPN portion of the first stage in the two-stage process is replaced with a binary classifier of preset strength to provide candidate regions. The binary classifier of preset strength is linked with the subsequent prediction regression training in the second stage, and the loss of samples that are difficult to classify is transferred to the binary classifier of preset strength in the first stage for optimization.

[0113] Furthermore, system 10 also includes a data preprocessing module, which includes:

[0114] The inspection module is used to denoise and extract the PCB area image of the board to be inspected, and obtain the PCB image to be inspected.

[0115] The color correction module is used to extract the PCB area image from the standard board by denoising, and to perform color correction on the PCB image to be inspected based on the color information.

[0116] The image registration module is used to register, merge, and mesh the corrected PCB image with the PCB area image extracted by denoising the standard board.

[0117] According to an embodiment of the present invention, a deep learning-based printed circuit board (PCB) defect detection system performs target detection inference by inputting the original PCB image into a deep neural network. Before target detection inference, the original PCB image undergoes data preprocessing and background and edge extraction. A deep learning method is used to align the original PCB image with a standard board image. After alignment, a deep target detection model is used for defect detection; this model is a two-stage target detection model integrating a single-stage target detection algorithm. Based on the deep target detection model, defect detection of the original PCB image is completed, and the detection results are output. This invention optimizes the efficiency of the defect detection scheme and process; and while ensuring the recall rate of PCB appearance defect detection, it improves the detection accuracy. It also has broad prospects for academic research and application, and can truly contribute to the PCB industry.

[0118] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0119] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0120] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0121] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0122] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0123] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0124] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0125] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting defects in printed circuit boards based on deep learning, characterized in that, Includes the following steps: The original PCB image is input into a deep neural network for target detection inference; Before object detection inference, the original PCB image is preprocessed and background and edge extraction is performed; wherein, deep learning methods are used to align the original PCB image with a standard board image. After aligning the original PCB image with the standard board image, a depth target detection model is used for defect detection. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm. Based on the aforementioned depth target detection model, the defect detection of the original PCB image is completed, and the detection results are output. The two-stage target detection model of the integrated single-stage target detection algorithm includes: a structural part and a loss design part; wherein, The structural component includes: directly replacing the two-stage RPN component with a single-stage detection model, and changing the detected categories from multiple categories to two categories; The Loss design includes: a training phase in which the classification weights from the first phase are passed to the classification regression in the second phase, and the parameters of the target class and the background class are trained using maximum likelihood estimation; and a testing phase in which the final classification weights are obtained by multiplying them according to the conditional probabilities. The two-stage RPN component is directly replaced with a single-stage detection model, reducing the number of detected categories from multiple to two, including: The RPN portion of the first stage in the two-stage process is replaced with a binary classifier of preset strength to provide candidate regions. The binary classifier of preset strength is linked with the subsequent prediction regression training in the second stage, and the loss of samples that are difficult to classify is transferred to the binary classifier of preset strength in the first stage for optimization.

2. The deep learning-based printed circuit board defect detection method according to claim 1, characterized in that, The step of aligning the original PCB image with the standard board image using a deep learning method includes: The ability to perceive image content is acquired through autonomous learning of masks, where H ab For the original image I a Convert to standard plate image I b The homography matrix has the following overall loss function: L n The definitions of L are as follows: L(I a ,I b )=||F a -F b ||1 I / a =H ab I a I / b =H ba I b Among them, L n Constraints indicate the desire for I a Through the output H ab After conversion, we get Figure I. / a Features and I b Approximately, F represents network training, add H ab and H ba Constraints of reciprocal relations, I a with I b The conversion method is the opposite.

3. The deep learning-based printed circuit board defect detection method according to claim 1, characterized in that, The data preprocessing includes: The PCB area image is extracted by denoising the board to be inspected to obtain the PCB image to be inspected; The standard board is denoised to extract the PCB area image, and the color of the PCB image to be detected is corrected according to the color information. The corrected PCB image is registered, merged, and meshed with the PCB area image extracted by denoising the standard board.

4. A deep learning-based printed circuit board defect detection system, characterized in that, include: The input module is used to input the raw PCB image into the deep neural network for target detection inference. The alignment module is used to perform data preprocessing and background and edge extraction on the original PCB image before target detection inference; wherein, a deep learning method is used to align the original PCB image with a standard board image. The detection module is used to align the original PCB image with the standard board image and then use a depth target detection model to perform defect detection. The depth target detection model is a two-stage target detection model that integrates a single-stage target detection algorithm. The output module is used to perform defect detection on the original PCB image based on the depth target detection model and output the detection results. The detection module includes: a structural component module and a loss design component module; wherein, The structural module is used to: directly replace the two-stage RPN part with a single-stage detection model, and change the detected categories from multiple categories to two categories; The Loss design module includes: a training module, used to pass the classification weights from the first stage to the classification regression in the second stage, and to train the parameters of the target class and the background class respectively using maximum likelihood estimation; and a testing module, used to obtain the final classification weights by multiplying them according to conditional probabilities. The structural module is also used for: The RPN portion of the first stage in the two-stage process is replaced with a binary classifier of preset strength to provide candidate regions. The binary classifier of preset strength is linked with the subsequent prediction regression training in the second stage, and the loss of samples that are difficult to classify is transferred to the binary classifier of preset strength in the first stage for optimization.

5. The deep learning-based printed circuit board defect detection system according to claim 4, characterized in that, The alignment module is also used for: The ability to perceive image content is acquired through autonomous learning of masks, where H ab For the original image I a Convert to standard plate image I b The homography matrix has the following overall loss function: L n The definitions of L are as follows: L(I a ,I b )=||F a -F b ||1 I / a =H ab I a I / b =H ba I b Where L n Constraints indicate the desire for I a Through the output H ab After conversion, we get Figure I. / a Features and I b Approximately, F represents network training, add H ab and H ba Constraints of reciprocal relations, I a with I b The conversion method is the opposite.

6. The deep learning-based printed circuit board defect detection system according to claim 4, characterized in that, It also includes a data preprocessing module, which comprises: The inspection module is used to denoise and extract the PCB area image of the board to be inspected, and obtain the PCB image to be inspected. The color correction module is used to extract the PCB area image from the standard board by denoising, and to perform color correction on the PCB image to be detected based on the color information. The image registration module is used to register, merge, and mesh the corrected PCB image with the PCB area image extracted by denoising the standard board.

Images