Printed circuit board defect detection method and equipment based on incremental learning

Abstract

The present invention discloses a method and device for detecting printed circuit board defects based on incremental learning, comprising the following steps: Step 1: For training of the PCB defect base class in the incremental task, a standard classification process is employed; Step 2: For training of the PCB defect incremental class, random event selection is first employed to generate an extensible feature representation and calculate defect prototype features; Step 3: Dynamic relational projection is employed to construct a correlation matrix between the features of the old PCB defect prototype and new defect samples; Step 4: The correlation matrix between the old and new defect samples and the features extracted from the new PCB defect are passed through a classifier to calculate the loss and optimize. The present invention utilizes random event selection to improve the scalability and optimization capabilities of the PCB defect class feature representation; and utilizes a self-improving defect prototype refinement mechanism to enhance the representation capabilities of new defect classes (such as scrap copper, spurious, and offset), while retaining the relationship features between existing PCB defects (such as leaks, rat bites, open circuits, and short circuits).

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology in the field of intelligent manufacturing, and in particular to a printed circuit board defect detection method and device based on incremental learning. Background Art

[0002] Industrial production is continuously evolving towards intelligent manufacturing. Printed circuit boards (PCBs) are widely used in a variety of electronic products, representing a massive market. However, visual defect inspection for product quality during PCB manufacturing adds significant costs to PCB production. Deep convolutional neural networks have achieved significant breakthroughs in a wide range of object detection tasks, making significant contributions to PCB defect detection. When classes are pre-defined and samples are sufficient, typical supervised learning can yield good recognition models. However, in practice, new defect classes not seen in the continuous data stream are likely to be encountered, requiring them to be added to the recognition task. This creates a problem of incremental learning. In this scenario, retraining the model on both new and old data is both time-consuming and computationally expensive. In many cases, old data may not be available due to data privacy concerns or limited storage space. A common solution is to fine-tune the existing model with the newly added defect data, but this can lead to the problem of forgetting features from old defect classes. Existing methods assume the availability of a large number of samples for new defect classes, whereas incremental defect classes are often atypical and have a small sample size. In industrial defect detection, as equipment ages, various new defects may appear in produced PCBs. These defect samples may not only be significantly different from existing samples, but also be small in number. Representation optimization and prototype reconstruction for these new defect classes are difficult to accomplish with minimal supervision, significantly complicating the detection task. This paper focuses on the ability to incrementally learn new defect classes from a small number of PCB defect samples (small-sample incremental learning). Summary of the Invention

[0003] The present invention proposes a printed circuit board defect detection method based on incremental learning, which can solve at least one of the above technical problems.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A printed circuit board defect detection method based on incremental learning comprises the following steps:

[0006] Step 1: For the training of the PCB defect base class in the incremental task, a standard classification process is used;

[0007] Step 2: For the training of PCB defect increment class, firstly, random event selection is used to generate scalable feature representation and calculate defect prototype features;

[0008] Step 3: Use dynamic relational projection to construct the correlation matrix between the prototype features of the old PCB defect class and the new defect class samples;

[0009] Step 4: The correlation matrix between the new and old class defect samples and the features extracted from the new class defects of the PCB are passed through the classifier to calculate the loss and optimize.

[0010] Furthermore, the step S1 specifically includes the following sub-steps S11 to S14:

[0011] S11: Input the query to be predicted containing defect image Q;

[0012] S12: Using basic feature extractor g e , by θ e Parameterized VGG or ResNet to learn the corresponding PCB base class defect representation:

[0013] M q =g e (Q; δ e )

[0014] S13: Using δ m Parameterized metric l m To measure the representation of all PCB defect classes and the learnable old class defect prototype δ p The relationship between:

[0015] S=softmax(l m (M q ,δ p ; δ m ))

[0016] where l m Represents a variety of classifiers, including non-parametric and parametric classifiers;

[0017] S14: Randomly extract query images from the pre-built PCB defect dataset, train and optimize δ, and minimize the loss function Loss under the supervision of the target label T:

[0018]

[0019] where δ * Including δ e , δ p and δ m ; Loss represents the cross entropy loss function.

[0020] Furthermore, the step S2 specifically includes the following sub-steps S21 to S23:

[0021] S21: A random event selection strategy is introduced into the learning process, and an incremental event with N defect categories and K defect samples per category is generated in each iteration. The query image Q and a set C of K defect samples randomly selected from the PCB defect base training set X1 are input into the defect detection model.

[0022] S22: In each iteration, from the known defect label space Y 1 Randomly sample N categories of defects from the dataset, and then select K samples as feature extractors; average the number of embeddings obtained for each PCB defect class:

[0023] M s =mean(g e (C; δ e ))

[0024] S23: Assuming that these N categories of defects have not been seen before this iteration, their corresponding prototypes will be eliminated,

[0025]

[0026] Where n(Y 1 ) represents the defect label set Y 1 The number of categories in , C represents the possible permutations in the defect set.

[0027] Furthermore, the step S3 specifically includes the following sub-steps S31 to S33:

[0028] S31: Utilizing the self-improvement prototype refinement mechanism h u Maintain the dependency of the original PCB defects and enhance the ability to distinguish new types of PCB defects; first, the defect class embedding and the prototype features of the old class defects are converted into a shared latent space,

[0029]

[0030]

[0031] in and Represents a set of standard convolutional blocks, including 1×1 convolution, batch normalization layer and ReLU activation layer;

[0032] S32: Calculate the cosine similarity between the old class defects and the new class defects in this space, thereby obtaining the relationship matrix Rela between the new and old class PCB defects;

[0033]

[0034]

[0035] S33: Use the relationship matrix Rela as the transfer coefficient for defect prototype feature refinement:

[0036]

[0037] Furthermore, the step S4 specifically includes the following sub-steps S41 to S42:

[0038] S41: Classify the query image Q given the corresponding defect class embedding and defect prototype features, that is:

[0039]

[0040] S42: Feature Representation δ e The optimization process of the parameters in [1] is integrated as follows. Under the condition of satisfying the mutual relationship projection, the PCB defect prototype features can be learned and jointly optimized under the representation of the selected set S:

[0041]

[0042] On the other hand, the present invention further discloses a computer-readable storage device storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the above method.

[0043] As can be seen from the above technical solution, the present invention's incremental learning-based printed circuit board defect detection method specifically employs a novel incremental defect prototype learning scheme based on incremental learning from a small sample of PCB defect classes. To improve the scalability and optimization capabilities of defect feature representation, a random event selection strategy is proposed, followed by a self-improvement prototype refinement mechanism to reorganize all PCB defect prototypes. A dynamic relationship projection module utilizes the relationship metric between old defect prototypes and new defect samples to constrain prototype updates during training and testing.

[0044] This invention utilizes random event selection to improve the scalability and optimization capabilities of PCB defect feature representation. It also utilizes a self-improving defect prototype refinement mechanism to enhance the representation of new defect types (such as scrap copper, spurious, and offset), while preserving the relationship characteristics between existing PCB defects (such as leaks, rat bites, opens, and shorts). This invention maintains robustness in PCB defect detection, maintaining stable recognition of existing PCB defect category features while effectively detecting newly added PCB defect categories. This allows for high detection accuracy even with a small number of newly added PCB defect samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the method principle of the present invention;

[0046] Figure 2 This is a schematic diagram of the random event selection principle of an embodiment of the present invention;

[0047] Figure 3 This is a schematic diagram of a dynamic relationship projection according to an embodiment of the present invention;

[0048] Figure 4 This is the experimental result 1 of the embodiment of the present invention;

[0049] Figure 5 This is the experimental result 2 of the embodiment of the present invention;

[0050] Figure 6 This is an example diagram of the newly added defect type miscellaneous copper in an embodiment of the present invention;

[0051] Figure 7 This is a new small sample defect detection effect diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0053] like Figure 1 As shown, the printed circuit board defect detection method based on incremental learning described in this embodiment includes:

[0054] Step 1: For the training of the PCB defect base class in the incremental task, a standard classification process is adopted.

[0055] Step 2: For the training of PCB defect increment class, random event selection is first used to generate scalable feature representation and calculate defect prototype features.

[0056] Step 3: Use dynamic relational projection to construct the correlation matrix between the prototype features of the old class of PCB defects and the new class of defect samples.

[0057] Step 4: The correlation matrix between the new and old class defect samples and the features extracted from the new class defects of the PCB are passed through the classifier to calculate the loss and optimize.

[0058] Specific implementation steps:

[0059] In particular, for the above step S1: for the training of the PCB defect base classes (leak holes, rat bites, open circuits and short circuits, etc.) in the incremental task, a standard classification process is adopted. Specifically, the following sub-steps S11 to S14 are included:

[0060] S11: Input the query defect image Q to be predicted.

[0061] S12: Using basic feature extractor g e , by θ e Parameterized VGG or ResNet to learn the corresponding PCB base class defect representation:

[0062] M q =g e (Q; δ e )

[0063] S13: Using δ m Parameterized metric l m To measure the representation of all PCB defect classes and the learnable old class defect prototype δ p The relationship between:

[0064] S=softmax(l m (M q ,δ p ; δ m ))

[0065] where l m A variety of classifiers can be represented, including non-parametric classifiers and parametric classifiers. The present invention uses a cosine classifier.

[0066] S14: Randomly extract query images from the pre-built PCB defect dataset, train and optimize δ, and minimize the loss function Loss under the supervision of the target label T:

[0067]

[0068] where δ * Including δ e , δ p and δ m In the present invention, Loss represents the cross entropy loss function.

[0069] Furthermore, regarding the above step S2: for the training of PCB defect increment classes (such as scrap copper, spurious, and offset), firstly, random event selection is used to generate scalable defect class feature representation and calculate defect prototype features. Specifically, the following sub-steps S21 to S23 are included:

[0070] S21: Introducing a random event selection strategy in the learning process (see Figure 2 ), and generates an incremental event of N categories of defects in each iteration, with each category having K defect samples. The query image Q and a set C of K defect samples of N categories randomly selected from the PCB defect base training set X1 are input into the defect detection model.

[0071] S22: Figure 1 As shown, in each iteration, from the known defect label space Y1 Randomly sample N categories of defects from , and then select K samples as feature extractors. The average number of embeddings obtained for each PCB defect class is:

[0072] M s =ean(g e (C; δ e ))

[0073] S23: Assuming that these N categories of defects have not been seen before this iteration, their corresponding prototypes will be eliminated,

[0074]

[0075] Where n(Y 1 ) represents the defect label set Y 1 The number of categories in , C represents the possible permutations in the defect set.

[0076] Furthermore, regarding the above step S3: using dynamic relational projection to construct a correlation matrix between the old class defect prototypes and the newly added class defect samples, the following sub-steps S31 to S33 are specifically included:

[0077] S31: Utilizing the self-improvement prototype refinement mechanism h u Maintain the dependency of the original PCB defects and enhance the ability to distinguish new types of PCB defects. First, the defect class embedding and the prototype features of the old class defects are converted into a shared latent space.

[0078]

[0079]

[0080] in and Represents a set of standard convolutional blocks, including 1×1 convolution, batch normalization layer and ReLU activation layer.

[0081] S32: Calculate the cosine similarity between the old class defects and the new class defects in this space, thereby obtaining the relationship matrix Rela between the old and new class PCB defects.

[0082]

[0083]

[0084] S33: Use the relationship matrix Rlea as the transfer coefficient for defect prototype feature refinement:

[0085]

[0086] Furthermore, regarding the above step S4: the new and old PCB defect correlation matrix and the features extracted from the new PCB defect are put through the classifier to calculate the loss and optimize. Specifically, the following sub-steps S41 to S42 are included:

[0087] S41: Classify the query image Q given the corresponding defect class embedding and defect prototype features, that is:

[0088]

[0089] S42: Feature Representation δ e The optimization process of the parameters in [1] is integrated as follows. Under the condition of satisfying the mutual relationship projection, the PCB defect prototype features can be learned and jointly optimized under the representation of the selected set S:

[0090]

[0091] In summary, the advantages of the incremental learning method for a small number of newly added defect category samples in the present invention, that is, the printed circuit board defect detection method based on incremental learning, are: using random event selection to improve the scalability and optimization capability of PCB defect category feature representation; using a self-improving defect prototype refinement mechanism to enhance the expression capability of new defect categories (such as miscellaneous copper, strays, and offsets), while retaining the relationship characteristics between the original PCB defects (such as leaks, rat bites, open circuits, and short circuits).

[0092] The following are some specific examples:

[0093] The present invention first conducted comprehensive experiments on datasets such as CIFAR-100, MiniImageNet, and CUB200. Using ResNet-18 as the backbone network, the same accuracy was achieved on all base class (i.e., session = 1) datasets. The fine-tuning method (Ft-CNN) was set as the baseline, and some regularization techniques were used for comparison. The SGD optimizer was used during training, with the initial learning rate set to 0.02 and the decay rate set to 0.0005. It was compared with the state-of-the-art methods of FSCIL and some classic methods of CIL (see Figure 4 ,5), this method surpasses the SOTA (state-of-the-art) method by 13% on the CIFAR-100 dataset, improves by 17% on the MiniImageNet dataset, and improves by 11% on the CUB200 dataset, proving the effectiveness of this method.

[0094] At the same time, the present invention uses several types of PCB defects to test the method. Based on the four known types of PCB defects (leak holes, rat bites, open circuits and short circuits, etc.), a new type of defective copper is added. Figure 6 shown.

[0095] The effectiveness of this method is tested when there are few new samples. Figure 7 , proving the effectiveness of this method in the field of new defect detection in intelligent manufacturing PCBs.

[0096] In another aspect, the present invention further discloses a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of any of the above methods.

[0097] On the other hand, the present invention further discloses a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of any of the above methods.

[0098] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any one of the methods in the above embodiments.

[0099] It is understandable that the system provided by the embodiment of the present invention corresponds to the method provided by the embodiment of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts of the above method.

[0100] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0101] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A printed circuit board defect detection method based on incremental learning, characterized in that: The following steps are included: Step S1: For the training of the PCB defect base class in the incremental task, a standard classification process is adopted; Step S2: For the training of PCB defect increment class, firstly, random event selection is used to generate scalable feature representation and calculate defect prototype features; Step S3: Using dynamic relational projection to construct the correlation matrix between the prototype features of the old PCB defect and the new defect samples; Step S4: The correlation matrix between the old and new defect samples and the features extracted from the new PCB defect are passed through the classifier to calculate the loss and optimize; The step S2 specifically includes the following sub-steps S21 to S23: S21: A random event selection strategy is introduced into the learning process, and an incremental event with N defect categories and K defect samples per category is generated in each iteration. The query image Q and a set C of K defect samples randomly selected from the PCB defect base training set X1 are input into the defect detection model. S22: In each iteration, from the known defect label space Randomly extract N categories of defects and then select K samples as feature extractors; Average the number of embeddings obtained for each PCB defect class: S23: Assuming that these N categories of defects have not been seen before this iteration, their corresponding prototypes will be eliminated; The step S3 specifically includes the following sub-steps S31 to S33: S31: Utilizing the self-improvement prototype refinement mechanism Maintaining the dependency of the original PCB defects and enhancing the ability to distinguish new types of PCB defects; first, the defect class embedding and the prototype features of the old class defects are converted into a shared latent space; S32: Calculate the cosine similarity between the old and new class defects in this space, and thus obtain the relationship matrix between the old and new class PCB defects ; S33: Relationship Matrix Transfer coefficient as feature refinement of defect prototype.

2. The method for detecting defects in a printed circuit board based on incremental learning according to claim 1, wherein: The step S1 specifically includes the following sub-steps S11 to S14: S11: Input the query to be predicted containing defect image Q; S12: Using basic feature extractor ,Depend on Parameterized VGG or ResNet to learn the corresponding PCB base class defect representation: S13: Exploit Parameterized metrics To measure the representation of all PCB defect classes and learn old class defect prototypes The relationship between: in Represents a variety of classifiers, including non-parametric and parametric classifiers; S14: Randomly extract query images from the pre-built PCB defect dataset, train and optimize , minimize the loss function under the supervision of the target label T Loss : in Included , and ; Loss represents the cross entropy loss function.

3. The method for detecting printed circuit board defects based on incremental learning according to claim 2, wherein: In step S13 The cosine classifier is used.

4. The method for detecting defects in a printed circuit board based on incremental learning according to claim 1, wherein: The step S4 specifically includes the following sub-steps S41 to S42: S41: Classify the query image Q given the corresponding defect class embedding and defect prototype features, that is: S42: Feature Representation The optimization process of the parameters in [1] is integrated as follows. Under the condition of satisfying the mutual relationship projection, the PCB defect prototype features can be learned and jointly optimized under the representation of the selected set S: 。 5. The method for detecting defects in a printed circuit board based on incremental learning according to claim 1, wherein: The base categories of PCB defects in step S1 include leak holes, rat bites, open circuits, and short circuits.

6. The method for printed circuit board defect detection based on incremental learning according to claim 1, wherein: The PCB defect increment categories in step S2 include scrap copper, stray and offset.

7. A computer-readable storage device storing a computer program, wherein when the computer program is executed by a processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 6.

Images