A printed circuit board defect detection method based on non-example-based incremental learning

Abstract

The present invention discloses a printed circuit board defect detection method based on non-example-based incremental learning, comprising acquiring a PCB image and performing a defect feature representation learning process based on prototype rehearsal; acquiring representative examples of old-class defects based on a dynamic structural reorganization strategy to ensure unbiased training of new-class defects; adopting a defect prototype selection mechanism based on an extensible embedding space to reduce confusion between old and new class defect features in the distillation part; and classifying defects through balanced calibration to obtain printed circuit board defect results. The present invention implements cyclic extension optimization through a dynamic structural reorganization strategy to obtain a structurally invariant representation of defect features. Main branch distillation is used to maintain the new network's discrimination of old defect features by aligning the invariant distribution defect feature knowledge on the old defect class. A prototype selection mechanism is used to combine the retained invariant defect knowledge with the input new defect samples to reduce feature confusion between similar defect classes.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent manufacturing defect detection, and in particular to a printed circuit board defect detection method based on non-example-based incremental learning. Background Art

[0002] With the development of the electronics industry, printed circuit boards (PCBs) are increasingly used, driving a growing demand for PCB production. During the PCB manufacturing process, defects can occur due to factors such as equipment aging. PCB defect detection is a time-consuming and labor-intensive process, increasing production costs. With the advent of deep learning and intelligent manufacturing, PCB defect detection has become intelligent. Deep neural networks have made significant progress under fully supervised conditions, leading to the development of PCB defect detection. When defect categories are known in advance and sufficient defect samples are available, training deep neural networks can yield effective recognition models. However, in practice, new defect classes not seen in continuous defect data are likely to be encountered, requiring them to be added to the recognition task. This presents the problem of incremental learning. In this scenario, retraining on all new and old defect samples is both time-consuming and computationally expensive. Furthermore, in many cases, due to data privacy concerns or limited storage space, old defect data may not be stored, leading to the gradual forgetting of old examples during model training. This significantly complicates the optimization of defect recognition models. This is known as non-example-based incremental learning in incremental learning. If new defect data is used to fine-tune the existing model, the problem of catastrophic forgetting of old defect class features may occur, which brings great difficulties to the detection task. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings of the existing technology. To achieve the above purpose, a printed circuit board defect detection method based on non-example-based incremental learning is adopted to solve the problems raised in the above background technology.

[0004] A printed circuit board defect detection method based on non-example-based incremental learning, comprising the following steps:

[0005] Step S1, obtaining a PCB image and performing a defect feature representation learning process based on prototype rehearsal;

[0006] Step S2: obtaining representative examples of old defects based on the dynamic structure reorganization strategy to ensure unbiased training of new defects;

[0007] Step S3: Using a defect prototype selection mechanism based on an extensible embedding space to reduce the confusion of new and old defect features in the distillation part;

[0008] Step S4: classify the defects through balance calibration to obtain printed circuit board defect results.

[0009] As a further solution of the present invention: the specific steps in step S1 include:

[0010] Step S11: Obtain a defective PCB image Q and use it as input for defect feature representation learning;

[0011] Step S12: In the m=1 basic stage, the defect feature extractor and defect classifier Standard defect classification model composed of Optimize under the supervision of training defect labels;

[0012] Step S13: In the m>1 increment stage, a basic defect feature extractor is used To learn the defect types, the corresponding representation is:

[0013]

[0014] Where, is the defect feature representation, is the defect feature extractor, Q is the input PCB image, are the trainable parameters of the feature extractor.

[0015] As a further solution of the present invention: the specific steps in step S2 include:

[0016] Step S21: Introduce a dynamic structural reorganization strategy during training and use structural expansion to add side branches to the current model block by block;

[0017] A residual adapter is inserted into each convolutional block of the fixed defect feature extractor in the previous training stage. The optimized defect feature flow is propagated only through the adapter, updating the most obvious locations while keeping the old defect features, which is expressed as:

[0018]

[0019] Where D t represents a linear transformation, represents fixed parameters, Represents a structure extension operation, Indicates the parameters of the adapter, represents the defect feature extractor in the previous stage;

[0020] Step S22: Use structural reparameterization to losslessly integrate the side branch defect feature information into the main branch. Through zero-filling operation and linear transformation, the parameters in the remaining structure are fused with the parameters of the original convolution kernel and batch normalization. The fusion formula is:

[0021]

[0022] Where, represents the updated defect feature extractor parameters;

[0023] Step S23: remove the adapter and keep the defect recognition network structure unchanged for the next update.

[0024] As a further solution of the present invention: the specific steps in step S3 include:

[0025] Step S31: Use the fully connected layer as the defect classifier Map defect feature representation to defect label space:

[0026]

[0027] Where, is the mapping of defect characteristics, is the defect classifier, is the trainable parameter of the defect classifier, U en represents the standard cross entropy loss function, is the defect label in the input image Q, Loss en Represents the cross entropy loss of the model;

[0028] Step S32: Use knowledge distillation to measure the obtained defect feature representation and the previous model Defect feature similarity between:

[0029]

[0030] Where U dis represents the Euclidean distance, and are the defect feature representations of the current stage and the previous stage, respectively. dis represents the distillation loss of the model;

[0031] Step S33: Calculate the normalized cosine score T between the newly added defect sample and all defect prototypes:

[0032]

[0033] Where Cos is the cosine operation, Ba represents the regularization operation, N represents the newly added defect samples, and Prototype represents all defect prototypes;

[0034] Step S34: Set a threshold. If T is greater than the threshold, add a mask M to the corresponding position of the distillation loss. dis , if T is less than the threshold, then add a mask M to the corresponding part of its cross entropy lossen , and finally the defect prototype balance loss is used as the final optimization loss function in the new stage:

[0035] Loss=M en (Loss eb )+αM dis (Liss diss )+βLoss p ;

[0036] Where α and β are the weights of the loss function, Loss p Prototype balancing loss for the model.

[0037] As a further solution of the present invention: the specific steps in step S4 include:

[0038] Step S41: upsampling the defect sample prototype to a set batch size to achieve calibration of the defect classifier;

[0039]

[0040] Where, is the upsampling operation, r A is the defect feature prototype after upsampling, y B is the upsampled defect label set of the initial defect prototype;

[0041] Step S42: Optimize and train the defect recognition model based on the loss function obtained previously.

[0042] Compared with the prior art, the present invention has the following technical effects:

[0043] Adopting the aforementioned technical solution, a self-sustaining representation expansion scheme is employed, implementing cyclic expansion optimization through a dynamic structural reorganization strategy to obtain a structurally invariant representation of defect features. Main branch distillation is used to maintain the new network's ability to distinguish between old defect features by aligning the invariant defect feature knowledge on old defect classes. A prototype selection mechanism combines the retained invariant defect knowledge with the input of new defect samples to reduce feature confusion between similar defect classes. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings:

[0045] Figure 1 A schematic diagram of the steps of the detection method disclosed in the embodiment of the present application;

[0046] Figure 2 A flowchart of the detection method disclosed in the present application;

[0047] Figure 3A schematic diagram of dynamic structural reorganization of an embodiment disclosed in this application;

[0048] Figure 4 Schematic diagram of the main branch distillation and prototype equilibrium of the embodiment disclosed in this application;

[0049] Figure 5 This is a schematic diagram of an example of a newly added defect in the embodiment disclosed in this application;

[0050] Figure 6 This is a schematic diagram of the new type of defect detection effect of the embodiment disclosed in this application. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0052] Please refer to Figure 1 and Figure 2 In an embodiment of the present invention, a printed circuit board defect detection method based on non-example-based incremental learning comprises the following steps:

[0053] Step S1: Acquire a PCB image and perform defect feature representation learning based on prototype rehearsal. The specific steps include:

[0054] S11: Input a PCB image Q containing defects.

[0055] S12: In the m=1 basic stage, the defect feature extractor and defect classifier Standard defect classification model composed of Optimization is performed under supervision with training defect labels.

[0056] S13: In the m>1 increment stage, a basic defect feature extractor is used To learn the corresponding representation of defect types:

[0057]

[0058] Where, is the defect feature representation, is the defect feature extractor, Q is the input PCB image, are the trainable parameters of the feature extractor.

[0059] Step S2: Obtain representative examples of old-class defects based on the dynamic structural reorganization strategy to ensure unbiased training of new-class defects. The specific steps include:

[0060] S21: Introduce dynamic structural reorganization strategies during training, such as Figure 3 The figure shows dynamic structural reorganization, which uses structural expansion to add side branches to the current model block by block. A residual adapter is inserted into each convolutional block of the fixed defect feature extractor from the previous training stage. The optimized defect feature flow is propagated only through the adapter, updating the most obvious locations while maintaining the old defect features.

[0061]

[0062] Where D t represents a linear transformation, represents fixed parameters, Represents a structure extension operation, Indicates the parameters of the adapter, Represents the defect feature extractor in the previous stage.

[0063] S22: Use structural reparameterization to losslessly integrate side branch defect feature information into the main branch. Through zero-padding operations and linear transformations, the parameters in the remaining structure are fused with the parameters of the original convolution kernel and batch normalization.

[0064]

[0065] Where, Represents the updated defect feature extractor parameters.

[0066] S23: Remove the adapter and keep the defect identification network structure unchanged for the next update.

[0067] Step S3: Using a defect prototype selection mechanism based on an extensible embedding space to reduce the confusion between new and old defect features in the distillation part. The specific steps include:

[0068] like Figure 4 As shown, the diagram is a schematic diagram of the main branch distillation and prototype equilibrium, that is, the defect prototype selection mechanism.

[0069] S31: Using fully connected layers as defect classifiers Map defect feature representation to defect label space:

[0070]

[0071] Where, is the mapping of defect characteristics, is the defect classifier, is the trainable parameter of the defect classifier, Uen represents the standard cross entropy loss function, is the defect label in the input image Q, Loss en Represents the cross entropy loss of the model.

[0072] S32: Use knowledge distillation to measure the obtained defect feature representation and the previous model Defect feature similarity between:

[0073]

[0074] Where U dis represents the Euclidean distance, and are the defect feature representations of the current stage and the previous stage, respectively. dis Represents the distillation loss of the model.

[0075] S33: Calculate the normalized cosine score T between the newly added defect sample and all defect prototypes.

[0076]

[0077] Where Cos is the cosine operation, Ba represents the regularization operation, N represents the newly added defect samples, and Protorype represents all defect prototypes.

[0078] S34: Set a threshold. If T is greater than the threshold, add a mask M to the corresponding position of its distillation loss. dis , if T is less than the threshold, then add a mask M to the corresponding part of its cross entropy loss en Finally, the defect prototype balance loss is used as the final optimization loss function in the new stage.

[0079] Loss=M en (Loss en )+αM dis (Loss diss )+βLoss p ;

[0080] Where α and β are the weights of the loss function, set to 10, Loss p Prototype balancing loss for the model.

[0081] Step S4: classify defects through balance calibration to obtain printed circuit board defect results. The specific steps include:

[0082] S41: Upsample the defect sample prototypes to the set batch size to achieve calibration of the defect classifier.

[0083]

[0084] Where, is the upsampling operation, r A is the defect feature prototype after upsampling, y B is the upsampled defect label set of the initial defect prototype.

[0085] S42: Optimize and train the defect recognition model based on the loss function obtained previously.

[0086] Beneficial effects of the present invention:

[0087] A dynamic structural reorganization strategy is proposed. While maintaining the distribution of old features along the structural expansion direction, it optimizes the newly added incremental features in the side branches and then arranges the distillation process in the main branch. In particular, a prototype selection mechanism is integrated into the joint training to strengthen the distinction between new and old categories. This method can effectively distinguish between new and old defect categories, reduce confusion between different defect categories, maintain the learning ability of old defect categories, and improve detection accuracy.

[0088] On the basis of the original defects (open circuit, short circuit and rat bite), a new type of miscellaneous copper defect is added and the effectiveness of this method is tested. Figure 5 As shown in the figure, the new defect example is as follows: Figure 6 As shown in the figure, the new defect detection effect is demonstrated, which proves the effectiveness of the method of the present invention in the field of new defect detection in intelligent manufacturing PCB.

[0089] Although the embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included within the scope of protection of the present invention.

Claims

1. A printed circuit board defect detection method based on non-example-based incremental learning, characterized in that: The specific steps include: Step S1, obtaining a PCB image and performing a defect feature representation learning process based on prototype rehearsal; Step S2: obtaining representative examples of old defects based on the dynamic structure reorganization strategy to ensure unbiased training of new defects; Step S3: Using a defect prototype selection mechanism based on an extensible embedding space to reduce the confusion of new and old defect features in the distillation part. The specific steps include: Step S31: Use the fully connected layer as the defect classifier , mapping the defect feature representation to the defect label space: ， ； Where, is the mapping of defect characteristics, is the defect classifier, are the trainable parameters of the defect classifier, represents the standard cross entropy loss function, is the defect label in the input image Q, Represents the cross entropy loss of the model; Step S32: Use knowledge distillation to measure the obtained defect feature representation and the previous model Defect feature similarity between: ， ； Where, represents the Euclidean distance, and are the defect feature representations of the current stage and the previous stage respectively, represents the distillation loss of the model; Step S33: Calculate the normalized cosine score between the newly added defect sample and all defect prototypes for: ； Where, is the cosine operation, represents the regularization operation, Indicates a new defect sample. Represents all defect prototypes; Step S34: Set the threshold. If If it is greater than the threshold, a mask is added to the corresponding position of its distillation loss ,if If it is less than the threshold, a mask is added to the corresponding part of its cross entropy loss , and finally the defect prototype balance loss is used as the final optimization loss function in the new stage: ； Where, and is the loss function weight, Prototype balancing loss for the model; Step S4: classify the defects through balance calibration to obtain printed circuit board defect results.

2. The printed circuit board defect detection method based on non-example-based incremental learning according to claim 1, characterized in that: The specific steps in step S1 include: Step S11: Obtain a defective PCB image Q and use it as input for defect feature representation learning; Step S12: In the m=1 basic stage, the defect feature extractor and defect classifier Standard defect classification model composed of Optimize under the supervision of training defect labels; Step S13: In the m>1 increment stage, a basic defect feature extractor is used To learn the defect types, the corresponding representation is: ； Where, is the defect feature representation, is the defect feature extractor, To input PCB image, are the trainable parameters of the feature extractor.

3. The printed circuit board defect detection method based on non-example-based incremental learning according to claim 1, characterized in that: The specific steps in step S2 include: Step S21: Introduce a dynamic structural reorganization strategy during training and use structural expansion to add side branches to the current model block by block; A residual adapter is inserted into each convolutional block of the fixed defect feature extractor in the previous training stage. The optimized defect feature flow is propagated only through the adapter, updating the most obvious locations while keeping the old defect features, which is expressed as: ； Where, represents a linear transformation, Indicates fixed parameters, ⊕ indicates structure extension operation, Indicates the parameters of the adapter, represents the defect feature extractor in the previous stage; Step S22: Use structural reparameterization to losslessly integrate the side branch defect feature information into the main branch. Through zero-filling operation and linear transformation, the parameters in the remaining structure are fused with the parameters of the original convolution kernel and batch normalization. The fusion formula is: ； Where, represents the updated defect feature extractor parameters; Step S23: remove the adapter and keep the defect recognition network structure unchanged for the next update.

4. The printed circuit board defect detection method based on non-example-based incremental learning according to claim 1, characterized in that: The specific steps in step S4 include: Step S41: upsampling the defect sample prototype to a set batch size to achieve calibration of the defect classifier; ， ； Where, is the upsampling operation, is the defect feature prototype after upsampling, is the upsampled defect label set of the initial defect prototype; Step S42: Optimize and train the defect recognition model based on the loss function obtained previously.

Images