Training and Classification Detection Methods for Small Sample PCB Defect Classification Models

Abstract

This embodiment discloses a training method and a classification detection method for a small-sample PCB defect classification model. The method includes: acquiring an initial training dataset for the small-sample PCB defect classification model and dividing it into several support sets and query sets; inputting the support sets and query sets into an embedding unit for feature extraction to obtain a first feature of the support set and a second feature of the query set, wherein the embedding unit is a feature extraction network with a multi-branch INCO structure; performing target type measurement on the first and second features through a dual-metric network to obtain a target measurement result; when the target measurement result is greater than or equal to a preset threshold, a trained model is obtained; when the target measurement result is less than the preset threshold, model training continues by adjusting the parameters in the model. This method can efficiently and accurately detect surface defects in PCB products, thereby improving the detection accuracy and efficiency of small-sample PCB defect classification.

Description

Technical Field

[0001] This disclosure relates to the field of PCB surface defect detection technology, and in particular to a training method and classification detection method for a small-sample PCB defect classification model. Background Technology

[0002] Against the backdrop of rapid development in information technologies such as artificial intelligence, big data, and 5G, user demand for electronic information products is increasing. Printed circuit boards (PCBs), as the most crucial component of electronic information products, not only significantly impact the performance and safety of these products but also, to a certain extent, represent a nation's innovation and informatization level. Deep learning methods are widely used in defect detection, offering higher accuracy and robustness compared to traditional methods. However, deep learning methods generally require a large amount of defect data to train the deep learning neural network model to obtain a model with superior detection performance.

[0003] In related technologies, the short update cycle of PCB product production lines and the high cost of annotating a large number of PCB defect images make it difficult to obtain sufficient training samples. Furthermore, PCB defects exhibit diverse morphologies, with significant differences in the morphological features of some defect categories. This presents numerous challenges for deep learning neural network models in extracting and measuring defect features during defect classification and detection, resulting in low detection accuracy and efficiency for small-sample PCB defect classification. Summary of the Invention

[0004] In view of this, the present disclosure provides a training method and a classification detection method for a small sample PCB defect classification model, which can solve the problems of limited training samples and complex and diverse PCB defect morphologies in the existing small sample PCB defect classification detection methods. It can efficiently and accurately detect surface defects of PCB products, thereby improving the detection accuracy and efficiency of small sample PCB defect classification.

[0005] Firstly, this disclosure provides a training method for a small-sample PCB defect classification model, employing the following technical solution:

[0006] Obtain the initial training dataset for the small sample PCB defect classification model;

[0007] The training dataset is divided into several task datasets, wherein each task dataset contains a support set and a query set;

[0008] The support set and query set are input into the embedding unit for feature extraction to obtain the first feature of the support set and the second feature of the query set. The embedding unit is a feature extraction network with a multi-branch INCO structure.

[0009] The first feature and the second feature are measured by a dual-metric network to obtain the target measurement result.

[0010] When the target measurement result is greater than or equal to the preset threshold, the initial small sample PCB defect classification model is successfully trained, and a trained small sample PCB defect classification model is obtained.

[0011] When the target metric result is less than the preset threshold, the parameters in the initial small sample PCB defect classification model are adjusted, and the initial small sample PCB defect classification model is trained again until the initial small sample PCB defect classification model is successfully trained.

[0012] In some embodiments, the training dataset includes a public image dataset and a small sample PCB defect image dataset, and the method further includes:

[0013] The initial small sample PCB defect classification model is pre-trained using the public image dataset as the first training sample to obtain a pre-trained small sample PCB defect classification model; wherein, the pre-trained small sample PCB defect classification model classifies the query images in the query set using the support images in the support set;

[0014] The small sample PCB defect image dataset is classified and labeled according to different defect types;

[0015] The pre-trained small sample PCB defect classification model is fine-tuned using the classified and labeled small sample PCB defect image dataset as the second training sample to obtain the trained small sample PCB defect classification model.

[0016] In some embodiments, a target type measurement is performed on the first feature and the second feature using a dual-metric network to obtain a target measurement result, including:

[0017] The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual measurement network.

[0018] The second similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the cosine metric network in the dual metric network.

[0019] The target measurement result is obtained based on the average of the first similarity and the second similarity.

[0020] Both the relation metric network and the cosine metric network contain RFB receptive field blocks.

[0021] In some embodiments, a first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual-measure network, including:

[0022] Calculate the mean value of the first feature of all support images belonging to the same category in the support set, and use the mean value of the feature as the feature prototype of the corresponding category;

[0023] By concatenating the second feature of the query image in the query set with the feature prototypes of each category, several feature pairs are obtained.

[0024] The cascaded feature pairs are input into the relation measurement network to calculate the first similarity.

[0025] In some embodiments, the first similarity is calculated using the following formula:

[0026]

[0027] In the formula, P1 (q,c) Let represent the first similarity, that is, the first similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the first similarity score. f φ Indicates an embedded unit; x q x represents a query sample in the query set; s,c g1 represents the support samples in the support set; g1 represents the relation metric network; || represents the feature concatenation operation between the support image and the query image;

[0028] The first loss function of the relation metric network is calculated using the following formula:

[0029]

[0030] In the formula, Represents the first loss function; Indicates the query sample x q The first predicted value; y (q，j) Indicates the query sample x q The first true value.

[0031] In some embodiments, the second similarity is calculated using the following formula:

[0032]

[0033] In the formula, Let represent the second similarity, that is, the second similarity score of the q-th sample in the few-sample task. The one-hot vector of the category of the q-th sample is obtained through the second similarity score. f φ (x s,c ) indicates that in the embedded unit f φ Supporting sample x s,c ;f φ (x q ) indicates that in the embedded unit f φ The query sample in the text; g2 represents the cosine metric network; y rc This represents the RFB receptive field block and convolution block operations; y cos This indicates the calculation of cosine similarity layers;

[0034] The second loss function of the cosine metric network is calculated using the following formula:

[0035]

[0036] In the formula, This represents the second loss function; Indicates the query sample x q The second predicted value; y (q，j) Indicates the query sample x q The second true value.

[0037] Secondly, this disclosure also provides a method for classifying and detecting defects in small-sample PCBs, employing the following technical solution:

[0038] Acquire an image of the PCB board to be inspected;

[0039] The PCB board image to be detected is input into the small sample PCB defect classification model trained using the training method described above.

[0040] Obtain the classification and detection results of defects on the PCB board image to be detected, output by the small sample PCB defect classification model.

[0041] Thirdly, embodiments of this disclosure also provide a training apparatus for a small-sample PCB defect classification model, comprising:

[0042] The dataset acquisition unit is configured to acquire the initial training dataset for the small sample PCB defect classification model;

[0043] A dataset partitioning unit is configured to divide the training dataset into several task datasets, wherein each task dataset contains a support set and a query set;

[0044] The dataset input unit is configured to input the support set and query set into the embedding unit for feature extraction to obtain a first feature of the support set and a second feature of the query set, wherein the embedding unit is a feature extraction network with a multi-branch INCO structure;

[0045] The dual-metric unit is configured to measure the target type of the first feature and the second feature through a dual-metric network to obtain the target measurement result.

[0046] The first determination unit is configured such that when the target measurement result is greater than or equal to a preset threshold, the initial small sample PCB defect classification model is successfully trained, and a trained small sample PCB defect classification model is obtained.

[0047] The second determination unit is configured to, when the target measurement result is less than a preset threshold, adjust the parameters in the initial small sample PCB defect classification model and continue to train the initial small sample PCB defect classification model until the initial small sample PCB defect classification model is successfully trained.

[0048] Fourthly, embodiments of this disclosure also provide a classification and detection device for small-sample PCB defects, comprising:

[0049] The image acquisition unit is configured to acquire an image of the PCB board to be inspected;

[0050] The image input unit is configured to input the PCB board image to be detected into a small sample PCB defect classification model trained using the training method described above.

[0051] The classification result acquisition unit is configured to acquire the classification detection result of the defects on the PCB board image to be detected, output by the small sample PCB defect classification model.

[0052] Fifthly, embodiments of this disclosure also provide an electronic device, which adopts the following technical solution:

[0053] The electronic device includes:

[0054] At least one processor; and,

[0055] A memory communicatively connected to the at least one processor; wherein,

[0056] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform either the training method for the small sample PCB defect classification model described above, or the classification and detection method for small sample PCB defects described above.

[0057] This disclosure provides a training method and classification detection method for a small-sample PCB defect classification model. Addressing the weaknesses of embedded units in nonlinear feature learning and the diverse morphologies and significant intra-class feature differences in PCB surface defects, a multi-branch INCO structure feature extraction network is proposed. This effectively expands the depth and width of the embedded unit network while emphasizing the correlation between features of the same target, helping the embedded units extract higher-level and semantically richer defect features from the PCB board.

[0058] Secondly, to address the issue of simple metric unit network structures and the tendency for single metric methods to introduce similarity bias, a dual-metric network is constructed. This allows PCB defect features to be more closely clustered in the feature space. This method effectively solves the problem of low classification accuracy for small-sample PCB defects and has high practical application value compared to other small-sample classification algorithms.

[0059] In summary, the embodiments disclosed herein can solve the problems of limited training samples and complex and diverse PCB defect morphologies in small-sample PCB defect classification and detection methods. They can efficiently and accurately detect surface defects in PCB products, thereby improving the detection accuracy and efficiency of small-sample PCB defect classification.

[0060] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 A flowchart illustrating a training method for a small-sample PCB defect classification model provided in this embodiment of the disclosure;

[0063] Figure 2 A schematic diagram of small sample PCB defect types provided in this disclosure embodiment;

[0064] Figure 3 A schematic diagram of a multi-branch INCO structure provided in an embodiment of this disclosure;

[0065] Figure 4 A schematic diagram of the improved embedded unit structure provided in the embodiments of this disclosure;

[0066] Figure 5 This is a schematic diagram of the structure of a dual-metric network provided in an embodiment of the present disclosure;

[0067] Figure 6 This is a schematic diagram of the structure of the relation measurement network provided in the embodiments of this disclosure;

[0068] Figure 7 This is a schematic diagram of the structure of the RFB receptive field block provided in an embodiment of this disclosure;

[0069] Figure 8 This is a schematic diagram of the structure of the small-sample PCB defect classification model provided in the embodiments of this disclosure;

[0070] Figure 9 A schematic diagram comparing the feature distribution maps extracted by the original embedding unit and the improved new embedding unit provided in the embodiments of this disclosure;

[0071] Figure 10 A schematic flowchart of a method for classifying and detecting defects in a small sample PCB provided in this embodiment of the present disclosure;

[0072] Figure 11 A schematic diagram of the structure of a training device for a small-sample PCB defect classification model provided in an embodiment of this disclosure;

[0073] Figure 12 A schematic diagram of the structure of a small sample PCB defect classification and detection device provided in an embodiment of this disclosure;

[0074] Figure 13 This is a schematic block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0075] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0076] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0077] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0078] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0079] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0080] like Figure 1 As shown, Figure 1 This is a flowchart illustrating a training method for a small-sample PCB defect classification model provided in this embodiment of the disclosure. The training method for a small-sample PCB defect classification model provided in this embodiment of the disclosure includes the following steps:

[0081] S101. Obtain the initial training dataset for the small sample PCB defect classification model.

[0082] S102. Divide the training dataset into several task datasets, where each task dataset contains a support set and a query set.

[0083] S103. Input the support set and query set into the embedding unit for feature extraction to obtain the first feature of the support set and the second feature of the query set. The embedding unit is a feature extraction network with a multi-branch INCO structure.

[0084] S104. The target type is measured by the first feature and the second feature through a dual-metric network to obtain the target measurement result.

[0085] S105. When the target measurement result is greater than or equal to the preset threshold, the initial small sample PCB defect classification model is successfully trained, and the trained small sample PCB defect classification model is obtained.

[0086] S106. When the target measurement result is less than the preset threshold, the parameters in the initial small sample PCB defect classification model are adjusted, and the initial small sample PCB defect classification model is trained again until the initial small sample PCB defect classification model is successfully trained.

[0087] Optionally, the initial small sample PCB defect classification model is tested using small sample PCB defect images from several task datasets. The classification performance of the initial small sample PCB defect classification model is evaluated based on various detection metrics. When the classification performance of the initial small sample PCB defect classification model meets the classification requirements, a trained small sample PCB defect classification model is obtained. The classification of small sample PCB defects is achieved through the trained small sample PCB defect classification model.

[0088] Optionally, the classification performance of the small-sample PCB defect classification model can be evaluated using the detection results of various detection indicators. For example, accuracy can be used as the model evaluation indicator. For category i, the accuracy refers to the proportion of data where both the label and classification result are of category i, and the proportion of data where neither the label nor the classification result is of category i, to the total number of data points. Specifically, for the small-sample PCB image defect classification problem in this embodiment, it refers to the proportion of correctly classified defects in each verification batch to the total number of verification images, as shown in the following formula:

[0089]

[0090] In the formula, TP represents a positive example that is predicted as a positive example; FP represents a negative example that is predicted as a positive example; FN represents a positive example that is predicted as a negative example; and TN represents a positive example that is predicted as a negative example.

[0091] It should be noted that the preset threshold is used to describe the size of the target measurement result, and users can select the value according to actual business needs. This disclosure embodiment does not limit this.

[0092] This disclosure addresses the shortcomings of weak nonlinear feature learning ability of embedded units and the diverse morphologies and large intra-class feature differences of PCB product surface defects. It proposes a feature extraction network with a multi-branch INCO structure, which effectively broadens the depth and width of the embedded unit network. At the same time, it emphasizes the correlation between the same target features, helping the embedded unit to extract more advanced and semantically rich defect features of the PCB board.

[0093] Secondly, to address the issue of simple metric unit network structures and the tendency for single metric methods to introduce similarity bias, a dual-metric network is constructed, which allows defect features to be more closely clustered in the feature space. Furthermore, embedding an RFB receptive field block at the front end of the dual-metric network enables it to acquire more contextual information, improving feature discriminability and enhancing its defect classification performance. This method effectively solves the problem of low accuracy in small-sample PCB defect classification and demonstrates superior performance compared to other small-sample classification algorithms, possessing significant practical application value.

[0094] In summary, the embodiments disclosed herein can solve the problems of limited training samples and complex and diverse PCB defect morphologies in small-sample PCB defect classification and detection methods. They can efficiently and accurately detect surface defects in PCB products, thereby improving the detection accuracy and efficiency of small-sample PCB defect classification.

[0095] In some embodiments, the training dataset includes a public image dataset and a small sample PCB defect image dataset. The training method for a small sample PCB defect classification model provided in this disclosure further includes:

[0096] The initial small sample PCB defect classification model is pre-trained using a public image dataset as the first training sample, resulting in a pre-trained small sample PCB defect classification model. The pre-trained small sample PCB defect classification model classifies query images in the query set using support images in the support set.

[0097] The small sample PCB defect image dataset is classified and labeled according to different defect types;

[0098] By using the classified and labeled small sample PCB defect image dataset as the second training sample, the pre-trained small sample PCB defect classification model is fine-tuned to obtain the trained small sample PCB defect classification model.

[0099] Optionally, the public image dataset uses the public miniImageNet dataset, which can contain small sample PCB defect images of more than 100 categories, with each category containing more than 600 small sample PCB defect images. It should be noted that the categories and number of small sample PCB defect images in the public image dataset can be set according to actual business needs, and this embodiment does not limit this.

[0100] like Figure 2 As shown, Figure 2 This is a schematic diagram of small sample PCB defect types provided in the embodiments of this disclosure. The defect types classified and labeled in the small sample PCB defect image dataset include at least one of the following: missing holes, rodent bites, open circuits, short circuits, burrs, and excess copper.

[0101] This embodiment of the disclosure adopts a fine-tuning strategy in the meta-learning concept. First, the initial small sample PCB defect classification model is pre-trained using the public image dataset miniImageNet, so that the initial small sample PCB defect classification model can learn how to classify the query image of the query set using the support images of the support set. Then, the pre-trained small sample PCB defect classification model is fine-tuned using the collected small sample PCB defect image dataset, so that the pre-trained small sample PCB defect classification model can better adapt to the classification of small sample PCB defects.

[0102] In some embodiments, the query set is represented by the following formula:

[0103] Q={(I q ,L)};

[0104] In the formula, Q represents the query set; L represents the label corresponding to the image in the query set; I represents the query set image; and q represents the q-th image in the query set.

[0105]

[0106] In the formula, S represents the support set; This represents the target location labeling mask for the support set; N represents the number of categories. The support set contains C randomly selected image types, with K images for each type; the query set contains Q images.

[0107] In step S102, the training dataset is divided into several task datasets, each containing a support set and a query set. Specifically, using a meta-learning approach, each data input into the network is referred to as a task dataset T. i Each task dataset is divided into a query set Q and a support set S; the task dataset T iAlso known as the C-way K-shot task, where C represents the number of classes of supporting samples in the support set, K represents the number of supporting samples in each class, and the query set contains Q images.

[0108] like Figure 3 As shown, Figure 3 This diagram illustrates a multi-branch INCO structure provided in an embodiment of this disclosure. The embodiment employs an Inception initiation module and residual convolutional groups to construct the multi-branch INCO structure. The Inception initiation module is designed using a parallel approach, combining multiple convolutional and pooling operations to form a multi-branch structure. The first branch first undergoes a 1×1 convolution to reduce the feature layer thickness, thereby reducing the computational load of the network model. Secondly, two 3×3 convolutional operations are used to amplify the network's receptive field, preventing the reduction in parameters from affecting network performance, while simultaneously helping the network extract more abstract semantic information about PCB defects.

[0109] The second branch is directly connected to a 3×3 convolution operation to extract more surface information from PCB defects.

[0110] The third branch is a 3×3 Maxpooling operation, which can retain more texture information in the features. The combination of feature information obtained from the above three branches enables the entire embedding unit to extract more abstract and richer defect feature information.

[0111] The aforementioned residual convolutional group is a network module built based on the residual concept. The feature map extracted by the Inception initiation module is first input into a 1×1 convolutional layer, and then a 3×3 convolution is used for downsampling. Next, the feature information is input into an ECA layer (Efficient Channel Attention). The ECA layer enables embedding units to enhance the correlation between features of the same target, improving the accuracy of PCB defect classification. Then, the feature map is input again into a 1×1 convolutional layer of the residual convolutional group to reduce the number of channels, finally outputting the feature map.

[0112] like Figure 4 As shown, Figure 4 This is a schematic diagram of the improved embedded unit structure provided in the embodiments of this disclosure. The embodiments of this disclosure improve upon the original embedded unit in related technologies by adopting... Figure 3 The multi-branch INCO structure shown replaces the third convolutional block of the original embedding unit to form an improved new embedding unit. The improved new embedding unit is used to extract features from the query image of the input query set and the support image of the support set.

[0113] In some embodiments, step S104 above, which involves measuring the target type of the first feature and the second feature using a dual-metric network to obtain the target measurement result, includes:

[0114] The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational metric network in the dual metric network.

[0115] The second similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the cosine metric network in the dual metric network.

[0116] The target measurement result is obtained based on the average of the first and second similarities.

[0117] Both the relation metric network and the cosine metric network contain RFB receptive field blocks.

[0118] like Figure 5 As shown, Figure 5 This is a schematic diagram of the structure of the dual-metric network provided in this embodiment. The embodiment combines a relational metric network and a cosine metric network to form a dual-metric network. RFB receptive field blocks are embedded at the front end of the relational metric network and the cosine metric network, respectively. The RFB receptive field blocks can further improve the dual-metric network, enabling it to obtain more contextual information and improve its measurement capabilities. The improved dual-metric network is used to measure the target category of the first feature of the obtained support set and the second feature of the query set.

[0119] Figure 5 The relational metric network in the model mainly consists of two Conv Block convolutional layer modules and two fully connected (FC) layers. Each Conv Block convolutional layer module can be composed of 3×3 convolution, batch normalization, and ReLU activation function.

[0120] Figure 5 The cosine metric network in this algorithm mainly consists of two Conv Blocks and a cosine similarity layer. Specifically, the input features of the support image and the query image do not require feature concatenation. We take the mean of the same type of features in the support image as the feature prototype of that type. The feature prototypes of both the query image and the support image are input into the cosine metric network, where the feature prototypes are flattened to one dimension. The cosine distance between the support image and the query image is obtained through the cosine similarity layer, and finally, the second similarity, i.e., the similarity score 2, is calculated.

[0121] In some embodiments, a first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational metric network in the dual metric network, including:

[0122] Calculate the mean of the first feature of all support images belonging to the same category in the support set, and use the mean of the feature as the feature prototype of the corresponding category.

[0123] By concatenating the second feature of the query image in the query set with the feature prototypes of each category, several feature pairs are obtained.

[0124] The cascaded feature pairs are input into the relation measurement network to calculate the first similarity.

[0125] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of the relational measurement network provided in the embodiments of this disclosure, such as... Figure 6 As shown, the first feature is obtained by extracting features from the support images in the support set through the relation metric network, and the second feature is obtained by extracting features from the query images in the query set. The feature mean of the first feature of all support images belonging to the same category in the support set is calculated, and the feature mean is used as the feature prototype of the corresponding category. The second feature and the feature prototype of each category are concatenated to obtain several feature pairs. The metric score of several feature pairs, i.e. the first similarity score, is calculated through the relation metric network. The query image and support image are one-hot encoded based on the first similarity score.

[0126] In some embodiments, the first similarity is calculated using the following formula:

[0127]

[0128] In the formula, P1 (q,c) Let represent the first similarity score, which is the first similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the first similarity score. f φ Indicates an embedded unit; x q This represents a query sample in the query set; x s,c denoted as support samples in the support set; g1 represents the relation metric network; || represents the feature concatenation operation between the support image and the query image; K>1 in the above formula, therefore it is necessary to calculate the feature mean by using the first feature of all support images of the same category, and use the feature mean as the feature prototype of the corresponding category before performing the feature concatenation operation.

[0129] When K=1, the formula for calculating the first similarity is:

[0130] P1 (q，c) =g1([f φ (x 1，c )||fφ (x q c = 1, ... C

[0131] The first loss function of the relation metric network is calculated using the following formula:

[0132]

[0133] In the formula, Represents the first loss function; Indicates the query sample x q The first predicted value; y (q，j ) represents the query sample x q The first true value.

[0134] In some embodiments, the second similarity is calculated using the following formula:

[0135]

[0136] In the formula, P2 (q,c) The second similarity score represents the second similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained from this second similarity score. f φ (x s,c ) indicates that in the embedded unit f φ Supporting sample x s,c ;f φ (x q ) indicates that in the embedded unit f φ The query sample in the text; g2 represents the cosine metric network; y rc Represents the RFB receptive field block and convolution block operations; y cos This indicates the calculation of cosine similarity layers;

[0137] The second loss function of the cosine metric network is calculated using the following formula:

[0138]

[0139] In the formula, This represents the second loss function; Indicates the query sample x q The second predicted value; y (q，j) Indicates the query sample x q The second true value.

[0140] The improved dual-metric unit is used to measure the target category of the first feature of the acquired support set and the second feature of the query set. The improved dual-metric unit includes an RFB receptive field block. The loss function of the dual-metric unit is represented by the sum of the first loss function of the relation metric network and the second loss function of the cosine metric network, as shown in the following formula:

[0141]

[0142] The target measurement result is obtained by averaging the first and second similarities, as shown in the following formula:

[0143] like Figure 7 As shown, Figure 7 This is a schematic diagram of the RFB receptive field block provided in an embodiment of this disclosure. The front end of the relation metric network and the cosine metric network are respectively embedded in the RFB receptive field block, further improving the dual metric unit and helping it obtain more contextual information, thereby improving its metric capabilities. The RFB receptive field block is a lightweight module that can be integrated into various lightweight networks, ultimately achieving the effect of improving the performance of lightweight networks while ensuring detection efficiency.

[0144] The RFB receptive field block combines the ideas of Inception and dilated convolution to simulate human visual perception. Each branch in the RFB receptive field block consists of regular convolution and dilated convolution. Different kernels in regular convolution control different receptive field sizes, while the dilation rate in dilated convolution simulates the relationship between receptive field and eccentricity in the human visual system. Figure 7 The “rate” in the text refers to the dilation rate of the dilated convolution.

[0145] Embedding the RFB receptive field block into the dual metric unit enables the dual metric unit to acquire more contextual information and focus on richer feature information in the feature map. Embedding the RFB receptive field block into the dual metric unit can effectively improve the recognition accuracy of the relation metric network and cosine metric network for the target (small sample PCB defects).

[0146] like Figure 8 As shown, Figure 8This is a schematic diagram of the structure of the small-sample PCB defect classification model provided in this embodiment of the disclosure. The training method of the small-sample PCB defect classification model provided in this embodiment of the disclosure is an improvement on the relational metric network in metric learning. The small-sample PCB defect classification model mainly consists of two parts: an embedding unit and a dual metric unit. The support image of the support set and the query image of the query set are input into the embedding unit. The features of the support set and the query set are extracted by the multi-layer convolutional blocks and multi-branch INCO structure in the embedding unit and input into the next module unit (i.e., the dual metric unit).

[0147] In the dual-metric unit, the cosine metric network receives the image feature information from the support set and the query set and directly performs similarity calculation to obtain the second similarity score (i.e., similarity score 2). The relation metric network first performs a feature concatenation operation on the second feature of the query set and the first feature of the support set. Then, the relation metric network measures and calculates the PCB defect feature information to obtain the first similarity score (i.e., similarity score 1). Finally, the first similarity score and the second similarity score are averaged to obtain the final prediction and discrimination result.

[0148] like Figure 9 As shown, Figure 9 This is a comparative diagram of the feature distribution maps extracted by the original embedding unit and the improved new embedding unit provided in the embodiments of this disclosure. 30 samples of 5 categories were randomly selected from the training dataset and input into the original embedding unit and the improved new embedding unit (i.e., the embedding unit that includes the multi-branch INCO structure). The distribution maps of the output feature vectors of the two units were visualized. The visualization results show that the feature information extracted by the new embedding unit based on the multi-branch INCO structure has better discriminability and better feature aggregation effect of the same category. This indicates that the new embedding unit can extract more advanced and more representative feature information, which is beneficial to improving the final defect classification effect of the small sample PCB defect classification model.

[0149] This disclosure's small-sample PCB defect classification model employs the ICD-Net algorithm to improve classification accuracy. In PCB defect classification, due to the diverse morphologies of PCB defects and significant differences in some intra-class features, simple feature extraction networks perform poorly when the sample size is small. This disclosure, however, introduces a multi-branch INCO structure into the embedding unit, enabling the acquisition of richer and more discriminative PCB defect information. Furthermore, by using a dual-metric unit based on the RFB receptive field block to measure feature information, similarity bias is avoided, further improving the accuracy of small-sample PCB defect classification. The ICD-Net algorithm proposed in this invention exhibits high accuracy in defect classification performance and has significant practical value.

[0150] The beneficial effects of the embodiments disclosed herein include:

[0151] (1) A multi-branch INCO structure is proposed in the embedding unit. This structure consists of an Inception start module and a residual convolution group, which can effectively increase the depth and width of the network and extract more advanced and semantically rich defect features of PCB.

[0152] (2) To address the issue of simple network structure and the tendency for single measurement methods to introduce similarity bias, a dual measurement unit is constructed. This dual measurement unit consists of the original relational measurement network and the cosine measurement network from the relational network. Using a dual measurement unit for similarity measurement allows the first feature of the support set and the second feature of the query set to simultaneously satisfy both measurement methods. This results in defect features of the same category being more closely clustered in the feature space, which is beneficial for the classification of PCB defects in small-sample PCB defect classification models.

[0153] (3) In the dual-metric unit, RFB receptive field blocks are introduced at the front end of the relation metric network and the cosine metric network respectively. The RFB receptive field blocks enable the relation metric network and the cosine metric network to obtain more contextual information, learn deeper features, improve the discriminability of features, and further improve the defect classification effect of the small sample PCB defect classification model.

[0154] like Figure 10 As shown, Figure 10 This is a schematic flowchart of a method for classifying and detecting small-sample PCB defects according to an embodiment of the present disclosure. The method includes the following steps:

[0155] S201. Obtain the image of the PCB board to be inspected.

[0156] S202. Input the PCB board image to be detected into the small sample PCB defect classification model trained using the training method of the small sample PCB defect classification model described above.

[0157] S203. Obtain the classification and detection results of defects on the PCB board image to be detected from the output of the small sample PCB defect classification model.

[0158] like Figure 11 As shown, Figure 11 This is a schematic diagram of the structure of a training device for a small-sample PCB defect classification model provided in an embodiment of this disclosure. The training device for a small-sample PCB defect classification model provided in an embodiment of this disclosure includes:

[0159] Dataset acquisition unit 11 is configured to acquire the initial training dataset for the small sample PCB defect classification model;

[0160] Dataset partitioning unit 12 is configured to divide the training dataset into several task datasets, wherein each task dataset contains a support set and a query set;

[0161] The dataset input unit 13 is configured to input the support set and the query set into the embedding unit for feature extraction, thereby obtaining the first feature of the support set and the second feature of the query set. The embedding unit is a feature extraction network with a multi-branch INCO structure.

[0162] Dual metric unit 14 is configured to measure the target type of the first feature and the second feature through a dual metric network to obtain the target measurement result;

[0163] The first determination unit 15 is configured to successfully train the initial small sample PCB defect classification model when the target measurement result is greater than or equal to a preset threshold, thus obtaining the trained small sample PCB defect classification model.

[0164] The second determination unit 16 is configured to adjust the parameters in the initial small sample PCB defect classification model and continue training the initial small sample PCB defect classification model until the initial small sample PCB defect classification model is successfully trained when the target measurement result is less than a preset threshold.

[0165] like Figure 12 As shown, Figure 12 This is a schematic diagram of a small-sample PCB defect classification and detection device provided in an embodiment of the present disclosure. The small-sample PCB defect classification and detection device provided in an embodiment of the present disclosure includes:

[0166] Image acquisition unit 21 is configured to acquire an image of the PCB board to be inspected;

[0167] Image input unit 22 is configured to input the PCB board image to be detected into a small sample PCB defect classification model trained using the training method described above.

[0168] The classification result acquisition unit 23 is configured to acquire the classification detection results of defects on the PCB board image to be detected, output by the small sample PCB defect classification model.

[0169] like Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. It illustrates a structural schematic diagram suitable for implementing the electronic device in the embodiment of the present disclosure. Figure 13 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0170] like Figure 13 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0171] Typically, the following devices can be connected to the I / O interface: input devices, such as sensors or visual information acquisition devices; output devices, such as displays; storage devices, such as magnetic tapes or hard drives; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices (such as edge computing devices) to exchange data. Although Figure 13 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0172] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from ROM. When the computer program is executed by a processing device, it performs the training method for a small-sample PCB defect classification model according to embodiments of this disclosure, or performs all or part of the steps of the small-sample PCB defect classification and detection methods described in the foregoing embodiments of this disclosure.

[0173] For a detailed description of this embodiment, please refer to the corresponding descriptions in the foregoing embodiments, which will not be repeated here.

[0174] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0175] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0176] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0177] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A training method for a small-sample PCB defect classification model, characterized in that, include: Obtain the initial training dataset for the small sample PCB defect classification model; The training dataset is divided into several task datasets, wherein each task dataset contains a support set and a query set; The support set and query set are input into the embedding unit for feature extraction to obtain the first feature of the support set and the second feature of the query set. The embedding unit is a feature extraction network with a multi-branch INCO structure. The first feature and the second feature are measured by a dual-metric network to obtain the target measurement result. When the target measurement result is greater than or equal to the preset threshold, the initial small sample PCB defect classification model is successfully trained, and a trained small sample PCB defect classification model is obtained. When the target metric result is less than the preset threshold, the parameters in the initial small sample PCB defect classification model are adjusted, and the initial small sample PCB defect classification model is trained again until the initial small sample PCB defect classification model is successfully trained. The first and second features are measured using a dual-metric network to obtain target measurement results, including: The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual measurement network. The second similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the cosine metric network in the dual metric network. The target measurement result is obtained based on the average of the first similarity and the second similarity. Both the relation metric network and the cosine metric network contain RFB receptive field blocks; The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual measurement network, including: Calculate the mean value of the first feature of all support images belonging to the same category in the support set, and use the mean value of the feature as the feature prototype of the corresponding category; By concatenating the second feature of the query image in the query set with the feature prototypes of each category, several feature pairs are obtained. The concatenated feature pairs are input into the relation measurement network to calculate the first similarity. The formula for calculating the first similarity is as follows: ； In the formula, Let represent the first similarity, that is, the first similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the first similarity score. ; Indicates an embedded unit; This represents a query sample in the query set; This represents the supporting samples in the support set; This represents the relational metric network; This indicates a feature concatenation operation between the supporting image and the query image; The first loss function of the relation metric network is calculated using the following formula: ； In the formula, Represents the first loss function; Indicates the query sample The first predicted value; Indicates the query sample The first true value; The formula for calculating the second similarity is as follows: ； In the formula, The second similarity score represents the second similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the second similarity score. ; Indicates in the embedded unit Supporting samples ; Indicates in the embedded unit Query samples in the text; This represents the RFB receptive field block and convolution block operations; This indicates the calculation of cosine similarity layers; The second loss function of the cosine metric network is calculated using the following formula: ； In the formula, This represents the second loss function; Indicates the query sample The second predicted value; Indicates the query sample The second true value.

2. The training method for the small-sample PCB defect classification model according to claim 1, characterized in that, The training dataset includes a public image dataset and a small sample PCB defect image dataset, and the method further includes: The initial small sample PCB defect classification model is pre-trained using the public image dataset as the first training sample to obtain a pre-trained small sample PCB defect classification model; wherein, the pre-trained small sample PCB defect classification model classifies the query images in the query set using the support images in the support set; The small sample PCB defect image dataset is classified and labeled according to different defect types; The pre-trained small sample PCB defect classification model is fine-tuned using the classified and labeled small sample PCB defect image dataset as the second training sample to obtain the trained small sample PCB defect classification model.

3. A method for classifying and detecting defects in small-sample PCBs, characterized in that, include: Acquire images of the PCB board to be inspected; The PCB board image to be detected is input into a small sample PCB defect classification model trained using the training method of the small sample PCB defect classification model according to any one of claims 1 to 2; Obtain the classification and detection results of defects on the PCB board image to be detected, output by the small sample PCB defect classification model.

4. A training device for a small-sample PCB defect classification model, characterized in that, include: The dataset acquisition unit is configured to acquire the initial training dataset for the small sample PCB defect classification model; A dataset partitioning unit is configured to divide the training dataset into several task datasets, wherein each task dataset contains a support set and a query set; The dataset input unit is configured to input the support set and query set into the embedding unit for feature extraction to obtain a first feature of the support set and a second feature of the query set, wherein the embedding unit is a feature extraction network with a multi-branch INCO structure; The dual-metric unit is configured to measure the target type of the first feature and the second feature through a dual-metric network to obtain the target measurement result. The first determination unit is configured such that when the target measurement result is greater than or equal to a preset threshold, the initial small sample PCB defect classification model is successfully trained, and a trained small sample PCB defect classification model is obtained. The second determination unit is configured to adjust the parameters in the initial small sample PCB defect classification model and continue to train the initial small sample PCB defect classification model until the initial small sample PCB defect classification model is successfully trained when the target measurement result is less than a preset threshold. The first and second features are measured using a dual-metric network to obtain target measurement results, including: The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual measurement network. The second similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the cosine metric network in the dual metric network. The target measurement result is obtained based on the average of the first similarity and the second similarity. Both the relation metric network and the cosine metric network contain RFB receptive field blocks; The first similarity is obtained by measuring the target type of the first feature of the support set and the second feature of the query set through the relational measurement network in the dual measurement network, including: Calculate the mean value of the first feature of all support images belonging to the same category in the support set, and use the mean value of the feature as the feature prototype of the corresponding category; By concatenating the second feature of the query image in the query set with the feature prototypes of each category, several feature pairs are obtained. The concatenated feature pairs are input into the relation measurement network to calculate the first similarity. The formula for calculating the first similarity is as follows: ； In the formula, Let represent the first similarity, that is, the first similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the first similarity score. ; Indicates an embedded unit; This represents a query sample in the query set; This represents the supporting samples in the support set; This represents the relational metric network; This indicates a feature concatenation operation between the supporting image and the query image; The first loss function of the relation metric network is calculated using the following formula: ； In the formula, Represents the first loss function; Indicates the query sample The first predicted value; Indicates the query sample The first true value; The formula for calculating the second similarity is as follows: ； In the formula, The second similarity score represents the second similarity score of the q-th sample in a few-sample task. The one-hot vector of the category of the q-th sample is obtained through the second similarity score. ; Indicates in the embedded unit Supporting samples ; Indicates in the embedded unit Query samples in the text; This represents the RFB receptive field block and convolution block operations; This indicates the calculation of cosine similarity layers; The second loss function of the cosine metric network is calculated using the following formula: ； In the formula, This represents the second loss function; Indicates the query sample The second predicted value; Indicates the query sample The second true value.

5. A classification and detection device for small-sample PCB defects, characterized in that, include: The image acquisition unit is configured to acquire an image of the PCB board to be inspected; The image input unit is configured to input the PCB board image to be detected into a small sample PCB defect classification model trained using the training method of the small sample PCB defect classification model according to any one of claims 1 to 2; The classification result acquisition unit is configured to acquire the classification detection result of the defects on the PCB board image to be detected, output by the small sample PCB defect classification model.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the training method of the small sample PCB defect classification model according to any one of claims 1 to 2, or to perform the classification and detection method of the small sample PCB defects according to claim 3.

Images