Printed circuit board defect detection method based on small sample learning, medium and equipment

Abstract

The application discloses a printed circuit board defect detection method, system and device based on small sample learning, which comprises the following steps: firstly, a printed circuit board defect detection network is trained; for an input PCB image, a defect feature is extracted by using an instance-level embedding backbone network to obtain global features; an output of the last block is refined by using a position weight distribution module to generate a defect position weight map; a corresponding region is cut from the input PCB image according to the weight map and then input into the instance-level embedding backbone to generate local features, and then an embedding is generated for each input image; the sample weight distribution module is applied to an embedding set of each defect support class to form a notice prototype of each defect class, and a network is trained by using a cross-entropy loss function. The application is a defect class perception auxiliary and inhibition network scheme learned from a small number of PCB defect class samples, which can pull the distribution of abnormal defect values to a defect class cluster and reduce the negative influence on defect class description.

Description

Technical Field

[0001] This invention relates to the field of printed circuit board (PCB) defect detection technology in the field of intelligent manufacturing, specifically to a PCB defect detection method, medium, and equipment based on few-sample learning. Background Technology

[0002] PCBs, as key components in electronic products, are widely used in many fields such as autonomous driving, computer manufacturing, and home appliances, and have a large market. However, due to limitations in production technology, some defects may occur during actual production. Furthermore, defect detection of PCBs often incurs additional costs.

[0003] The emergence of deep learning has enabled the rapid development of object detection and has also helped the PCB defect detection industry by reducing costs. Generally, deep learning performs well in defect detection when sufficient PCB defect data is available. However, annotating enough PCB defect data is expensive, time-consuming, and labor-intensive. In actual production processes, the amount of defect data obtained may be limited, and the number of defect samples for each defect type is also insufficient. Insufficient defect samples pose significant challenges to the detection task. This invention focuses on the ability to learn PCB defect types (few-shot learning) with limited defect data support. Summary of the Invention

[0004] The present invention proposes a method, system and device for detecting defects in printed circuit boards based on few-sample learning, which can at least solve one of the technical problems in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for detecting defects in printed circuit boards based on few-shot learning involves performing the following steps using a computer device.

[0007] The process includes the following steps: first, training a printed circuit board (PCB) defect detection network; then, using the trained PCB defect detection network to inspect the PCB to be inspected. The training steps for the PCB defect detection network are as follows.

[0008] Step 1: For the input PCB image, use an instance-level embedding backbone network to extract defect features and obtain global features;

[0009] Step 2: Refine the output of the last block using the position weight allocation module to generate a defect position weight map;

[0010] Step 3: Based on the weight map, cut out the corresponding region from the input PCB image and input it into the instance-level embedding backbone to generate local features, and then generate embeddings for each input image;

[0011] Step 4: Apply the sample weight allocation module to the embedding set of each defect support class to form an attention prototype for each defect category, and train the network using the cross-entropy loss function.

[0012] Further, step S1 specifically includes,

[0013] S11: Input the PCB image containing defects to be predicted;

[0014] S12: Input the PCB image into the instance-level embedded backbone network to extract defect features;

[0015] S13: Perform global average pooling on the feature maps output by the instance-level embedded backbone network to obtain global defect features.

[0016] Further, step S2 specifically includes,

[0017] S21: For a defect sample from a support set containing defect type labels, let L ij This represents the global feature map of the j-th image in the i-th defect support class output by the backbone network, for L ij Global average pooling is used to obtain the global defect features S at the sample level. ij ;

[0018] S22: Average of all sample-level defect features in the i-th defect support class. ij The class-level global defect features that are less affected by abnormal PCB image values ​​are calculated.

[0019] S23: Calculate the positive attention O of the intra-class similarity of defects at each location according to the following formula. ij and the total suppression intensity Z at each location ij This allows us to obtain the weighted map W of the PCB image support set. ij W ij In the suppression intensity Z ij Under the influence of this, abnormal defect features that may interfere with the identification of a specific defect class in other defect classes are included in the scope of consideration of the features of the specific defect class, thereby improving the distinguishability of the specific defect class.

[0020]

[0021]

[0022] W ij =O ij +Z ij

[0023] In the formula, K represents the number of samples of the defect class, and N represents the number of defect categories;

[0024] S24: For the q-th image in the query set that does not contain defect type labels, calculate the defect feature map L. q Its sample-level global defect features S q The dot product between them yields the corresponding weighted graph. The W obtained above ij and W q Softmax is applied to classify all locations.

[0025] Furthermore, step S3 specifically includes,

[0026] S31: Change the global defect feature map from (1+W) ij ) and (1+W q The defect attention map is weighted and a refined defect attention map is generated. Both the defect attention map and the input PCB image are upsampled to a predefined size, and the position with the largest value in the defect attention map is selected as the center of the selected area.

[0027] S32: Based on the selected region, a fixed-size patch is cropped from the input PCB image, and then fed back into the backbone network. Its output is then subjected to global average pooling and used as a local defect feature.

[0028] S33: Connect the global and local defect features obtained above to form an instance-level defect embedding; the defect embedding of the j-th PCB image for the i-th defect support class is denoted as m. ij The embedding of the q-th PCB image in the defect query set is denoted as m. q .

[0029] Further, step S4 specifically includes,

[0030] S41: Obtain the set of K defect embeddings of the i-th type of defect as M. i =(m i1 m i2 , ..., m ik ), take M i The mean (M) i );

[0031] S42: Calculate the defect positive attention and defect negative attention according to the following formula. According to o i and z i This module will identify defect features that have a high degree of similarity to descriptions of the same defect class but a low degree of similarity to descriptions of other defect classes, further enhancing the ability to identify abnormal defect features.

[0032]

[0033]

[0034] S43: A defect attention prototype t is obtained by combining all defect embeddings for each supporting defect category using a weighted sum. i And add the average values ​​of K supporting defect features as residual connections;

[0035] t i =mean(M i )+exp(α)·softmax(o i +z i )·M i

[0036] In the formula, α is a parameter learned during training. For classification of single defect samples of N defect classes, sigmoid is used instead of softmax to prevent the weights from being simplified to 1.

[0037] S44: Use Euclidean distance to calculate the distance between the defect prototype and the defect query embedding. Calculate the probability that the defect query prototype pairs belong to the same defect class support set according to the following formula. Then, use the cross-entropy loss function to train the model. During the testing phase, take the defect type with the highest probability as the final output.

[0038]

[0039] In the formula, t i It is the attention prototype of the i-th type of defect, m g It queries the defect embedding of the defect sample, D(t). i m q ) is t i and m q The Euclidean distance between them.

[0040] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0041] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0042] As can be seen from the above technical solutions, the printed circuit board defect detection method and device based on few-sample learning of the present invention, namely a defect class perception auxiliary and suppression network scheme that learns from a small number of PCB defect class samples, pulls the distribution of abnormal defect values ​​toward defect class clusters, reducing the negative impact on defect class description. A location weight allocation module is introduced to search for information-rich local regions in the input defect image, enhancing the correction of representative defect embeddings. The sample weight allocation module selects distinguishable embeddings throughout the defect class, promoting the generation of preferred defect prototypes.

[0043] Specifically, the advantages of the small-sample learning method for PCB defect categories based on a limited number of samples in this invention are as follows: It utilizes a location weight allocation module to search for local regions rich in defect information within the PCB image, enhancing the correction of representative defect embedding; and it uses a sample weight allocation module to reduce the influence of defect outliers and improve the contrast of prototypes for all defect categories. This method can learn defect categories even with a limited number of defect samples, and can correctly identify defect types in PCB images with strong interference and high noise levels, achieving high detection accuracy. Attached Figure Description

[0044] Figure 1 This is a flowchart of the present invention;

[0045] Figure 2 This is a schematic diagram of the position weight allocation module according to an embodiment of the present invention;

[0046] Figure 3 This is a schematic diagram of the sample weight allocation module according to an embodiment of the present invention;

[0047] Figure 4 This is a schematic diagram of the experimental results of an embodiment of the present invention;

[0048] Figure 5 This is an example diagram of an open circuit defect in an embodiment of the present invention;

[0049] Figure 6 This is a diagram showing the effect of detecting defects in a newly added small sample in an embodiment of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0051] like Figure 1 As shown in the figure, the printed circuit board defect detection method based on few-sample learning described in this embodiment performs the following steps using a computer device:

[0052] Step 1: For the input PCB image, use instance-level embedding backbone network to extract defect features and obtain global features.

[0053] Step 2: Refine the output of the last block through the position weight allocation module to generate a defect position weight map.

[0054] Step 3: Based on the weight map, cut out the corresponding region from the input PCB image and input it into the instance-level embedding backbone to generate local features, and then generate embeddings for each input image.

[0055] Step 4: Apply the sample weighting module to the embedding set of each defect support class to form an attention prototype for each defect category. Train the network using the cross-entropy loss function.

[0056] Specific implementation steps:

[0057] Regarding step S1 above: For the input PCB image, instance-level embedding backbone network is used to extract defect features and obtain global features. Specifically, this includes the following sub-steps S11 to S13:

[0058] S11: Input the PCB image containing defects to be predicted.

[0059] S12: Input the PCB image into the instance-level embedded backbone network for defect feature extraction.

[0060] S13: Perform global average pooling on the feature maps output by the instance-level embedded backbone network to obtain global defect features.

[0061] Regarding step S2 above: the output of the last block is refined through the position weight allocation module to generate a defect position weight map. Specifically, this includes the following sub-steps S21 to S24:

[0062] S21: As Figure 2 For defect samples from a support set containing defect type labels, let L ij This represents the global feature map of the j-th image in the i-th defect support class output by the backbone network. For L... ij Global average pooling is used to obtain the global defect features S at the sample level. ij .

[0063] S22: Average of all sample-level defect features in the i-th defect support class. ij The class-level global defect features that are less affected by abnormal PCB image values ​​are calculated.

[0064] S23: Calculate the positive attention O of the intra-class similarity of defects at each location according to the following formula. ij and the total suppression intensity Z at each location ijThis allows us to obtain the weighted map W of the PCB image support set. ij W ij In the suppression intensity Z ij Under the influence of this mechanism, abnormal defect features that may interfere with the identification of a specific defect class from other defect classes are included in the scope of consideration for features of the specific defect class, thereby improving the distinguishability of the specific defect class.

[0065]

[0066]

[0067] W ij =O ij +Z ij

[0068] In the formula, K represents the number of samples of the defect class, and N represents the number of defect categories.

[0069] S24: For the q-th image in the query set that does not contain defect type labels, calculate the defect feature map L. q Its sample-level global defect features S q The dot product between them yields the corresponding weighted graph. The W obtained above ij and W q Softmax is applied to classify all locations.

[0070] Regarding step S3 above: The corresponding region is cut out from the input PCB image based on the weight map and then input into the instance-level embedding backbone to generate local features, thereby generating an embedding for each input image. Specifically, this includes the following sub-steps S31 to S33:

[0071] S31: Change the global defect feature map from (1+W) ij ) and (1+W q The defect attention map is weighted and a refined defect attention map is generated. Both the defect attention map and the input PCB image are upsampled to a predefined size, and the location with the largest value in the defect attention map is selected as the center of the selected region.

[0072] S32: A fixed-size patch is cropped from the input PCB image based on the selected region, and then fed back into the backbone network. Its output is then subjected to global average pooling as a local defect feature.

[0073] S33: Connect the global and local defect features obtained above to form an instance-level defect embedding. The defect embedding for the j-th PCB image of the i-th defect support class is denoted as m. ij The embedding of the q-th PCB image in the defect query set is denoted as m. q .

[0074] Regarding step S4 above: the sample weight allocation module is applied to the embedding set of each defect support class to form an attention prototype for each defect category. The network is trained using the cross-entropy loss function. Specifically, this includes the following sub-steps S41 to S44:

[0075] S41: As Figure 3 The set of K defect embeddings of the i-th type of defect is M. i =(m i1 m i2 , ..., m ik ), take M i The mean (M) i ).

[0076] S42: Calculate the defect positive attention and defect negative attention according to the following formula. According to o i and z i This module will identify defect features that have a high degree of similarity to descriptions of the same defect class but a low degree of similarity to descriptions of other defect classes, further enhancing the ability to identify abnormal defect features.

[0077]

[0078]

[0079] S43: A defect attention prototype t is obtained by combining all defect embeddings for each supporting defect category using a weighted sum. i And add the average of K supporting defect features as residual connections.

[0080] t i =mean(M i )+exp(α)·softmax(o i +z i )·M i

[0081] In the formula, α is a parameter learned during training. For classification of single defect samples of N defect classes, sigmoid is used instead of softmax to prevent the weights from being simplified to 1.

[0082] S44: Utilize Euclidean distance to calculate the distance between the defect prototype and the defect query embedding. The probability that a defect query prototype pair belongs to the same defect class support set is calculated using the following formula. Then, the model is trained using the cross-entropy loss function. During the testing phase, the defect type with the highest probability is taken as the final output.

[0083]

[0084] In the formula, t iIt is the attention prototype of the i-th type of defect, m q This queries the defect embedding of defect samples. D(t) i m q ) is t i and m q The Euclidean distance between them.

[0085] In summary, the advantages of the few-sample learning method for PCB defect categories according to embodiments of the present invention are as follows: The location weight allocation module searches for local regions rich in defect information in the PCB image, and the generated defect weight map incorporates similar feature values ​​of other defect categories, enhancing the correction of representative defect features; the sample weight allocation module enhances the recognition ability of specific defect categories through attention, reduces the influence of defect outliers, and improves the contrast between features of all defect categories. This method can effectively learn defect categories when obtaining a small number of defect samples, and can correctly identify defect types and distinguish and identify similar defect categories in PCB images with strong interference information and high noise, achieving high detection accuracy.

[0086] This invention first conducted experiments on the MiniImageNet and TieredImageNet datasets, performing 5-class 5-sample and 5-class single-sample experiments. ResNet-12 was used as the backbone network, and the Adam optimizer was applied during training with an initial learning rate of 0.002. Comparisons were then made with state-of-the-art methods (see...). Figure 4 The method achieved high accuracy on both datasets, proving its effectiveness.

[0087] This method was also used for PCB defect detection testing. For open-circuit defects, this type of defect included 15 samples, examples of which are shown below. Figure 5 As shown.

[0088] The effectiveness of this method was tested with a small sample size. The results for detecting open-circuit defects are shown in the table below. Figure 6 This demonstrates the effectiveness of the method in the field of small sample defect detection for intelligent manufacturing PCBs.

[0089] In another aspect, the present invention also discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the method described above.

[0090] In another aspect, the present invention also discloses a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the method described above.

[0091] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the printed circuit board defect detection methods based on few-shot learning in the above embodiments.

[0092] It is understood that the system provided in the embodiments of the present invention corresponds to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant content can be referred to the corresponding parts of the above method.

[0093] This application also provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other via the communication bus.

[0094] Memory, used to store computer programs;

[0095] When the processor executes the program stored in the memory, it implements the above-mentioned printed circuit board defect detection method based on few-shot learning.

[0096] The communication bus mentioned in the aforementioned electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0097] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0098] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0099] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0100] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0101] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0102] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0103] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in printed circuit boards based on few-sample learning, characterized in that, The process includes the following steps: first, training a printed circuit board (PCB) defect detection network; then, using the trained PCB defect detection network to inspect the PCB to be inspected. The training steps for the PCB defect detection network are as follows. Step 1: For the input PCB image, use an instance-level embedding backbone network to extract defect features and obtain global features; Step 2: Refine the output of the last block using the position weight allocation module to generate a defect position weight map, specifically including: S21: For defect samples from a support set containing defect type labels, let This represents the global feature map of the j-th image in the i-th defect support class output by the backbone network. Global average pooling is used to obtain global defect features at the sample level. ; S22: Average of all sample-level defect features in the i-th defect support class The class-level global defect features that are less affected by abnormal PCB image values ​​are calculated. S23: Calculate the positive attention score for intra-class similarity of defects at each location using the following formula. and the total inhibition intensity at each location This allows us to obtain a weighted map of the PCB image that supports the set. ; ； ； ； In the formula This represents the number of samples in the defect class. Indicates the number of defect categories; S24: For the q-th image in the query set that does not contain defect type labels, calculate the defect feature map. Its sample-level global defect features The dot product between them yields the corresponding weighted graph. Regarding the above-mentioned acquisition and Softmax is applied to classify all locations; Step 3: Based on the weight map, cut out the corresponding region from the input PCB image and input it into the instance-level embedding backbone to generate local features, and then generate embeddings for each input image; Step 4: Apply the sample weight allocation module to the embedding set of each defect support class to form an attention prototype for each defect category, and train the network using the cross-entropy loss function.

2. The printed circuit board defect detection method based on few-sample learning according to claim 1, characterized in that: Step S1 specifically includes, S11: Input the PCB image containing defects to be predicted; S12: Input the PCB image into the instance-level embedded backbone network to extract defect features; S13: Perform global average pooling on the feature maps output by the instance-level embedded backbone network to obtain global defect features.

3. The printed circuit board defect detection method based on few-sample learning according to claim 1, characterized in that: Step S3 specifically includes: S31: The global defect feature map is changed by ( )and( The defect attention map is weighted and a refined defect attention map is generated. Both the defect attention map and the input PCB image are upsampled to a predefined size, and the position with the largest value in the defect attention map is selected as the center of the selected area. S32: Based on the selected area, a fixed-size patch is cropped from the input PCB image, and then fed back into the backbone network. Its output is then subjected to global average pooling and used as a local feature of the defect. S33: Connect the global and local defect features obtained above to form an instance-level defect embedding; the defect embedding of the j-th PCB image for the i-th defect support class is denoted as... The embedding of the q-th PCB image in the defect query set is denoted as .

4. The printed circuit board defect detection method based on few-sample learning according to claim 3, characterized in that: Step S4 specifically includes: S41: Obtain the set of K defect embeddings for the i-th type of defect as follows: ,Pick average ; S42: Calculate the defect positive attention and defect negative attention according to the following formula; ； ； S43: Use a weighted sum to combine all defect embeddings for each supporting defect category to obtain a defect attention prototype. And add the average values ​​of K supporting defect features as residual connections; ； In the formula This is a parameter learned during training. For classifying single defect samples of N defect classes, it is used... replace To prevent the weight from being simplified to 1; S44: Use Euclidean distance to calculate the distance between the defect prototype and the defect query embedding. Calculate the probability that the defect query prototype pairs belong to the same defect class support set according to the following formula. Then, use the cross-entropy loss function to train the model. During the testing phase, take the defect type with the highest probability as the final output. ； In the formula, It is the attention prototype of the i-th type of defect. It queries the defect embedding of defect samples. yes and The Euclidean distance between them.

5. A computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

6. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method as claimed in any one of claims 1 to 4.

Images