Method, electronic device, and system for detecting solder paste defects

Abstract

A method, an electronic device, and a system for detecting solder paste defects are disclosed. The method includes: receiving a first image and a label from an automated X-ray inspection machine, wherein the label indicates a bounding box of a solder paste; enlarging the bounding box to update the bounding box; generating a detection result of a defect of the solder paste according to the first image and the bounding box by a machine learning model; and outputting the detection result.

Description

Methods, electronic devices and systems for detecting solder paste defects This invention relates to an image recognition technology, and more particularly to a method, electronic device, and system for detecting solder paste defects. Printed circuit boards contain various electronic components soldered using surface mount technology (SMT), such as capacitors, resistors, transistors, chips, connectors, or dual in-line package (DIP) components. The solder paste used for soldering may contain defects such as foreign matter or incomplete filling. Since these defects are difficult to detect with the naked eye, automated X-ray inspection (AXI) machines are used to identify them. However, in practice, AXI machines often misjudge good solder paste as defective, leading to an excessively high overkill rate. This results in unnecessary manpower being spent on re-inspecting defective solder paste. This invention provides a method, electronic device, and system for detecting solder paste defects, which can reduce the over-detection rate of solder paste defects. This invention discloses a method for detecting solder paste defects, comprising: receiving a first image of solder paste and a label from an automated X-ray inspection machine, wherein the label indicates a bounding box of the solder paste; magnifying the bounding box to update the bounding box; using a machine learning model to generate a detection result of solder paste defects based on the first image and the bounding box; and outputting the detection result. The present invention further discloses an electronic device for detecting solder paste defects, comprising a communication interface, a processor, and memory. The communication interface is configured to receive at least one image and a label of solder paste. The processor is electrically connected to the communication interface. The memory is electrically connected to the processor and configured to store program code, wherein the program code instructs the processor to perform the aforementioned method for detecting solder paste defects based on at least one image and label. This invention further discloses a system for detecting solder paste defects, comprising an automatic X-ray inspection machine, an electronic device as described above, an electronic inspection station, and a manual inspection station. The automatic X-ray inspection machine is configured to capture at least one image of the solder paste on a circuit board and generate a label with at least one image. The electronic device is connected to the automatic X-ray inspection machine and configured to generate a detection result of solder paste defects based on the at least one image and the label. The electronic inspection station is connected to the electronic device and configured to receive the circuit board when the detection result indicates a pass. The manual inspection station is connected to the electronic device and configured to receive the circuit board when the detection result indicates a fail. The method, electronic device, and system for detecting solder paste defects of the present invention have the following features: (1) AXI machine and electronic device are used to perform two defect detections. If the detection result of the second defect detection is still unqualified, the personnel at the manual inspection station will perform a third detection to avoid unnecessary manual inspection; (2) Multiple machine learning models are used to perform defect detection on multiple solder pastes in multiple solder paste groups to solve the problem of high overkill rate of a single machine learning model; (3) The bounding box of the input image is magnified and preprocessed to solve the problem of offset or inaccurate bounding boxes generated by the existing AXI machine; (4) For thicker solder paste, such as ball grid arrays and electroplated vias, multiple images corresponding to multiple cross-sections are spliced ​​for image recognition to fully detect solder paste defects; and (5) For thicker solder paste, the weights of the neurons in the two twin networks used are exactly the same, which can save the memory space occupied by the neural network. Figure 1 is a schematic diagram of a system 1 for detecting solder paste defects according to an embodiment of the present invention. System 1 may include an automated X-ray inspection (AXI) machine 14, an electronic device 10, a next electronic inspection station 15, a manual inspection station 16, and a repair station 17. The AXI machine 14 is configured to capture at least one image of the solder paste on a circuit board and generate a label for at least one image. The electronic device 10 is connected to the AXI machine 14 and configured to generate a detection result of solder paste defects based on the at least one image and the label. The next electronic inspection station 15 is connected to the electronic device 10 and configured to receive the circuit board when the detection result indicates a pass. The manual inspection station 16 is connected to the electronic device 10 and the next electronic inspection station 15 and configured to receive the circuit board when the detection result indicates a fail, for manual inspection by personnel. The repair station 17 is connected to the manual inspection station 16 and configured to receive the printed circuit board from the manual inspection station 16 for repair by personnel. In one embodiment, system 1 may include a conveyor belt (not shown) configured to transport circuit boards to designated machines and testing stations. In practical applications, the AXI machine 14 can photograph, image-recognize, and perform initial defect detection on the solder paste on the printed circuit board. If the solder paste is free of defects, the AXI machine 14 records it as qualified on a label, and the conveyor belt automatically sends the printed circuit board to the next electronic inspection station 15. If the solder paste is defective, the AXI machine 14 records it as unqualified on a label and the delimited box corresponding to the defect, and then transmits the image of the solder paste and the corresponding label to the electronic device 10. It should be understood that because the over-detection rate of the existing AXI machine 14 is too high, this invention uses the electronic device 10 to perform a second defect detection. If the detection result of the second defect detection is still unqualified, a third inspection will be performed by personnel at the manual inspection station 16 to avoid unnecessary manual inspection. Furthermore, the applicant noted that because solder paste comes in various shapes and the criteria for judging defects are inconsistent, a single machine learning model struggles to adapt to the diverse judgment criteria, resulting in an excessively high over-detection rate. Therefore, the electronic device 10 incorporates multiple machine learning models to perform defect detection on multiple solder pastes from multiple solder paste groups, thereby addressing the problem of excessively high over-detection rates with a single machine learning model. In detail, the AXI machine 14 can determine the solder paste category name and the corresponding solder paste group, and record the above information in a label. If the label indicates that the solder paste belongs to the first solder paste group, the electronic device 10 uses the first machine learning model 11 to perform image recognition; if the label indicates that the solder paste belongs to the second solder paste group, the electronic device 10 uses the second machine learning model 12 to perform image recognition; and if the label indicates that the solder paste belongs to the third solder paste group, the electronic device 10 uses the third machine learning model 13 to perform image recognition. In one embodiment, the label may indicate the solder paste category name, solder paste group, the two diagonal coordinates of the bounding box, timestamp, and other relevant information for personnel to read production line data. In one embodiment, a solder paste group corresponds to at least one solder paste category name. Specifically, Figure 2A is a schematic diagram of table ta1 showing qualified samples (i.e., solder paste without defects) and unqualified samples (i.e., solder paste with defects) of a first solder paste group according to an embodiment of the present invention. The solder paste belonging to the first solder paste group has a solder paste category name including, but not limited to, "universal". Figure 2B is a schematic diagram of table ta2 showing qualified and unqualified samples of the second solder paste group according to an embodiment of the present invention. The solder paste belonging to the second solder paste group includes, but is not limited to, the following solder paste category names: quad flat no-lead (QFN), small outline transistor (SOT), gullwing, ground pad, resistor, capacitor, press-fit, and chip. Figure 2C is a schematic diagram of a table ta3 showing qualified and unqualified samples of the third solder paste group according to an embodiment of the present invention. Solder paste belonging to the third solder paste group includes, but is not limited to, ball grid array (BGA) and plated through hole (PTH) solder pastes. Figure 3 is a flowchart of a method 3 for detecting solder paste defects according to an embodiment of the present invention, wherein method 3 may be implemented by an electronic device 10. In step S301, the electronic device 10 can receive at least one image of solder paste and a tag from the AXI machine 14, and then read the information in the tag. In step S302, if the label indicates that the solder paste belongs to the first solder paste group, the electronic device 10 executes steps S303…S305; if the label indicates that the solder paste belongs to the second solder paste group, the electronic device 10 executes steps S306…S307; and if the label indicates that the solder paste belongs to the third solder paste group, the electronic device 10 executes steps S308…S310. Finally, in step S311, for at least one image and label of a solder paste, the electronic device 10 can output the corresponding detection result. Figure 4 is a schematic diagram of image recognition of the first solder paste group according to an embodiment of the present invention. Referring to Figures 3 and 4, when the electronic device 10 receives the solder paste image IMG and tag from the AXI machine 14, it can read the two diagonal coordinates P1 and P2 of the bounding box B, that is, the range of the bounding box B can be defined by the upper left coordinate P1 and the lower right coordinate P2. In step S303, the electronic device 10 performs preprocessing on the solder paste image. Specifically, the preprocessing unit 40 of the electronic device 10 can enlarge the bounding box B to update it to bounding box B'. The preprocessing unit 40 can adjust the values ​​of at least one diagonal coordinate P1, P2 to generate updated diagonal coordinates P1', P2'. Therefore, the preprocessing unit 40 can define the updated bounding box B' based on the updated diagonal coordinates P1', P2'. It should be understood that when the AXI machine 14 performs defect detection on the solder paste, the bounding box generated by the AXI machine 14 may be offset or inaccurate. This may affect the accuracy of image recognition of the bounding box. To solve the above problem, the electronic device 10 of the present invention can perform preprocessing including enlarging the bounding box generated by the AXI. In step S304, the binarization unit 41 of the electronic device 10 can perform binarization processing on multiple pixels of the image IMG within the bounding box B' to generate a binarized image IMGb. For example, the binarization unit 41 can perform statistical calculations on the grayscale values ​​of multiple pixels according to Otsu's method to select a threshold for distinguishing the foreground and background, and then perform binarization processing according to the threshold to generate the binarized image IMGb. In the binarized image IMGb, pixels belonging to the foreground are set to black (the grayscale value of black is 0), and pixels belonging to the background are set to white (the grayscale value of white is 255). Further, the binarization unit 41 can set black pixels to binary 1 and white pixels to binary 0 to generate a binarized image IMGb represented in binary bits. In step S305, since the tag indicates that the solder paste belongs to the first solder paste group, the electronic device 10 uses the first machine learning model 11 to perform image recognition on the binarized image IMGb to generate a defect detection result. The first machine learning model 11 may include, but is not limited to, the MobileNetV3 model. In one embodiment, the electronic device 10 may receive historical images of solder paste and historical tags, wherein the historical tags can mark the bounding boxes of defects in the solder paste on the historical images. The electronic device 10 may train the first machine learning model 11 based on the historical images and historical tags using a supervised learning algorithm. Referring to Figure 3, in step S306, the electronic device 10 can magnify the bounding box on the solder paste image to update the bounding box. The method of magnifying the bounding box is similar to that in step S303, and therefore will not be described again. In step S307, since the tag indicates that the solder paste belongs to the second solder paste group, the electronic device 10 uses the second machine learning model 12 to perform image recognition on the portion of the image within the bounding box to generate a defect detection result. The second machine learning model 12 may include, but is not limited to, the MobileNetV3 model. In one embodiment, the electronic device 10 may receive historical images of solder paste and historical tags, wherein the historical tags can mark the bounding box of the solder paste defect on the historical image. The electronic device 10 may train the second machine learning model 12 based on the historical images and historical tags using a supervised learning algorithm. In one embodiment, the second machine learning model 12 may be configured in the AXI machine 14. The electronic device 10 may transmit a portion of an image within a defined bounding box to the AXI machine 14. The AXI machine 14 may perform image recognition on the portion based on the second machine learning model 12 or a conventional model of the AXI machine 14, thereby generating a defect detection result. The electronic device 10 may receive the defect detection result from the AXI machine 14. Referring to Figure 3, in step S308, the electronic device 10 can magnify the bounding box on the solder paste image to update the bounding box. The method of magnifying the bounding box is similar to that in step S303, and therefore will not be described again. In step S309, the electronic device 10 can vertically stitch together multiple images of solder paste to generate a stitched image. Specifically, the electronic device 10 can receive multiple images of the same solder paste from the AXI machine 14, wherein the multiple images may correspond to the same label. The multiple images are multiple cross-sections of the same solder paste, such as the first cross-section, the second cross-section, and the third cross-section as shown in FIG2C. In step S310, since the label indicates that the solder paste belongs to the third solder paste group, the electronic device 10 uses the third machine learning model 13 to perform image recognition on the part of the stitched image within the bounding box to generate a defect detection result. Figures 5 and 6 are schematic diagrams and flowcharts, respectively, of the training process of the third machine learning model 13' according to an embodiment of the present invention. The third machine learning model 13' under training includes a first model 51, a second model 52, and a fully connected network 53, wherein the first model 51 and the second model 52 are Siamese network models. In step S601, the electronic device 10 can receive a first image group, a first tag T1 corresponding to the first image group, a second image group, and a second tag T2 corresponding to the second image group. The first image group may include images L11, L12, and L13 corresponding to different sections of solder paste, and the second image group may include images L21, L22, and L23 corresponding to different sections of solder paste. The first tag T1 can indicate the bounding boxes of the multiple images L11, L12, and L13 in the first image group, the solder paste type name, and whether the first image group has defects. The second tag T2 can indicate the bounding boxes of the multiple images L21, L22, and L23 in the second image group, the solder paste type name, and whether the second image group has defects. In step S602, the preprocessing unit 50 of the electronic device 10 can stitch each image in the first image group (e.g., image L11, L12 or L13) into a first stitched image C1, and can stitch each image in the second image group (e.g., image L21, L22 or L23) into a second stitched image C2. In one embodiment, before stitching, the preprocessing unit 50 may enlarge the bounding boxes on the image group or the stitched image to update the bounding boxes. The method for enlarging the bounding boxes is similar to that in step S303, and therefore will not be described again. In step S603, the first model 51 can extract a first feature vector (embedding set) Emb1 from the first stitched image C1. The second model 52 can extract a second feature vector Emb2 from the second stitched image C2. Specifically, the electronic device 10 can input a portion of the first stitched image C1 within the bounding box into the first model 51. The first model 51 can extract the first feature vector Emb1 from this portion using structures such as convolutional layers or pooling layers. The electronic device 10 can input a portion of the second stitched image C2 within the bounding box into the second model 52. The second model 52 can extract the second feature vector Emb2 from this portion using structures such as convolutional layers or pooling layers. In step S604, the electronic device 10 can calculate the probability vector L of the loss function 54 based on the first feature vector Emb1 and the second feature vector Emb2, the first label T1 and the second label T2. Specifically, the fully connected network 53 of the electronic device 10 can generate a spatial vector D based on the first feature vector Emb1 and the second feature vector Emb2; in one embodiment, the dimensions of the first feature vector Emb1, the second feature vector Emb2 and the spatial vector D are [64, 1]. The spatial vector D can indicate the similarity or distance between the first feature vector Emb1 and the second feature vector Emb2. For example, the spatial vector D can be the Euclidean distance between the first feature vector Emb1 and the second feature vector Emb2. The electronic device 10 can input the spatial vector D into the loss function 54 to generate the probability vector L; in one embodiment, the dimensions of the probability vector L are [1, 2], representing the probabilities of passing and failing, respectively. In one embodiment, the loss function 54 may include a contrastive loss function, as shown in Equation (1): (1) Where L is the probability vector, Y is the label value, D is the Euclidean distance between the first feature vector Emb1 and the second feature vector Emb2, and m is the boundary threshold. The boundary threshold m can be user-defined. When the first label T1 and the second label T2 are the same, the label value Y can be 1. When the first label T1 and the second label T2 are different, the label value Y can be 0. When the first label T1 indicates that the first image group has defects and the second label T2 indicates that the second image has defects, the label value Y can be 1. When the first label T1 indicates that the first image group has defects but the second label T2 indicates that the second image does not have defects, the label value Y can be 0. In step S605, the electronic device 10 may update at least one of the first model 51 (or the second model 52) and the fully connected network 53 according to the probability vector L of the loss function 54. In one embodiment, the electronic device 10 may adjust the weights of the first model 51 and the second model 52 (e.g., the weights of neurons in the neural network) based on weight sharing. That is, the weights of neurons in the first model 51 and the second model 52 are exactly the same, thus saving memory space occupied by the neural network. In step S606, the electronic device 10 can determine whether the training dataset contains a next pair of image groups. If the next pair of image groups exists, step S601 is repeated. If the next pair of image groups does not exist, step S607 is executed. In step S607, the electronic device 10 can verify the performance of the model. Specifically, the electronic device 10 can use a third machine learning model 13' to perform image recognition on the verification dataset to generate defect detection results, wherein the verification dataset may contain multiple pairs of image groups and multiple pairs of labels corresponding to the multiple pairs of image groups. Then, the electronic device 10 calculates the performance of the third machine learning model 13' (e.g., metrics associated with the confusion matrix such as accuracy or recall) based on the detection results of the verification dataset. In step S608, the electronic device 10 can determine whether the training of the third machine learning model 13' is complete. If the training is complete, step S610 is executed. If the training is not yet complete, step S609 is executed. For example, the electronic device 10 can determine whether the performance of the third machine learning model 13' is higher than the required performance. If the performance of the third machine learning model 13' is higher than the required performance, the electronic device 10 can determine that the training is complete. If the performance of the third machine learning model 13' is lower than or equal to the required performance, the electronic device 10 can determine that the training is not yet complete. In step S609, if training is not yet complete, the electronic device 10 may adjust the hyperparameters of the third machine learning model 13' and retrain the third machine learning model 13' based on the updated hyperparameters. In step S610, the electronic device 10 can store the trained third machine learning model 13. Specifically, the electronic device 10 packages the network framework parameters of the first model 51 (or the second model 52) and the fully connected network 53 and the corresponding multiple neuron weights into a file according to a specific neural network data exchange format, and stores it as the trained third machine learning model 13. Figures 7 and 8 are schematic diagrams and flowcharts, respectively, of the inference process of the third machine learning model 13 according to an embodiment of the present invention. In step S801, the electronic device 10 can receive an input image group, an input tag Tn corresponding to the input image group, a reference image group, and a reference tag TR corresponding to the reference image group. The input image group may include images Ln1, Ln2, and Ln3 corresponding to different sections of the solder paste, and the reference image group may include images R1, R2, and R3 corresponding to different sections of the solder paste. The input tag Tn indicates the bounding box corresponding to the input image group, the solder paste category name, and whether there are defects. The reference tag TR indicates the bounding box corresponding to the reference image group, the solder paste category name, and whether there are defects. In step S802, the preprocessing unit 50 of the electronic device 10 can stitch each image in the input image group (e.g., image Ln1, Ln2 or Ln3) into an input stitched image Cn, and can stitch each image in the reference image group (e.g., image R1, R2 or R3) into a reference stitched image CR. In one embodiment, the preprocessing unit 50 may enlarge the bounding boxes on the image group or the stitched image to update the bounding boxes. The method for enlarging the bounding boxes is similar to that in step S303, and therefore will not be described again. In step S803, the electronic device 10 can input the input stitched image Cn and the reference stitched image CR to the third machine learning model 13. The first model 51 of the third machine learning model 13 can extract the input feature vector Embn from the input stitched image Cn. The second model 52 of the third machine learning model 13 can extract the reference feature vector EmbR from the reference stitched image CR. In step S804, the fully connected network 53 of the electronic device 10 can generate a spatial vector D based on the input feature vector Embn and the reference feature vector EmbR, wherein the spatial vector D, for example, indicates the Euclidean distance between the input feature vector Embn and the reference feature vector EmbR. In step S805, the electronic device 10 can input the spatial vector D into the Softmax function 55 to normalize the spatial vector D into a probability distribution with a sum of 1, wherein the probability distribution indicates the probability that the input image group and the reference image group belong to the same category. The electronic device 10 can generate a detection result based on the probability distribution. The detection result can indicate whether the solder paste in the input image group has defects. Assuming the reference image group has defects, if the output of the Softmax function 55 indicates that the probability of the input image group and the reference image group belonging to the same category is greater than a threshold, then the detection result generated by the electronic device 10 can indicate that the input image group has defects. If the output of the Softmax function 55 indicates that the probability of the input image group and the reference image group belonging to the same category is less than or equal to the threshold, then the detection result generated by the electronic device 10 can indicate that the input image group does not have defects. On the other hand, according to the description in Figure 1, when a defective solder paste is detected for the first time, the AXI machine 14 records the defect and the corresponding delimited box on the label, and then transmits the image of the solder paste and the label to the electronic device 10. Therefore, in the inference process, the input image group Tn received by the electronic device 10 are all images of defective solder paste. Assuming that the reference image group is not defective, if the output of the Softmax function 55 indicates that the probability of the input image group and the reference image group belonging to the same category is greater than a threshold, then the detection result generated by the electronic device 10 can indicate that the input image group is not defective. If the output of the Softmax function 55 indicates that the probability of the input image group and the reference image group belonging to the same category is less than or equal to the threshold, then the detection result generated by the electronic device 10 can indicate that the input image group is defective. Figure 9 is a schematic diagram of an electronic device 10 according to an embodiment of the present invention. The electronic device 10 may include a processor 90, a memory 91, a communication interface 93, and a user interface 94. The processor 90 is electrically connected to the memory 91, the communication interface 93, and the user interface 94, and is used to access and execute program code 92 stored in the memory 91 to perform various functions of the electronic device 10. The method 3 for detecting solder paste defects in Figure 3, the training flow in Figure 6, and the inference flow in Figure 8 can be compiled into program code 92 to implement modules or applications such as a first machine learning model 11, a second machine learning model 12, a third machine learning model 13, a preprocessing unit 40, a binarization unit 41, a preprocessing unit 50, a first model 51, a second model 52, or a fully connected network 53. The processor 90 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontroller (MCU), microprocessor, digital signal processor (DSP), programmable controller, application-specific integrated circuit (ASIC), graphics processing unit (GPU), image signal processor (ISP), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar elements or combinations thereof. The memory 91 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar element or combination thereof, for storing program code 92 that can be executed by the processor 90. The communication interface 93 transmits or receives signals wirelessly or via a wired connection. The communication interface 93 can also perform operations such as low-noise amplification, impedance matching, mixing, up- or down-frequency conversion, filtering, amplification, and similar operations. In one embodiment, the processor 90 can receive images and corresponding tags from an external electronic device (e.g., an AXI machine 14) via the communication interface 93. User interface 94 is, for example, a human machine interface (HMI) that includes input or output devices. User interface 94 can output multimedia information such as images or sound for user reference. For example, processor 90 can output the detection results of images to the user through user interface 94. Figure 10 is a flowchart of a method for detecting solder paste defects according to an embodiment of the present invention, wherein the method can be implemented by an electronic device 10. In step S1001, a first image of the solder paste and a label are received from an AXI machine, wherein the label indicates the bounding box of the solder paste. In step S1002, the bounding box is enlarged to update the bounding box. In step S1003, a machine learning model is used to generate a detection result of the solder paste defects based on the first image and the bounding box. In step S1004, the detection result is output. In summary, the method, electronic device and system for detecting solder paste defects of the present invention have the following characteristics: (1) The AXI machine and the electronic device are used to perform two defect detections. If the detection result of the second defect detection is still unqualified, the personnel at the manual inspection station will perform a third detection to avoid unnecessary manual inspection; (2) Multiple machine learning models are used to perform defect detection on multiple solder pastes of multiple solder paste groups to solve the problem of the high overkill rate of a single machine learning model; (3) The bounding box of the input image is magnified and other preprocessed to solve the problem of the bounding box generated by the existing AXI machine being offset or inaccurate; (4) For thicker solder paste, such as ball grid arrays and electroplated vias, multiple images corresponding to multiple cross-sections are spliced ​​to perform image recognition, which can fully detect solder paste defects; and (5) For thicker solder paste, the weights of the neurons of the two twin networks used are exactly the same, which can save the memory space occupied by the neural network. 1: System 10: Electronic Device 11: First Machine Learning Model 12: Second Machine Learning Model 13, 13': Third Machine Learning Model 14: Automatic X-ray Inspection Machine 15: Next Electronic Inspection Station 16: Manual Inspection Station 17: Repair Station 3: Method 40: Preprocessing Unit 41: Binarization Unit 51: First Model 52: Second Model 53: Fully Connected Network 54: Loss Function 55: Softmax Function 90: Processor 91: Memory 92: Code 93: Communication Interface 94: User Interface B, B': Bounding Box C1: First Stitched Image C2: Second Stitched Image Cn: Input Stitched Image CR: Reference Stitched Image D: Spatial Vector Emb1: First Feature Vector Emb2: Second Feature Vector Embn: Input Feature Vector EmbR: Reference Feature Vector IMGb: Binarized Image IMG, L11…L13, L21…L23, Ln1…Ln3, R1…R3: Image L: Probability Vector P1, P1': Top left point P2, P2': Bottom right point S301…S311, S601…S610, S801…S805, S1001…S1004: Step T1: First label T2: Second label ta1, ta2, ta3: Table Tn: Input label TR: Reference label Y: Label value Figure 1 is a schematic diagram of a system for detecting solder paste defects according to an embodiment of the present invention. Figure 2A is a schematic diagram of a table showing qualified and unqualified samples of a first solder paste group according to an embodiment of the present invention. Figure 2B is a schematic diagram of a table showing qualified and unqualified samples of a second solder paste group according to an embodiment of the present invention. Figure 2C is a schematic diagram of a table showing qualified and unqualified samples of a third solder paste group according to an embodiment of the present invention. Figure 3 is a flowchart of a method for detecting solder paste defects according to an embodiment of the present invention. Figure 4 is a schematic diagram of image recognition of a first solder paste group according to an embodiment of the present invention. Figures 5 and 6 are schematic diagrams and flowcharts of the training process of a third machine learning model according to an embodiment of the present invention, respectively. Figures 7 and 8 are schematic diagrams and flowcharts of the inference process of a third machine learning model according to an embodiment of the present invention, respectively. Figure 9 is a schematic diagram of an electronic device according to an embodiment of the present invention. Figure 10 is a flowchart of a method for detecting solder paste defects according to an embodiment of the present invention. S1001, S1002, S1003, S1004: Steps

Claims

1. A method for detecting solder paste defects, applicable to electronic devices, comprising: The electronic device receives a first image and a label of solder paste from an automated X-ray inspection machine, wherein the label indicates the delimitation frame of the solder paste; The electronic device magnifies the bounding box to update the bounding box; The electronic device uses a machine learning model to generate a defect detection result for the solder paste based on the first image and the bounding box, comprising: determining, based on the label, that the solder paste corresponds to a first solder paste group; in response to the solder paste corresponding to the first solder paste group, performing binarization on a portion of the first image within the bounding box to generate a binarized image; detecting the binarized image using the machine learning model to generate the detection result; and outputting the detection result by the electronic device. The method described in request item 1, wherein the step of performing the binarization includes: The binarization is performed by the electronic device according to the Otsu algorithm. The method described in Request 1, wherein the machine learning model includes MobileNetV3. The method as described in claim 1, wherein the solder paste in the first solder paste group corresponds to general. The method of claim 1, wherein the step of the electronic device using the machine learning model to generate the detection result of the defect of the solder paste based on the first image and the bounding box further includes: The electronic device determines, based on the tag, that the solder paste corresponds to the second solder paste group; In response to the solder paste corresponding to the second solder paste group, the electronic device transmits a portion of the first image within the delimited frame to the automated X-ray inspection machine, and receives the detection result corresponding to the portion from the automated X-ray inspection machine. As described in claim 5, the solder paste in the second solder paste group corresponds to one of the following: square flat leadless, small outline transistor, gull wing, ground pad, resistor, capacitor, laminator, and wafer. The method of claim 1, wherein the step of the electronic device using the machine learning model to generate the detection result of the defect of the solder paste based on the first image and the bounding box further includes: The electronic device receives a second and a third image of the solder paste from the automatic X-ray detector, wherein the first, second, and third images correspond to different cross-sections of the solder paste; the electronic device determines, based on the label, that the solder paste corresponds to a third solder paste group; in response to the solder paste corresponding to the third solder paste group, the electronic device stitches the first, second, and third images to generate a stitched image; and the electronic device generates the detection result based on the stitched image. The method as described in claim 7, wherein the machine learning model includes a Siamese network model, and wherein prior to the step of generating the detection result by the electronic device based on the stitched image, the method further includes a training process comprising: The electronic device extracts a first feature vector from the first stitched image through the first model in the twin network model; The electronic device extracts a second feature vector from the second stitched image through the second model in the twin network model; and the electronic device generates a spatial vector based on the first feature vector and the second feature vector through a fully connected network. The method described in request item 8, wherein the training process further includes: The electronic device calculates the probability vector of the loss function based on the spatial vector; The electronic device updates the first model and the second model based on the probability vector. The method as described in claim 9, wherein the step of the electronic device updating the first model and the second model according to the probability vector includes: The electronic device adjusts the first weight of the first model and the second weight of the second model based on weight sharing. The method described in request item 9, wherein the loss function includes a contrastive loss function. The method as described in claim 7, wherein the step of the electronic device generating the detection result based on the stitched image includes: The electronic device executes a Softmax function on the output of the machine learning model to generate the detection result. As described in claim 7, wherein the solder paste in the third solder paste group corresponds to one of the following: ball grid array and plated via. An electronic device for detecting solder paste defects includes: The communication interface is configured to receive at least one image and a tag of solder paste; The processor is electrically connected to the communication interface; and memory, electrically connected to the processor, configured to store code, wherein the code instructs the processor to perform the method for detecting solder paste defects as described in claim 1, based on the at least one image and the tag. A system for detecting solder paste defects includes: An automatic X-ray inspection machine configured to capture at least one image of solder paste on a circuit board and generate a label for the at least one image; an electronic device as claimed in claim 14 connected to the automatic X-ray inspection machine configured to generate a detection result of defects in the solder paste based on the at least one image and the label; and an electronic inspection station connected to the electronic device configured to receive the circuit board when the detection result indicates that it is qualified. And a manual inspection station, connected to the electronic device, configured to receive the circuit board when the detection result indicates that it is unqualified. A method for detecting solder paste defects, applicable to electronic devices, comprising: The electronic device receives a first image and a label of solder paste from an automated X-ray inspection machine, wherein the label indicates the delimitation frame of the solder paste; The electronic device magnifies the bounding box to update the bounding box; The electronic device uses a machine learning model to generate a defect detection result for the solder paste based on the first image and the bounding box, including: the electronic device receiving a second image and a third image of the solder paste from the automatic X-ray detector, wherein the first image, the second image, and the third image correspond to different cross-sections of the solder paste; the electronic device determining, based on the label, that the solder paste corresponds to a first solder paste group; in response to the solder paste corresponding to the first solder paste group, the electronic device stitching the first image, the second image, and the third image to generate a stitched image; the electronic device generating the detection result based on the stitched image; and the electronic device outputting the detection result. The method of claim 16, wherein the step of the electronic device using the machine learning model to generate the detection result of the defect of the solder paste based on the first image and the bounding box further includes: The electronic device determines, based on the tag, that the solder paste corresponds to the second solder paste group; In response to the solder paste corresponding to the second solder paste group, the electronic device performs binarization on a portion of the first image within the bounding box to generate a binarized image; and the electronic device uses the machine learning model to detect the binarized image to generate the detection result. The method described in request item 17, wherein the step of performing the binarization includes: The binarization is performed by the electronic device according to the Otsu algorithm. The method described in request item 17, wherein the machine learning model includes MobileNetV3. The method as described in claim 17, wherein the solder paste in the second solder paste group corresponds to the general. The method of claim 16, wherein the step of the electronic device using the machine learning model to generate the detection result of the defect of the solder paste based on the first image and the bounding box further includes: The electronic device determines, based on the tag, that the solder paste corresponds to a third solder paste group; In response to the solder paste corresponding to the third solder paste group, the electronic device transmits a portion of the first image within the delimited frame to the automated X-ray inspection machine, and receives the detection result corresponding to the portion from the automated X-ray inspection machine. As described in claim 21, the solder paste in the third group of solder pastes corresponds to one of the following: square flat leadless, small outline transistor, gull wing, ground pad, resistor, capacitor, laminator, and wafer. The method as described in claim 16, wherein the machine learning model includes a Siamese network model, and wherein prior to the step of generating the detection result by the electronic device based on the stitched image, the method further includes performing a training process, including: The electronic device extracts a first feature vector from the first stitched image through the first model in the twin network model; The electronic device extracts a second feature vector from the second stitched image through the second model in the twin network model; and the electronic device generates a spatial vector based on the first feature vector and the second feature vector through a fully connected network. The method as described in request item 23, wherein the training process further includes: The electronic device calculates the probability vector of the loss function based on the spatial vector; The electronic device updates the first model and the second model based on the probability vector. The method as described in claim 24, wherein the step of the electronic device updating the first model and the second model according to the probability vector includes: The electronic device adjusts the first weight of the first model and the second weight of the second model based on weight sharing. The method as described in request item 24, wherein the loss function includes a contrastive loss function. The method as described in claim 16, wherein the step of the electronic device generating the detection result based on the stitched image includes: The electronic device executes a Softmax function on the output of the machine learning model to generate the detection result. The method as described in claim 16, wherein the solder paste in the first solder paste group corresponds to one of the following: ball grid array and plated via. An electronic device for detecting solder paste defects includes: The communication interface is configured to receive at least one image and a tag of solder paste; The processor is electrically connected to the communication interface; and memory, electrically connected to the processor, configured to store code, wherein the code instructs the processor to perform the method for detecting solder paste defects as described in claim 16 based on the at least one image and the tag. A system for detecting solder paste defects includes: An automated X-ray inspection machine is configured to capture at least one image of solder paste on a circuit board and generate a label for the at least one image; The electronic device as described in claim 29 is connected to the automatic X-ray inspection machine and configured to generate a detection result of defects in the solder paste based on the at least one image and the label; an electronic inspection station is connected to the electronic device and configured to receive the circuit board when the detection result indicates that it is qualified; And a manual inspection station, connected to the electronic device, configured to receive the circuit board when the detection result indicates that it is unqualified.

Images