AOI-oriented welding spot defect rapid adaptation detection method, device and system and storage medium

Abstract

The embodiment of the invention provides an AOI-oriented welding spot defect rapid adaptation detection method, device and system and a storage medium, and the method comprises the steps: (1) obtaining a whole image of an AOI image, constructing a supervised model, employing the supervised model to position welding spots on the whole image, and carrying out the preliminary judgment, and the judgment result comprises good products and defects; (2) cutting the front backbone network with the supervision model into a lightweight feature extractor, performing welding spot sub-graph extraction representation on the image which is preliminarily judged as a good product by using the lightweight feature extractor, and constructing a good product feature vector library; (3) cutting a to-be-rechecked welding spot sub-graph from the whole graph of which the judgment result is a non-good product according to the detection frame, and inputting the to-be-rechecked welding spot sub-graph into the feature extraction network to obtain a feature vector of the to-be-rechecked welding spot sub-graph; and (4) rechecking the obtained feature vector through the Euclidean distance and a quantile threshold value based on good product distribution, and performing welding spot defect judgment according to a preset threshold value. The method has the advantages of rapid defect detection and improved precision.

Description

Technical Field

[0001] This invention relates to the fields of image processing and machine vision technology, and in particular to a rapid adaptation detection method, device, system and storage medium for solder joint defects oriented to AOI. Background Technology

[0002] In printed circuit board (PCB) soldering and surface mount technology (SMT) production, solder joint defects (such as insufficient solder, solder bridging, and cold solder joints) directly affect the electrical performance and reliability of the product. Automated optical inspection (AOI), as a key online quality control method, relies on high-resolution industrial cameras, coaxial / lateral multi-light sources, and stable motion platforms to obtain high-contrast images to highlight the geometric and reflective morphological features of solder joints. However, manual visual inspection based on AOI images is inefficient, difficult, and easily affected by subjective factors; therefore, automated intelligent inspection has become an industry trend. Existing AOI solder joint defect detection methods mainly include template matching, supervised learning, and unsupervised learning.

[0003] Template matching methods (such as CN202010306399.1 and CN202311815535.X) match the region to be detected with a pre-established reference image template, such as based on normalized cross-correlation, structural similarity, or feature point matching. When the matching score is lower than the threshold or the geometric deviation exceeds the threshold, it is judged as a defect. However, template matching methods have the following drawbacks: (1) Good products have diverse appearances and significant changes in light reflection, making it difficult for templates to cover all shapes, resulting in high maintenance costs and a high false alarm rate; (2) Template matching methods are more suitable for detecting overall shape defects. For small solder joint details, it is difficult to achieve stable alignment under template alignment errors and reflection saturation, leading to missed detections.

[0004] Supervised deep learning methods (such as CN201910795988.8 and CN202011542008.2) identify good and bad products in AOI images through classification / detection / segmentation models. The supervision relies on a large amount of labeled data for training, which requires manual labeling of good and bad samples. This has the following drawbacks: (1) Severe class imbalance (i.e., the number of good products is much greater than that of bad products) and scarcity of defects lead to training difficulties and limited generalization; (2) Standard differences and conflicting labeling across workshops / products make it difficult for a single model to be uniformly adapted; (3) New products require a large amount of labeling and retraining, which makes deployment and updates complex and timeliness poor; (4) Online updates require on-site labeling and training, which has poor engineering feasibility and high maintenance costs.

[0005] Unsupervised learning (such as generative adversarial networks) methods (such as CN201910020529.2) usually learn the low-dimensional latent space distribution of normal samples as positive samples, map the image to be inspected to the same latent space, and judge the anomaly based on the difference from the distribution of positive samples and the threshold. They have the following defects: (1) They can only use the whole image of the device as input for modeling, which makes it difficult to locate small solder joint defects, resulting in poor solder joint detection effect and insufficient interpretability; (2) They are affected by background / device differences, the attribution of anomalies is unclear, and the value of maintenance guidance is limited; (3) The training of generative adversarial networks is unstable and hyperparameter sensitive, and it is difficult to train and update the model in real time on site.

[0006] In view of the shortcomings of existing methods, there is an urgent need to provide a detection method that can achieve rapid defect detection and reduce false alarm / false negative rates. Summary of the Invention

[0007] This application provides a rapid adaptation detection method, device, system, and storage medium for solder joint defects in AOI, which solves the problems of low detection accuracy and false / missed detection in existing panel defect detection methods.

[0008] In a first aspect, embodiments of this application provide a rapid adaptive detection method for solder joint defects for AOI (Automated Optical Inspection), comprising the following steps:

[0009] Step 1: Obtain the complete AOI image, construct a supervised model, use the supervised model to locate solder joints in the complete image and make a preliminary judgment. The judgment results include good products and defects.

[0010] Step 2: Prune the pre-backbone network of the supervised model into a lightweight feature extractor. Use the lightweight feature extractor to extract and represent the solder joint sub-images of the images initially judged as good products, and construct a good product feature vector library.

[0011] Step 3: Cut the image of the non-defective product from the whole image in Step 1 into a sub-image of the solder joint to be re-inspected according to the detection frame, normalize the sub-image of the solder joint to be re-inspected, and input it into the feature extraction network to obtain the feature vector of the sub-image of the solder joint to be re-inspected.

[0012] Step 4: Re-examine the feature vector obtained in Step 3 using Euclidean distance and quantile thresholds based on the good product distribution, and determine the solder joint defects according to the preset threshold.

[0013] Furthermore, the supervised model constructed in step one also includes:

[0014] Collect and label samples of the entire AOI image;

[0015] Add inspection boxes and category labels to the solder joints;

[0016] A supervised model is trained to identify solder joint locations and defect categories.

[0017] Furthermore, step one, which uses a supervised model to locate and preliminarily determine the solder joints in the entire image, also includes:

[0018] Set thresholds for detection bounding box scores, classification scores, and distance scores for feature retrieval;

[0019] The entire AOI image is input into a supervised model to obtain the solder joint detection box, category, and classification score;

[0020] If the initial judgment result is good and the classification score is greater than or equal to the classification threshold, then the product is directly output as good; if the initial judgment result is defective, or good but the classification score is less than the classification threshold, then the feature retrieval and anomaly judgment process is initiated.

[0021] Furthermore, step two, which involves constructing a feature vector library for high-quality products, also includes:

[0022] Extract feature vectors from the good product subgraph in step one, cluster the feature vectors using unsupervised clustering, and write the clustered feature vectors into the good product feature index library.

[0023] Furthermore, it also includes:

[0024] For manually confirmed false positive good products, add them to the inventory and update the clustering / structure of the good product feature index.

[0025] Furthermore, step four also includes:

[0026] The feature vector of the weld point sub-image to be re-inspected is used for nearest neighbor search. If the re-inspection distance score is greater than the set threshold, the re-inspection is judged as a defect; otherwise, the re-inspection is judged as a good product.

[0027] Furthermore, it also includes:

[0028] Adjust the quantile threshold to improve the compatibility of multi-line bodies / multi-standards.

[0029] Secondly, embodiments of this application provide a rapid adaptation detection device for solder joint defects for AOI (Automated Optical Inspection), comprising the following modules:

[0030] The image acquisition module is used to acquire panel AOI images and construct a training set;

[0031] The solder joint detection and initial screening module outputs solder joint detection boxes, categories, and classification scores based on a supervised detection model.

[0032] The sub-image trimming and normalization module is used to trim the weld point sub-images according to the detection frame, unify the size, and perform standardization / normalization processing;

[0033] The feature extraction module is used to prune and freeze the pre-backbone network of the supervised detection model and output the feature vector of the sub-image.

[0034] The feature index library building module is used to extract feature vectors from the good subgraphs of the training set using the feature extraction module, and to organize the feature vectors into a clustering index structure using unsupervised clustering.

[0035] The re-inspection module is used to call the feature extraction module and feature index library to perform feature extraction and nearest neighbor retrieval on the subgraph to be inspected, and to determine anomalies based on preset thresholds.

[0036] Furthermore, it also includes:

[0037] The online incremental and adaptation module is used to incrementally add manually confirmed false positive good products to the database and update the clustering / structure of the feature index library.

[0038] Furthermore, it also includes:

[0039] The explanation and visualization output module is used to output the solder joint detection box, defective solder joint sub-image, and defect type.

[0040] Thirdly, embodiments of this application provide a computer system including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the rapid adaptation detection method for solder joint defects for AOI as described above.

[0041] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the rapid adaptation detection method for solder joint defects oriented to AOI as described above.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] (1) The method provided by the present invention can improve the stability and transferability of solder joint defect detection and reduce false alarms / false negatives when the good products are diverse and the judgment standards are inconsistent (or even contradictory) across customers / production lines.

[0044] (2) The method provided by this invention can significantly improve the detection performance and generalization ability of the model under the conditions of scarce defect samples and extreme class imbalance (the proportion of good products is significantly higher than that of defective products);

[0045] (3) Supervised learning methods fail under conditions of small sample size / unlabeled on-site data. The method provided by this invention can enable the model to quickly adapt to new workshops, new products and new standards by using unlabeled data.

[0046] (4) The method provided by this invention provides interpretability and positioning capability at the solder joint level and the area level, which facilitates front-line machine adjustment and maintenance;

[0047] (5) The method provided by the present invention can reduce the cost of online maintenance and update, and supports incremental updates and multi-line body differentiated configuration. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the method flow provided in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the device composition provided in an embodiment of the present invention;

[0050] Figure 3 Training flowchart of the method provided in the embodiments of the present invention

[0051] Figure 4 The detection flowchart is provided for the method of the present invention. Detailed Implementation

[0052] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0053] Figure 1 The flowchart of a rapid adaptive detection method for solder joint defects for AOI provided by an embodiment of the present invention includes the following steps:

[0054] Step 1: Obtain the complete AOI image, construct a supervised model, use the supervised model to locate solder joints in the complete image and make a preliminary judgment. The judgment results include good products and defects.

[0055] Step 2: Prune the pre-backbone network of the supervised model into a lightweight feature extractor. Use the lightweight feature extractor to extract and represent the solder joint sub-images of the images initially judged as good products, and construct a good product feature vector library.

[0056] Step 3: Cut the image of the non-defective product from the whole image in Step 1 into a sub-image of the solder joint to be re-inspected according to the detection frame, normalize the sub-image of the solder joint to be re-inspected, and input it into the feature extraction network to obtain the feature vector of the sub-image of the solder joint to be re-inspected.

[0057] Step 4: Re-examine the feature vector obtained in Step 3 using Euclidean distance and quantile thresholds based on the good product distribution, and determine the solder joint defects according to the preset threshold.

[0058] Based on the same inventive concept, embodiments of the present invention provide a rapid adaptation detection system for solder joint defects for AOI (Automated Optical Inspection), such as... Figure 2 As shown, it includes the following functional modules:

[0059] The image acquisition module mainly includes an industrial camera, a coaxial / side light source, and a motion platform, which are used to output AOI images and construct training and testing sets;

[0060] The solder joint inspection and initial screening module, based on a supervised inspection model (such as YOLO series or RT-DETR), outputs the solder joint inspection box, category (good / defect type) and its classification score (category probability or score).

[0061] The sub-image trimming and normalization module is used to trim the weld point sub-images according to the detection frame, unify the size, and perform standardization / CLAHE lighting normalization;

[0062] The feature extraction module prunes and freezes the front backbone network of the trained supervised model and outputs a fixed-length sub-graph feature vector, preferably 256 to 1024 dimensions.

[0063] The feature index library construction module uses the feature extraction module to extract feature vectors from the good subgraphs of the training set, and uses unsupervised clustering (such as K-Means) to organize the feature vectors into a cluster index structure. This data structure is the feature index library to be used, which supports "two-stage retrieval", namely cluster center screening → intra-cluster nearest neighbor search.

[0064] The re-inspection module calls the feature extraction module and feature index library to perform feature extraction and nearest neighbor retrieval on the subgraph to be inspected, and makes anomaly judgments based on preset thresholds.

[0065] Preferably, it also includes an online incremental and adaptation module, which is used to incrementally add manually confirmed false positive good products to the database and update the clustering / structure of the feature index library; in addition, it can also quickly switch between different feature index libraries to adapt to multiple standards.

[0066] Preferably, it also includes an explanation and visualization output module, which is used to output the weld joint detection box, defect weld joint sub-image, and defect type.

[0067] To facilitate understanding by those skilled in the art of the methods and corresponding model training processes provided in the embodiments of this application, Figure 3 Here is a flowchart of the training process in one embodiment of the present invention, and the training process is as follows:

[0068] Step T1 (Training the supervised model): Collect and label AOI image samples, and perform detection boxes and category labeling on solder joints. The category labeling includes good products and various defect types. Specifically, defect types include insufficient solder, solder bridging, cold solder joints, etc. Train a supervised model (such as YOLO series or RT-DETR) to identify solder joint locations and defect categories.

[0069] Step T2 (Crop solder joint sub-images): Use the solder joint coordinate frames output by the trained supervised model to crop good solder joint sub-images in batches on the training set, and normalize the size and lighting (preferred range is 64×64~128×128) to form a good solder joint sub-image training set.

[0070] Step T3 (Feature Extraction Model): Prune the front backbone network of the trained supervised model into a feature extraction model and freeze the parameters; in the feature extraction model, preferably, fine-tune it using a good weld point sub-image training set to improve the feature representation capability;

[0071] Step T4 (Unsupervised Clustering): Extract feature vectors from the training good solder joint sub-images, and perform unsupervised clustering (such as K-Means) on all extracted feature vectors to generate cluster centers and inverted indexes;

[0072] Step T5 (Building a feature index library): Write the clustered feature vectors into the good product feature index library.

[0073] Figure 4 This is a schematic diagram of the monitoring process corresponding to an embodiment of the present invention, and the specific process is as follows:

[0074] Step D1 (Setting Thresholds): Set the detection box score threshold, classification score threshold, and feature retrieval distance score threshold;

[0075] Step D2 (Supervised Model Inference): Input the AOI image to be inspected into the supervised model to obtain the solder joint detection box, category and classification score;

[0076] The output results are processed according to the following rules: (1) If the category is judged as good and the classification score is ≥ the classification threshold, then the good product is directly output; (2) If the category is judged as defective, or judged as good but the classification score is < the classification threshold, then the feature retrieval and anomaly judgment process is entered.

[0077] Step D3 (Sub-image trimming): Trim the sub-image of the solder joint to be re-inspected according to the inspection frame, and perform size and lighting normalization processing;

[0078] Step D4 (Feature Extraction): Normalize the sub-image to be re-examined and input it into the feature extraction network to obtain the feature vector to be examined;

[0079] Step D5 (Two-stage nearest neighbor search): In the feature index library, first calculate the distance between the feature vector to be detected and the center vector of each cluster, and select the Top-M nearest clusters; then perform nearest neighbor search within the selected clusters to calculate the minimum distance score and similar samples.

[0080] The distance between vectors is calculated using Euclidean distance, and the formula is as follows:

[0081]

[0082] in These are feature vectors from the Good Product Feature Index Library. Let n be the feature vector to be detected, and n be the dimension of the feature vector. and These are the k-th dimension components;

[0083] Step D6 (Re-inspection): When the minimum distance score is greater than the distance score threshold, it is judged as an abnormal solder joint; otherwise, it is judged as a good solder joint.

[0084] Step D7 (Output and Record): Output the final determination and record it for subsequent online incremental and threshold updates.

[0085] Preferably, the method provided by the present invention, through online incremental processing, also has the following advantages:

[0086] (1) Incremental entry: For manually confirmed false positives of good products, their features are added to the corresponding good product feature index library to improve coverage and reduce false positives; (2) Incremental clustering maintenance: New good product features are absorbed and updated according to the nearest cluster; when the size of a single cluster exceeds the upper limit or the cluster center drift exceeds the threshold, local or full re-clustering is triggered; (3) Multiple standards: For the same product under different customer standards, the threshold and good product feature index library are maintained independently in different namespaces for quick switching; (4) Versioning and rollback: Both the good product feature index library and the threshold are version managed, supporting rollback and auditing.

[0087] This application also provides a computer system including a computer program that, when executed by a processor, implements the above-described method.

[0088] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0089] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0090] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0091] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0092] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0093] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0094] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0095] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0096] To facilitate a better understanding of the implementation process and advantages of this invention by those skilled in the art, the invention is further illustrated through the following practical application examples.

[0097] Practical application cases

[0098] A workshop produces four types of boards (A / B / C / D) on the same production line. The number of solder joints on each board ranges from 500 to 1000, with a median of approximately 730. A 5MP industrial camera is used for imaging, with monochrome 8-bit and a combination of coaxial and side lighting. The original field of view is stitched together to obtain the whole board image.

[0099] The input to the supervisory detector is a complete image of a single device obtained by cropping the device coordinates; the input to the re-inspection sub-image is cropped to 96×96 according to the detection frame and normalized to [0,1] after CLAHE.

[0100] Data was collected over 10 days and divided by time. The first 7 days were used for training and database construction, and the last 3 days were used for testing. The test set included 460 good boards and 40 defective boards, with a total of approximately 365k solder joints.

[0101] In practical applications, the selection of models and the setting of parameters are as follows:

[0102] The supervisory detector uses YOLOv8-s (which can be equivalent to RT-DETR-R18), with a classification score threshold of 0.60 (used for good / defect determination, which is adjustable), and the categories are set to {good, defective (sub-types such as insufficient solder, cold solder joints, etc.)};

[0103] Backbone pruning and feature dimension: Take the first four modules of the YOLOv8-s backbone network and prune to remove the detection head branches; perform L2 normalization on the output features to obtain the feature vector f∈R^512 (feature dimension=512), and freeze the parameters;

[0104] Feature library and retrieval: The namespace maintains 4 independent good product libraries according to board type, with an initial sample size of approximately 10 good boards × 700 solder joints ≈ 7000 vectors for each library; KMeans clustering is used, with a total number of clusters set to 256, and Euclidean distance is used as the distance metric; the index uses a two-stage retrieval method, with Top-M clusters = 8 and the number of nearest neighbors searched within each cluster K = 5; the distance threshold is based on the q = 97th percentile of the good product-to-good product distance distribution, and multiplied by an adjustable coefficient β ∈ [0.95, 1.05], where the default adjustable coefficient β = 1.00;

[0105] Latency and Resources: In an NVIDIA GeForce RTX 4090 single-card environment, the supervised detector's single-graph inference speed is approximately 2ms, and the single solder joint subgraph feature extraction and retrieval is approximately 20ms.

[0106] After going live, the training and monitoring process is carried out, specifically corresponding to T1–T5 and D1–D7, as shown in the implementation case:

[0107] Step T1: Perform supervised training, label solder joint boxes and categories, train YOLOv8-s for 80 epochs, and mAP50 (good / defective) is 0.971.

[0108] Step T2: Perform sub-image cropping, use the detector to extract good solder joint sub-images from the training set, unify them to 96×96 and perform illumination normalization.

[0109] Step T3: Build a feature extractor, prune the backbone to the fourth module and freeze it; optionally, fine-tune the good subgraph by 3 epochs to improve intra-class compactness.

[0110] Step T4: Perform clustering and database construction, extract all good product features and cluster them with K=256 to obtain cluster centers and inverted indexes.

[0111] Step T5: Write the index and save the feature index library and the 97th percentile median threshold Dist_q97 (automatically estimated and versioned by quantile).

[0112] Step D1: Set the threshold. The classification threshold is 0.60 (can be adjusted by product / customer); the distance threshold = Dist_q97 × β (default β = 1.0, can be adjusted on-site using the slider).

[0113] Step D2: Conduct initial screening. If the product is judged to be good and the classification score is ≥ the classification threshold, it is released directly; the rest (defective or low classification score good) enter the re-inspection.

[0114] Steps D3 to D5: Complete the re-inspection process, including cropping to 96×96, extracting features f_q∈R^512, two-stage retrieval (Top-8 clusters, K=5 within each cluster) and obtaining the minimum distance score;

[0115] Step D6: Complete the judgment. If the minimum distance score is greater than the distance threshold, it is judged as abnormal; otherwise, it is a good product. False good products that have been manually confirmed can be added to the inventory incrementally (online index update, approximately <5 s / 1000 records).

[0116] Step D7: Output solder joint bounding box, final category, similar good sample and distance score, and record for threshold recalibration and version rollback.

[0117] Comparison of experimental results and effects

[0118] The solder joint-level metrics used in the application case include precision, recall, and AUROC. AUROC is obtained by plotting the receiver operating characteristic (ROC) curve across thresholds and calculating the area under the curve. Solder joint-level precision, recall, and AUROC are used to evaluate the model's discriminative ability, but they do not directly reflect the release performance. Even if these metrics are high, they may not meet actual field requirements. Production workshops are more concerned with board-level first-pass yield (BTR), which refers to the proportion of boards judged as "zero-defect" in a sample of good circuit boards, calculated as good boards divided by the total number of boards. Assuming that solder joint judgments on each circuit board are approximately independent, the probability of a good board achieving "zero false alarms" is calculated as (1 - solder joint-level false alarm rate) raised to the power of the number of solder joints per board. Initially, this workshop only used supervisory inspection, with approximately 700 solder joints per board and a solder joint-level false alarm rate of approximately 5.0%. Based on this, the calculated BTR was (1 - 0.05)^700 ≈ 0, meaning that almost all good boards were falsely judged and required rework.

[0119] The test set consisted of 460 good boards and 40 defective boards, with a median of approximately 730 solder joints per board. Compared to supervised inspection methods only, as shown in Table 1, this invention improved both accuracy and recall at the solder joint level, and further enhanced AUROC. Simultaneously, the false alarm rate at the solder joint level significantly decreased from approximately 5.0% to approximately 0.012%, leading to a significant increase in the board-level first-pass yield from approximately 0% to approximately 90%–92%. While maintaining high recall, the system effectively suppressed false alarms through a two-stage retrieval process and a threshold mechanism based on good product distribution, significantly improving the first-pass performance of good boards on the production line and demonstrating its engineering application value. The system simultaneously outputs the specific defect type, the three most similar good product sub-images, and their distances, facilitating localization and interpretation. On-site adjustments can be made quickly according to customer or product quality boundaries by fine-tuning the distance threshold scaling factor β or switching namespaces, without requiring model retraining. Newly added, manually confirmed good product samples are added to the database, and the relevant local indexes are rebuilt within 10 minutes; thresholds and indexes support versioning and rollback to meet auditing and traceability requirements.

[0120] Table 1 Comparison of results between existing technical methods and the method of this invention

[0121] Therefore, this invention provides a corresponding rapid solder joint detection method, device, system, and storage medium based on a hybrid AOI solder joint defect detection framework of "supervised detector localization and initial screening + feature extraction network pruned by supervised model + nearest neighbor retrieval of good product feature library + adjustable threshold judgment". It uses a supervised model to accurately locate solder joints on the whole device image and make an initial judgment on good / defective. The pre-backbone network of the trained supervised model is pruned into a lightweight feature extractor, which is used to extract features from the good product solder joint sub-image to construct a good product feature vector library. In the re-inspection stage, solder joints judged as defects or with insufficient classification scores enter the feature retrieval re-inspection step. Anomalies are judged and interpreted by Euclidean distance and quantile thresholds based on the distribution of good products (or manually fine-tuned thresholds). This invention can significantly reduce false alarms / false negatives in panel detection and achieve rapid adaptation to multiple lines / standards.

[0122] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A rapid adaptive detection method for solder joint defects for AOI (Automated Optical Inspection), characterized in that, Includes the following steps: Step 1: Obtain the complete AOI image, construct a supervised model, use the supervised model to locate solder joints in the complete image and make a preliminary judgment. The judgment results include good products and defects. Step 2: Prune the pre-backbone network of the supervised model into a lightweight feature extractor. Use the lightweight feature extractor to extract and represent the solder joint sub-images of the images initially judged as good products, and construct a good product feature vector library. Step 3: Cut the image of the non-defective product from the whole image in Step 1 into a sub-image of the solder joint to be re-inspected according to the detection frame, normalize the sub-image of the solder joint to be re-inspected, and input it into the feature extraction network to obtain the feature vector of the sub-image of the solder joint to be re-inspected. Step 4: Re-examine the feature vector obtained in Step 3 using Euclidean distance and quantile thresholds based on the good product distribution, and determine the solder joint defects according to the preset threshold.

2. The detection method according to claim 1, characterized in that, Step one, which involves building a supervised model, also includes: Collect and label samples of the entire AOI image; Add inspection boxes and category labels to the solder joints; A supervised model is trained to identify solder joint locations and defect categories.

3. The detection method according to claim 1, characterized in that, Step one, which uses a supervised model to locate solder joints in the entire image and make preliminary judgments, also includes: Set thresholds for detection bounding box scores, classification scores, and distance scores for feature retrieval; The entire AOI image is input into a supervised model to obtain the solder joint detection box, category, and classification score; If the initial judgment result is good and the classification score is greater than or equal to the classification threshold, then the product is directly output as good; if the initial judgment result is defective, or good but the classification score is less than the classification threshold, then the feature retrieval and anomaly judgment process is initiated.

4. The detection method according to claim 1, characterized in that, Step two, which involves constructing a feature vector library for high-quality products, also includes: Extract feature vectors from the good product subgraph in step one, cluster the feature vectors using unsupervised clustering, and write the clustered feature vectors into the good product feature index library.

5. The detection method according to claim 1, characterized in that, Also includes: For manually confirmed false positive good products, add them to the inventory and update the clustering / structure of the good product feature index.

6. The detection method according to claim 1, characterized in that, Step four also includes: The feature vector of the weld point sub-image to be re-inspected is used for nearest neighbor search. If the re-inspection distance score is greater than the set threshold, the re-inspection is judged as a defect; otherwise, the re-inspection is judged as a good product.

7. The detection method according to claim 1, characterized in that, Also includes: Adjust the quantile threshold to improve the compatibility of multi-line bodies / multi-standards.

8. A rapid adaptation and detection device for solder joint defects for AOI, comprising the following modules: The image acquisition module is used to acquire panel AOI images and construct a training set; The solder joint detection and initial screening module outputs solder joint detection boxes, categories, and classification scores based on a supervised detection model. The sub-image trimming and normalization module is used to trim the weld point sub-images according to the detection frame, unify the size, and perform standardization / normalization processing; The feature extraction module is used to prune and freeze the pre-backbone network of the supervised detection model and output the feature vector of the sub-image. The feature index library building module is used to extract feature vectors from the good subgraphs of the training set using the feature extraction module, and to organize the feature vectors into a clustering index structure using unsupervised clustering. The re-inspection module is used to call the feature extraction module and feature index library to perform feature extraction and nearest neighbor retrieval on the subgraph to be inspected, and to determine anomalies based on preset thresholds.

9. A computer system, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the rapid adaptation detection method for solder joint defects for AOI as described above.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the rapid adaptation detection method for solder joint defects for AOI as described above.

Images