A PCB defect detection method and system based on adaptive multi-sub-model

By using an adaptive multi-submodel system, the priority and feature training of PCB defect detection are dynamically adjusted, which solves the problems of detection delay and resource waste caused by fixed priority in existing technologies, and achieves efficient and accurate PCB defect detection.

CN120707492BActive Publication Date: 2026-03-17EAST CHINA JIAOTONG UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-12
Publication Date
2026-03-17

AI Technical Summary

Technical Problem

In existing PCB defect detection technologies, fixed priority order leads to delays in high-frequency defect detection, low resource utilization, poor detection results for small sample defect types, and threshold judgment relies on human experience, which can easily lead to missed or false detections.

Method used

An adaptive multi-submodel system is adopted, which dynamically adjusts the priority order of submodels through dynamic priority control, feature sharing mechanism and uncertainty assessment, combined with reinforcement surrogate learning and Bayesian network. Multi-scale features are extracted by sharing backbone network, submodels are trained by gradient masking mechanism, and detection accuracy is improved by confidence threshold screening and cascade control.

Benefits of technology

It achieves rapid response to high-frequency defects, accurate identification of defects in small samples, efficient utilization of computing resources, improved real-time performance and accuracy of detection, and reduced computational burden.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120707492B_ABST
    Figure CN120707492B_ABST
Patent Text Reader

Abstract

This invention discloses a PCB defect detection method and system based on an adaptive multi-sub-model. The method involves acquiring images of the PCB surface using a high-resolution camera and preprocessing the images; using reinforcement learning for dynamic priority decision-making, rationally selecting the priority of sub-models based on equipment status and computational resource load; employing YOLOv8 as the backbone network, with multiple sub-models working in parallel and trained using gradient masks; implementing multi-level cascaded detection through confidence thresholding and variance analysis, and optimizing model parameters based on batch statistical results; and optimizing the sub-models through incremental training based on the recall and false positive rates of each detection. This invention proposes an adaptive multi-sub-model, which, through dynamic priority control and feature sharing mechanisms, dynamically adjusts the detection strategy while ensuring high accuracy, reducing computational burden and achieving efficient and accurate PCB defect detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical fields:

[0001] This invention belongs to the field of PCB defect detection technology, and specifically relates to a PCB defect detection method and system based on an adaptive multi-submodel. Background technology:

[0002] In the electronics manufacturing industry, the quality control of printed circuit boards (PCBs) is a crucial step in ensuring the performance and stability of electronic products. With technological advancements, traditional manual inspection methods can no longer meet the demands for high efficiency and precision. Therefore, automated inspection methods based on computer vision and artificial intelligence have become a research hotspot. Target detection technology, as an important branch of computer vision, has gained increasing attention in PCB defect detection in recent years. Target detection methods analyze each pixel in a PCB image to automatically identify and locate defect areas, greatly improving inspection efficiency and accuracy. PCB defect detection methods based on target detection offer advantages such as high efficiency, automation, and precision, and are gradually replacing traditional inspection methods, becoming an indispensable part of industrial production.

[0003] In traditional object detection methods, models learn to identify various defects and false positives by inputting a large number of images of different types. In actual production, different types of defects have different probabilities of occurrence. During training, this can lead to lower recognition accuracy for certain defect types. Low-frequency defect types may affect the overall recognition accuracy, and the occurrence of some defects may be related to equipment parameter settings.

[0004] Patent CN202210720601.4 discloses a "PCB Defect Detection System and Method." This method uses multiple "detection sub-models" to identify different defect types on printed circuit boards (PCBs). Each sub-model focuses on learning a specific defect type. The system inputs the target PCB image into each sub-model in priority order. If the predicted probability of the current sub-model exceeds a preset threshold, the defect type is determined; otherwise, the system continues to input the next sub-model. The system can also dynamically adjust the sub-model priority based on actual detection results. Compared to traditional single-model detection methods, this method greatly improves detection efficiency and accuracy. However, its fixed priority order may lead to delays in high-frequency defect detection. Furthermore, independent training of each sub-model can easily lead to feature redundancy, low resource utilization, and poor detection performance for small sample defect types. In addition, existing threshold judgments rely on human experience, which can easily lead to missed or false detections due to threshold rigidity.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention:

[0006] The purpose of this invention is to provide a PCB defect detection method and system based on an adaptive multi-submodel. Through the collaborative design of dynamic priority control, feature sharing mechanism and uncertainty assessment, it achieves rapid response to high-frequency defects, accurate identification of small-sample defects and efficient utilization of computing resources, meeting the high real-time and high robustness quality inspection requirements in industrial scenarios, thereby overcoming the shortcomings of the above-mentioned prior art.

[0007] To achieve the above objectives, this invention provides a PCB defect detection system based on an adaptive multi-sub-model, comprising an image acquisition module, a dynamic priority control module, a detection execution module, and an evaluation feedback module. The image acquisition module acquires images of bare PCB boards using a high-resolution camera. The dynamic priority control module dynamically adjusts the priority order of defect detection sub-models through reinforcement surrogate learning and a Bayesian network. The detection execution module integrates a YOLOv8-based multi-sub-model network, employs a shared backbone network to extract multi-scale features of the images, and combines a gradient masking mechanism for specific training of the sub-models. The evaluation feedback module uses confidence thresholding and variance analysis to achieve multi-level cascaded detection, and optimizes the parameters of the sub-models based on batch statistical results, dynamically adjusting the detection strategy.

[0008] A PCB defect detection method based on an adaptive multi-submodel includes the following steps:

[0009] S01. The image acquisition module acquires images of bare PCB boards using a high-resolution industrial camera and performs image preprocessing.

[0010] S02. The dynamic priority control module deploys reinforcement agent learning and a Bayesian network to dynamically generate a priority queue of detection sub-models based on equipment parameters, defect distribution, and computing resources. Sub-models are determined based on defect types, including vias, short circuits, rodent bites, stray wires, copper spatter, and open circuits. Reinforcement agent learning collects real-time data on the PCB ID, batch number, the average defect distribution of the last 10 detections, and GPU utilization, encoding the collected data into a 64-dimensional state vector. The module dynamically adjusts the detection order of sub-models based on PCB status and defect distribution, monitoring PCB parameters, defect types, and their distribution in real-time, and automatically adjusting the execution order of sub-models based on the monitoring data to prioritize areas more likely to have defects. The Bayesian network simultaneously analyzes the conditional probabilities of equipment process parameters and defect types to verify the rationality of the priority queue. Equipment process parameters include drilling machine power and chip mounting pressure.

[0011] S03. Multi-scale features are extracted using the YOLOv8 backbone network with CSPDarknet architecture. Feature activation values ​​are calculated for each sub-model, key channels are selected for gradient mask generation, and these key channels are used to train the sub-model. The gradient mask is adjusted according to the importance of each channel to ensure that each sub-model can focus on the most critical features, thereby improving the targeting and efficiency of detection.

[0012] S04. Combining confidence threshold and historical variance, perform confidence screening and cascade control on sub-models; classify the confidence of sub-model outputs to determine whether to trigger cascade control, looping until the confidence reaches the target or all sub-models are traversed; if the confidence of a detected defect is low and the historical variance exceeds the preset range, the system will automatically start the next detection stage to reduce missed detections; improve NMS processing of overlapping boxes, calculate the IoU value for all detection boxes pairwise, and retain only the detection box with the highest confidence for those that do not meet the retention conditions;

[0013] S05. After detection, batch statistical analysis of the performance indicators of each sub-model is performed, and the priority score of the sub-model is calculated. The performance indicators include recall rate and false positive rate. The priority queue of sub-models is reordered according to the priority score. Incremental training is triggered for sub-models with a recall rate of <70% for three consecutive batches. The conditional probability parameters are updated by gradient descent, and the conditional probability of defect-equipment parameters is adjusted according to the new data. The reinforcement learning policy library is updated, and the new state-action-reward data is added to the training set.

[0014] Preferably, in the technical solution, in step S01, the image preprocessing method is to perform illumination correction on the image using a local neighborhood normalization algorithm; assuming that the image size acquired by an industrial camera is 1024×1024 pixels, the illumination correction formula is:

[0015]

[0016] Where x, y represent coordinate points, I r (x,y) represents the brightness value at position (x,y) of each pixel in the original image. This represents the average brightness of neighboring pixels. This represents the sum of pixel brightness values ​​in a 9×9 neighborhood, where ε is a zero-prevention constant.

[0017] Preferably, in the technical solution, step S02, reinforcement agent learning includes constructing a state space and generating decisions, whereby the state space is defined as:

[0018] s t =[E ID ,B,D avg ,G]∈R 64 (2),

[0019] Among them, E ID The device ID is an 8-dimensional simplified code used to identify the device; B represents the batch number, and D represents the batch number. avg is the average defect distribution of the last 10 detections; G is the normalized GPU utilization, let low = 0, medium = 1, high = 2;

[0020] The state space is constructed by encoding the collected data into a 64-dimensional state vector, and then the 64-dimensional state vector s t As input to decision generation, the decision generation process is inferred and output using the dual-delay deep deterministic policy gradient (TD3) algorithm. The decision generation output includes a 6-dimensional vector π∈R of the sub-model priority queue. 6 , represents the calling order of the 6 types of defect models and the confidence threshold adjustment Δτ∈[-0.2,+0.2];

[0021] Decision generation is dynamically adjusted using the following reward function r:

[0022]

[0023] Among them, R t This represents the true positive rate for the current batch, with a weight of 3. This is a penalty for high GPU load.

[0024] The resulting sub-model priority queue has a 6-dimensional vector π∈R 6 The drilling machine power and patch pressure are collected in real time and validated using a Bayesian network. The results are then processed according to the sub-model priority queue and the 6-dimensional vector π∈R. 6 The final priority queue is generated and sent to the detection and execution module.

[0025] Preferably, in the technical solution, in step S03, for each sub-model, the feature channel contribution is calculated using the Grad-CAM algorithm, key channels with a contribution greater than or equal to the activation threshold are retained for gradient backpropagation, and the remaining channels are frozen for updating.

[0026]

[0027] Among them, C kc Let θ be the activation value of channel c for defect k, which is the feature channel contribution calculated by the Grad-CAM algorithm, and θ be the activation threshold.

[0028] θ is dynamically adjusted based on GPU utilization:

[0029]

[0030] Preferably, in the technical solution, step S04, the cascade control step includes: according to the priority queue obtained in step S02, prioritizing the invocation of the short-circuit defect detection sub-model; the short-circuit defect detection sub-model performs inference on the input sub-graph and outputs the confidence level p and the coordinates of the location box of the short-circuit defect; classifying the confidence level p, directly outputting when the confidence level is high, triggering variance analysis when the confidence level is medium, and skipping subsequent detections when the confidence level is low; when variance analysis is triggered, extracting the sequence {p1, p2, ..., p} of the confidence levels of the last ten short-circuit defects. 10}, calculate historical variance:

[0031]

[0032] Where, p k,i To represent the confidence level of defect type k in the i-th detection, p k,j Let k be the confidence level of defect type k in the j-th detection. When the condition is identified as a high variance state, the next level of detection is triggered.

[0033] After triggering the next level of detection, the mouse bite sub-model is called, and the confidence level p of the mouse bite defect is output. The confidence level p is graded and judged. If the confidence level is high, it is output directly. If the confidence level is medium, the variance analysis is triggered. If the variance analysis is high, the next level sub-model is called again until the confidence level reaches the standard or all sub-models are traversed.

[0034] Improve NMS handling of overlapping bounding boxes, retaining the following rules:

[0035]

[0036] For detection frames that do not meet the retention criteria, only the one with the highest confidence level is retained.

[0037] Preferably, in the technical solution, in step S05, based on the recall rate R... k And false positive rate F k Update the priority of the detection sub-model according to the following formula:

[0038]

[0039] Bayesian network parameter updates are performed by updating the conditional probability parameter ω through gradient descent.

[0040]

[0041] Where η is the learning rate, η is 0.01, D is the defect type, c is the equipment parameter, and ω is the learning rate. new Let ω be the new conditional probability parameter. old The old conditional probability parameters are used; the conditional probabilities of the defect-equipment parameters are adjusted based on the new data.

[0042] The incremental training mechanism only updates the fully connected layer parameters of the sub-model, and the loss function is:

[0043] L=α·L cls +(1-α)·L box ,α=0.8 (9),

[0044] Among them, L c1s For cross-entropy classification loss, L box CIoU localization loss is used, and the backbone network parameters are frozen during training.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The system employs a dynamic priority decision-making module to rationally select the priority of sub-models based on equipment status and computing resource load, ensuring efficient utilization of computing resources. During the detection process, multiple sub-models work in parallel, and a gradient mask training strategy ensures the specific learning of each sub-model, avoiding unnecessary waste of computing resources. The system utilizes confidence filtering and cascading control mechanisms to automatically trigger a higher-precision sub-model for secondary detection when the confidence level of the detection result is low or the variance is large, further improving detection accuracy. The system optimizes the sub-models through an incremental training mechanism based on the recall and false positive rates of each detection. Through the cooperation of these modules, the system can dynamically adjust the detection strategy while maintaining high accuracy, reducing the computational burden and achieving efficient and accurate PCB defect detection. Attached image description:

[0047] Figure 1 This is a flowchart of a PCB defect detection method based on an adaptive multi-submodel according to the present invention.

[0048] Figure 2 This is a functional block diagram of a PCB defect detection system module based on an adaptive multi-submodel according to the present invention.

[0049] Figure 3 This is a block diagram of the module structure of a PCB defect detection system based on an adaptive multi-submodel according to the present invention. Detailed implementation method:

[0050] The specific embodiments of the present invention will be described in detail below, but it should be understood that the scope of protection of the present invention is not limited to the specific embodiments.

[0051] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0052] like Figure 3 As shown, a PCB defect detection system based on an adaptive multi-sub-model includes an image acquisition module, a dynamic priority control module, a detection execution module, and an evaluation feedback module. The image acquisition module acquires images of bare PCB boards using a high-resolution camera. The dynamic priority control module dynamically adjusts the priority order of defect detection sub-models through reinforcement surrogate learning and a Bayesian network. The detection execution module integrates a YOLOv8-based multi-sub-model network, employs a shared backbone network to extract multi-scale features of the images, and combines a gradient masking mechanism for specific training of the sub-models. The evaluation feedback module uses confidence thresholding and variance analysis to achieve multi-level cascaded detection and optimizes the parameters of the sub-models based on batch statistical results, dynamically adjusting the detection strategy.

[0053] like Figure 1-2 As shown, a PCB defect detection method based on an adaptive multi-submodel includes the following steps:

[0054] S01. The image acquisition module acquires images of bare PCB boards using a high-resolution industrial camera and performs image preprocessing.

[0055] The image preprocessing method involves performing illumination correction on the image using a local neighborhood normalization algorithm. Assuming an image captured by an industrial camera has a size of 1024×1024 pixels, the illumination correction formula is:

[0056]

[0057] Where x, y represent coordinate points, I r (x,y) represents the brightness value at position (x,y) of each pixel in the original image. This represents the average brightness of neighboring pixels. This represents the sum of pixel brightness values ​​in a 9×9 neighborhood, where ε is a zero-prevention constant.

[0058] Lighting correction is performed using the average of 9×9 neighboring pixels, which avoids the effects of uneven lighting and ensures the accuracy of subsequent image processing.

[0059] Then, image segmentation and geometric correction are performed. For example, for a 1024×1024 image, the system divides it into multiple 512×512 pixel sub-images, with a 128-pixel overlap area between each sub-image to ensure that edge defects are not missed during the segmentation process.

[0060] S02. The dynamic priority control module deploys reinforcement agent learning and a Bayesian network to dynamically generate a priority queue of detection sub-models based on equipment parameters, defect distribution, and computing resources. Sub-models are determined based on defect types, including vias, short circuits, rodent bites, stray wires, copper spatter, and open circuits. Reinforcement agent learning collects real-time data on the PCB ID, batch number, the average defect distribution of the last 10 detections, and GPU utilization, encoding the collected data into a 64-dimensional state vector. The module dynamically adjusts the detection order of sub-models based on PCB status and defect distribution, monitoring PCB parameters, defect types, and their distribution in real-time, and automatically adjusting the execution order of sub-models based on the monitoring data to prioritize areas more likely to have defects. The Bayesian network simultaneously analyzes the conditional probabilities of equipment process parameters and defect types to verify the rationality of the priority queue. Equipment process parameters include drilling machine power and chip mounting pressure.

[0061] Reinforcement agent learning includes constructing a state space and generating decisions. The state space is defined as:

[0062] s t =[E ID ,B,D avg ,G]∈R 64 (2),

[0063] Among them, E ID The device ID is an 8-dimensional simplified code used to identify the device; B represents the batch number, and D represents the batch number. avg is the average defect distribution of the last 10 detections; G is the normalized GPU utilization, let low = 0, medium = 1, high = 2;

[0064] The state space is input with the PCB ID, batch number, mean of the last 10 defect distributions, and GPU utilization. Assume the PCB ID is encoded as an 8-dimensional vector [0,1,0,0,0,0,0,0], the batch number is B = 1500, and the normalized value is B / 1000 = 1.5. Assume there are currently six types of defects (vias, short circuits, rodent bites, stray wires, copper smears, and open circuits). The mean number of defects from the last 10 inspections is extracted as follows: vias (18%), short circuits (12%), rodent bites (5%). The GPU utilization is medium load (1). The resulting state space is s. t = [0,1,0,0,0,0,0,0,1.5,0.18,0.12,0.05,1] (Dimensions: 8+1+3+1=13, padding with zeros for any dimension less than 64);

[0065] The state space is constructed by encoding the collected data into a 64-dimensional state vector, and then the 64-dimensional state vector s t As input to decision generation, the decision generation process is inferred and output using the dual-delay deep deterministic policy gradient (TD3) algorithm. The decision generation output includes a 6-dimensional vector π∈R of the sub-model priority queue.6 , represents the calling order of the 6 types of defect models and the confidence threshold adjustment Δτ∈[-0.2,+0.2];

[0066] The TD3 algorithm generates a priority queue: a 6-dimensional vector π = [0.9, 0.2, 0.1, 0.05, 0.03, 0.01], representing the sequence of leaks → short circuits → mouse bites → stray copper → spurious emissions → open circuits. The confidence threshold is lowered from 0.85 to 0.8.

[0067] Decision generation is dynamically adjusted using the following reward function r:

[0068]

[0069] Among them, R t This represents the true positive rate for the current batch, with a weight of 3. This is a penalty for high GPU load; R t =0.92 (flaw detection rate 92%), GPU not overloaded (G=1, penalty term is 0), assuming a single cycle takes 9 seconds, the calculated r=1.86;

[0070] Finally, the Bayesian network was used for verification. Assuming the drilling machine power = 25W (standard 20W), the calculated P(leaking hole | power = 25W) = 0.85, confirming that the priority is reasonable.

[0071] According to the sub-model priority queue, the 6-dimensional vector π∈R 6 Generate the final priority queue, π = [1,4,2,6,5,3] (leaks → short circuits → mouse bites → stray copper → spurious → open circuits), and update the confidence threshold to τ = 0.85 + (-0.1) = 0.75, which is lowered from 0.85 to 0.75 to improve sensitivity;

[0072] S03. Based on the preprocessed 512×512 sub-image, multi-scale features are extracted using a YOLOv8 backbone network with a CSPDarknet architecture. The backbone network outputs three sets of feature maps (shallow features: 80×80×256, mid-level features: 40×40×512, deep features: 20×20×1024). Assuming the input sub-image contains a mouse bite defect (0.1mm×0.3mm in size) located in the upper left corner of the image, in the shallow feature map, the maximum activation value of all channels c∈{1,2,...,C} of each sub-model is calculated using the following formula:

[0073] max(A c ) = max (i,j) A c (i,j)

[0074] Among them, A c(i,j) represents the activation value of the c-th channel at position (i,j). The mouse bite model has 256 channels. Assume the maximum activation value of each channel is: max(A 32 =0.7, max(A) 45 =0.6, max(A) 78 Given θ = 0.3, current GPU utilization = medium (G = 1), using the default threshold θ = 0.5, generate the mask matrix:

[0075]

[0076] Then generate the gradient mask matrix of the mouse bite model.

[0077] During backpropagation, a mask is applied to the gradient of each channel, allowing only... The channel participates in weight updates;

[0078] During the training of the mouse bite model, the weights of the backbone network are kept frozen, channels 32 and 45 participate in gradient calculation to update the parameters of the fully connected layer, and the gradient of channel 78 is masked while the weights remain unchanged.

[0079] S04. Combining confidence threshold and historical variance, perform confidence screening and cascading control on sub-models; classify the confidence output of sub-models according to their level, and decide whether to trigger cascading control, looping until the confidence reaches the standard or all sub-models are traversed; if the confidence is ≥0.85, output the detection result directly and terminate subsequent sub-model calls; if the confidence is ∈[0.6,0.85) and the historical variance of the same type of defect is ≥0.1 (calculated based on the most recent 50 similar detection results), trigger the next level sub-model detection; for overlapping detection boxes with an intersection-over-union ratio (IoU) ≥0.5, only retain the one with the highest confidence to suppress redundant detection results; improve NMS processing of overlapping boxes, calculate the IoU value for each pair of all detection boxes, and for detection boxes that do not meet the retention conditions, only retain the one with the highest confidence;

[0080] The cascaded control steps include: Firstly, based on the dynamic priority queue, the short-circuit defect detection sub-model is called. The short-circuit defect detection sub-model performs inference on the input subgraph and outputs the confidence level p and the bounding box coordinates of the short-circuit defect. Assuming the short-circuit sub-model outputs a confidence level of p = 0.78 and bounding box coordinates (x1, y1, x2, y2), high confidence levels are directly output (p ≥ 0.85), medium confidence levels trigger variance analysis (p ∈ [0.6, 0.85)), and low confidence levels skip subsequent detections (p < 0.6). With the current confidence level p = 0.78, variance analysis is triggered. When variance analysis is triggered, the sequence of the confidence levels of the last ten short-circuit defects {p1, p2, ..., p} is extracted. 10}, calculate historical variance:

[0081]

[0082] Where, p k,i To represent the confidence level of defect type k in the i-th detection, when When the condition is identified as a high variance state, the next level of detection is triggered.

[0083] After triggering the next level of detection, the mouse bite sub-model is called, and the confidence level p of the mouse bite defect is output. The confidence level p is graded and judged. If the confidence level is high, it is output directly. If the confidence level is medium, the variance analysis is triggered. If the variance analysis is high, the next level sub-model is called again until the confidence level reaches the standard or all sub-models are traversed.

[0084] Improve NMS handling of overlapping bounding boxes, retaining the following rules:

[0085]

[0086] For detection frames that do not meet the retention criteria, only the one with the highest confidence level is retained.

[0087] S05. After detection, batch statistical analysis of the performance indicators of each sub-model is performed, and the priority score of the sub-model is calculated. The performance indicators include recall rate and false positive rate. The priority queue of sub-models is reordered according to the priority score. Incremental training is triggered for sub-models with a recall rate of <70% for three consecutive batches. The conditional probability parameters are updated by gradient descent, and the conditional probability of defect-equipment parameters is adjusted according to the new data. The reinforcement learning policy library is updated, and the new state-action-reward data is added to the training set.

[0088] After every 100 tests, the detection results of each sub-model are summarized, and the recall rate R is calculated. k Inspection rate F k According to the recall rate R k And false positive rate F k Update the priority of the detection sub-model according to the following formula:

[0089]

[0090] Add a 0.1 smoothing term to the denominator to avoid division by zero and balance the impact of false positives. Press P. k Generate a new priority queue from high to low;

[0091] Bayesian network parameter updates are performed by updating the conditional probability parameter ω through gradient descent.

[0092]

[0093] Where η is the learning rate, η is 0.01, D is the defect type, c is the equipment parameter, and ω is the learning rate. new Let ω be the new conditional probability parameter.old The old conditional probability parameters are used; the conditional probabilities of the defect-equipment parameters are adjusted based on the new data.

[0094] Incremental training trigger condition: 3 consecutive batches of R k <0.7; Using the latest 100 samples containing this defect, only unfreeze the fully connected layer parameters of the sub-model, while keeping the weights of the shared feature extraction layer unchanged;

[0095] The incremental training mechanism only updates the fully connected layer parameters of the sub-model, and the loss function is:

[0096] L=α·L cls +(1-α)·L box ,α=0.8 (9),

[0097] Among them, L c1s For cross-entropy classification loss, L box CIoU localization loss is used, and the backbone network parameters are frozen during training.

[0098] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. An adaptive multi-submodel-based PCB defect detection method, comprising the following steps: S01, an image acquisition module acquires a PCB bare board image through a high-resolution industrial camera, and pre-processes the image; S02, a dynamic priority control module deploys reinforcement agent learning and a Bayesian network, dynamically generates a priority queue of detection submodels according to device parameters, defect distribution and computing resources; the submodels are determined according to defect types, which include holes, short circuits, mouse bites, stray, copper, and open circuits; the reinforcement agent learning collects the ID, batch number, defect distribution average of the last 10 detections, and GPU utilization rate data of the PCB in real time, encodes the collected data into a 64-dimensional state vector; the detection order of the submodels is dynamically adjusted according to the PCB state and defect distribution, the PCB parameters, defect types and distribution are monitored in real time, and the execution order of the submodels is automatically adjusted based on the monitoring data; the Bayesian network synchronously analyzes the conditional probability of device process parameters and defect types, and verifies the rationality of the priority queue; the device process parameters include the power of the drilling machine and the patch pressure; S03, a YOLOv8 backbone network based on CSPDarknet architecture is used to extract multi-scale features, the feature activation values are calculated for each submodel, the key channels are selected for gradient mask generation, and the submodels are trained using these key channels; The gradient mask is adjusted according to the importance of each channel to ensure that each submodel can focus on the most critical features; S04, combining confidence threshold and historical variance, the submodels are subjected to confidence screening and cascade control; the confidence level output by the submodel is graded to determine whether to trigger cascade control, and the process is repeated until the confidence level meets the standard or all submodels are traversed; if the confidence level of the detected defect is low and the historical variance exceeds the preset range, the system will automatically start the next detection stage to reduce missed detection; The NMS processing of overlapping boxes is improved, and the IoU value is calculated for all detection boxes; for detection boxes that do not meet the retention conditions, only the one with the highest confidence is retained; S05, after the detection is completed, the performance indicators of each submodel are batched and counted, and the priority score of the submodel is calculated; the performance indicators include recall rate and false detection rate; the priority queue of the submodel is reordered through the priority score; The submodel with a recall rate of less than 70% for three consecutive batches triggers incremental training; the conditional probability parameters are updated through gradient descent, and the conditional probability of defects-device parameters is adjusted according to new data; The reinforcement learning strategy library is updated, and new state-action-reward data is added to the training set.

2. The adaptive multi-submodel based PCB defect detection method of claim 1, wherein: In step S01, the image preprocessing method is to perform illumination correction on the image through a local neighborhood normalization algorithm; assuming that the image size collected by a certain industrial camera is 1024x1024 pixels, the illumination correction formula is: (1), where x, y represent the coordinate point, represents the luminance value of each pixel position in the original image, represents the average value of the luminance of the neighboring pixels, represents the sum of the luminance values of the pixels in the 9x9 neighborhood, is a constant to prevent zero.​ 3. The adaptive multi-submodel based PCB defect detection method of claim 1, wherein: In step S02, reinforcement agent learning includes building a state space and generating decisions, the state space is defined as: (2), wherein, is an 8-dimensional simplified encoding of the device ID, identifying the device; B is a batch number, is the average of the distribution of defects in the last 10 inspections; G is the normalized GPU utilization, with low = 0, medium = 1, and high = 2; The constructed state space encodes the collected data into a 64-dimensional state vector, and then the 64-dimensional state vector is input into the decision generation model As the input of decision generation, the decision generation infers and outputs by a double-delay deep deterministic policy gradient TD3 algorithm, and the decision generation output includes a 6-dimensional vector of a sub-model priority queue , indicating the calling order of the 6 defect models and the confidence threshold adjustment amount ; The decision generation is dynamically adjusted by the following reward function r: (3), wherein, is the true positive rate for the current batch, with a weight of 3, is a penalty term when the GPU load is high; The obtained sub-model priority queue 6-dimensional vector Real-time acquisition of the power of the drilling machine and the patch pressure, verification through a Bayesian network, generation of a final priority queue according to the sub-model priority queue 6-dimensional vector And issuing the final priority queue to a detection execution module.

4. The adaptive multi-submodel based PCB defect detection method of claim 1, wherein: In step S03, for each submodel, the Grad-CAM algorithm is used to calculate the contribution degree of the feature channel, the key channels with a contribution degree greater than or equal to the activation threshold are retained for gradient backpropagation, and the remaining channels are frozen for updating: (4), wherein, is the activation value of the channel c to the defect k, i.e., the feature channel contribution degree calculated by the Grad-CAM algorithm, is an activation threshold value; Dynamic adjustment according to GPU utilization: (5)。 5. The adaptive multi-submodel based PCB defect detection method of claim 1, wherein: In step S04, the steps of the cascade control include: according to the priority queue obtained in step S02, preferentially calling the short-circuit defect detection sub-model, the short-circuit defect detection sub-model performing reasoning on the input sub-graph, and outputting the confidence p of the short-circuit defect and the positioning box coordinates; performing hierarchical judgment on the confidence p, directly outputting when the confidence is high, triggering variance analysis when the confidence is medium, and skipping subsequent detection when the confidence is low; when the variance analysis is triggered, a sequence of the confidence of the short-circuit defect in the last ten times is extracted , and the historical variance is calculated: (6), wherein, is the confidence of defect type k in the i-th detection, is the confidence of defect type k in the j-th detection, when is determined as high variance state, triggering the next level detection; Triggering the next level of detection after calling the mouse bite model, output the confidence p of the mouse bite defect, and grade the confidence p. High confidence is directly output, medium confidence triggers variance analysis, and low confidence skips subsequent detection. If the variance analysis is high variance state, continue to call the next level of sub-model until the confidence meets the standard or all sub-models are traversed. The improved NMS processing overlapping boxes retains the rules as follows: , For detection boxes that do not meet the retention conditions, only the one with the highest confidence is retained.

6. The adaptive multi-submodel based PCB defect detection method of claim 1, wherein: In step S05, according to the recall rate and the false detection rate , the detection sub-model priority is updated according to the following formula : (7), Bayesian network parameter updating, updating conditional probability parameters by gradient descent , (8), wherein, is a learning rate, is taken as 0.01, D is a defect type, c is a device parameter, is a new conditional probability parameter, is an old conditional probability parameter; adjusts the conditional probability of defect-device parameter according to new data; The incremental training mechanism only updates the full connection layer parameters of the sub-model, and the loss function is as follows: (9), wherein, is a cross-entropy classification loss, is a CIoU localization loss, and the backbone network parameters are frozen during training.

7. A PCB defect detection system based on adaptive multi-submodel, characterized in that: The system detects by the method in any one of claims 1-6.

Citation Information

Patent Citations

  • PCB defect detection system and PCB defect detection method

    CN114820598A

  • Coflow scheduling system based on TD3 algorithm

    CN116582502A