A defect detection method, device, apparatus and storage medium

Abstract

The application discloses a defect detection method, device, equipment and storage medium. The method comprises the following steps: acquiring a real-time video stream by performing real-time image acquisition on a detection area of a sample to be detected; inputting the real-time video stream into a preset target detection model to perform real-time inference, and obtaining a bounding box of a suspected defect, a defect category and confidence data; performing risk quantification processing on the bounding box of the suspected defect, the defect category and the confidence data to obtain a defect attention index, comparing the defect attention index with a preset risk threshold to obtain a risk level evaluation result; and adaptively adjusting scanning motion, imaging acquisition and illumination control parameters based on the evaluation result, and obtaining defect detection data through hierarchical detection processing. The application realizes real-time closed-loop control of the detection process, greatly improves the detection efficiency and the micro defect detection rate, and is suitable for the defect detection scene of precision components.

Description

Technical Field

[0001] This application relates to the field of microscopic defect detection technology, and in particular to a defect detection method, apparatus, equipment and storage medium. Background Technology

[0002] With the continuous upgrading of electronic manufacturing technology, the integration and assembly density of PCBs (Printed Circuit Boards) are constantly increasing, and the width of solder joints and traces is gradually developing towards the micrometer level, placing increasingly higher demands on the accuracy and efficiency of PCB defect detection. However, existing PCB defect detection equipment struggles to balance the detection capability of minute defects with the high-speed detection requirements of production lines. It either drastically reduces scanning speed to ensure detection accuracy, affecting production line efficiency, or sacrifices imaging quality to keep up with production line pace, leading to missed detection of minute defects and seriously impacting PCB product yield and quality control efficiency. To ensure PCB product quality, precise and efficient automated detection of various defects on PCBs is necessary.

[0003] Current methods are generally divided into traditional rule-based automatic optical inspection methods and offline deep learning inspection methods. Traditional rule-based automatic optical inspection methods typically use a fixed scanning speed and a general illumination mode to perform a full-coverage scan of the PCB board, relying on manually designed image processing rules and traditional computer vision algorithms for defect identification. However, in scenarios with complex PCB backgrounds and strong reflective interference, this method has poor environmental adaptability, is prone to generating a large number of false alarms, requires a lot of manual review, and cannot adapt optimal imaging parameters for different defect types, resulting in insufficient contrast and low detection rates for some defects. Offline deep learning inspection methods typically first complete the image acquisition and storage of the entire PCB board area, and then input the acquired images in batches into a deep learning model for offline defect analysis. However, this open-loop processing mode cannot use the recognition results to control the hardware acquisition process in reverse. Problems such as motion blur and focus misalignment generated during the shooting process cannot be corrected in real time. At the same time, indiscriminate acquisition of the entire area generates massive amounts of redundant data, resulting in a serious waste of computing power and storage resources.

[0004] Therefore, how to achieve real-time closed-loop control of the PCB defect detection process, break through the industry bottleneck of difficulty in balancing detection efficiency and recognition accuracy, improve the detection rate of minute defects in complex scenarios, and reduce the risk of false alarms and missed detections is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the defect detection method, apparatus, device, and storage medium provided in this application can realize real-time closed-loop control of the PCB defect detection process, overcome the industry bottleneck of difficulty in balancing detection efficiency and recognition accuracy, improve the detection rate of minute defects in complex scenarios, and reduce the risk of false alarms and missed detections. The defect detection method, apparatus, device, and storage medium provided in this application are implemented as follows: This application provides a defect detection method, including: Image acquisition and processing are performed on the sample area to be tested to obtain a real-time video stream; The real-time video stream is input into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence data. The bounding box, defect category, and confidence level data of the suspected defects are subjected to risk quantification processing to obtain the defect attention index; The defect attention index is compared with a preset risk threshold to obtain a risk level assessment result. Based on the risk level assessment results, the scanning motion parameters, imaging acquisition and illumination control parameters are adjusted to obtain the detection control parameters; Based on the aforementioned detection control parameters, the current detection area is subjected to graded detection processing to obtain defect detection data.

[0006] In some embodiments, the risk quantification processing of the bounding box, defect category, and confidence data of the suspected defect to obtain a defect attention index includes: The bounding box of the suspected defect is subjected to size conversion and spatial distribution statistical processing to obtain actual physical size data and defect distribution density data; The defect categories are subjected to a preset risk level matching process to obtain the type risk weight coefficient; The confidence data is subjected to pre-set confidence interval matching processing to obtain the confidence correction coefficient; The actual physical size data is subjected to reverse weighting to obtain size weight coefficients; The size weighting coefficient, the type risk weighting coefficient, the confidence correction coefficient, and the defect distribution density data are weighted and fused to obtain a risk quantification value. The risk quantification values ​​are summarized and statistically processed to obtain the defect attention index.

[0007] In some embodiments, the process of comparing the defect attention index with a preset risk threshold to obtain a risk level assessment result includes: The defect attention index is compared with a preset risk threshold to obtain the comparison difference data. The comparison difference data is subjected to preset risk interval matching processing to obtain the initial risk level classification result; The confidence data is subjected to threshold filtering to obtain effective defect feature data; The valid defect feature data and the initial risk level classification result are subjected to consistency verification processing to obtain the verified initial risk level assessment result. The defect category data is subjected to risk weight adaptation processing to obtain the level correction parameters; The initial risk level assessment result after verification is calibrated and fused with the level correction parameters to obtain the risk level assessment result.

[0008] In some embodiments, the adjustment of scanning motion parameters, imaging acquisition and illumination control parameters based on the risk level assessment results to obtain detection control parameters includes: The risk level assessment results are subjected to hierarchical analysis and strategy matching to obtain the parameter adjustment benchmark strategy; The parameter adjustment baseline strategy and the current operating parameters are fused and planned to obtain the scanning motion adjustment parameters; The parameter adjustment benchmark strategy is adapted to the operating state to obtain the imaging acquisition adjustment parameters; The defect category data is subjected to scene attribute matching processing to obtain lighting control adjustment parameters; The scanning motion adjustment parameters, the imaging acquisition adjustment parameters, and the illumination control adjustment parameters are subjected to spatiotemporal alignment and time delay compensation calibration to obtain multi-parameter linkage control data. The multi-parameter linkage control data is subjected to compliance verification processing to obtain the detection control parameters.

[0009] In some embodiments, the step of inputting the real-time video stream into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence score data includes: The real-time video stream is subjected to frame extraction, format standardization, and image preprocessing to obtain the image data to be inferred; The image data to be inferred is input into a preset target detection model for multi-scale feature extraction and defect feature recognition to obtain initial defect detection result data. The initial defect detection result data is subjected to non-maximum suppression and redundant detection box removal to obtain a defect candidate box dataset. The defect candidate box dataset is subjected to category classification and confidence level calibration to obtain defect category data and confidence level data; The defect candidate box dataset is subjected to pixel coordinate extraction and boundary range normalization to obtain bounding box data; The bounding box data, defect category data, and confidence score data are subjected to compliance verification and invalid data removal to obtain the bounding box, defect category, and confidence score data of suspected defects.

[0010] In some embodiments, the step of performing graded detection processing on the current detection area based on the detection control parameters to obtain defect detection data includes: The risk level assessment results are graded and matched with execution strategies to obtain a graded detection execution plan; The hierarchical detection execution scheme is adapted to the basic operating parameters to obtain continuous detection control instructions; The graded detection execution scheme is associated and adapted with the defect category and the confidence data to obtain a high-precision review control instruction. The high-precision verification control command and the detection control parameters are fused and analyzed to obtain the linkage execution command; Based on the linked execution command, the sample to be tested is subjected to multi-focal length image acquisition, depth-of-field fusion and secondary defect verification reasoning to obtain a clear full-depth-of-field image and defect confirmation data. The defect confirmation data, acquisition location data, acquisition time data, and the full-view depth-of-field clear image are associated, bound, and structured for verification to obtain defect detection data.

[0011] The apparatus provided in this application includes: The processing module is used to acquire and process images of the sample area to be tested, and obtain a real-time video stream. The processing module is also used to input the real-time video stream into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category and the confidence data; The processing module is also used to perform risk quantification processing on the bounding box, defect category and confidence data of the suspected defect to obtain the defect attention index. The assessment module is used to compare the defect attention index with a preset risk threshold to obtain a risk level assessment result. The processing module is also used to adjust the scanning motion parameters, imaging acquisition and illumination control parameters based on the risk level assessment results to obtain detection control parameters; The grading module is used to perform grading detection processing on the current detection area based on the detection control parameters to obtain defect detection data.

[0012] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0013] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.

[0014] The defect detection method, apparatus, device, and storage medium provided in this application embodiment acquires real-time images of the detection area of ​​the sample under test to obtain a real-time video stream; the real-time video stream is input into a preset target detection model for real-time inference to obtain the bounding box, defect category, and confidence level data of suspected defects; the bounding box, defect category, and confidence level data of suspected defects are subjected to risk quantification processing to obtain a defect attention index, which is then compared with a preset risk threshold to obtain a risk level assessment result; based on the assessment result, the scanning motion, imaging acquisition, and illumination control parameters are adaptively adjusted, and defect detection data is obtained through graded detection processing. This enables real-time closed-loop control of the PCB defect detection process, overcoming the industry bottleneck of balancing detection efficiency and recognition accuracy, improving the detection rate of minute defects in complex scenarios, reducing the risk of false alarms and missed detections, and solving the technical problems mentioned in the background art. Attached Figure Description

[0015] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram illustrating the implementation process of a defect detection method provided in an embodiment of this application; Figure 2 A schematic diagram illustrating the implementation process of obtaining a defect attention index provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a defect detection device provided in an embodiment of this application. Detailed Implementation

[0017] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0018] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0019] This application provides a defect detection method, apparatus, equipment, and storage medium, preferably applied to automated microscopic inspection of PCB solder joints after reflow oven in SMT (Surface Mount Technology) production lines, achieving efficient and accurate closed-loop detection of micron-level defects on PCB boards. The detection system based on this application includes an imaging unit, a motion unit, an edge computing unit, and a main control unit. The imaging unit includes a 20-megapixel global shutter industrial camera, an automatic zoom microscope lens, and a multi-mode illumination unit. The multi-mode illumination unit integrates coaxial light, ring light, and bottom backlight. The bottom backlight is installed below the motorized stage, and the center of the stage uses a glass surface for transmission illumination. The motion unit includes a high-precision linear motor-driven XY motorized stage, a Z-axis focal length adjustment mechanism, and a motion controller supporting online speed regulation. The edge computing unit uses an edge computing module and deploys a deep learning target detection model accelerated by TensorRT. The main control unit communicates with the imaging unit, motion unit, and edge computing unit to achieve data interaction and command issuance.

[0020] Figure 1 This is a schematic diagram illustrating the implementation flow of a defect detection method provided in an embodiment of this application, including steps 101 to 106. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a defect detection method. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0021] Step 101: Image acquisition and processing of the sample area to be tested is performed to obtain a real-time video stream.

[0022] In this embodiment, the PCB board under test is fixed at the center of the electric stage. A motion controller controls the stage to perform an S-shaped coverage scan of the entire test area of ​​the PCB board at a first speed. Simultaneously, an industrial camera continuously acquires real-time images of the PCB board test area at a fixed frame rate. The acquired continuous image frames are integrated into a real-time video stream, which is then transmitted to the edge computing unit with low latency using shared memory technology. The first speed is a pre-set high-speed cruising speed, preferably 50 mm / s in this embodiment, and the frame rate of the industrial camera is preferably 60 fps.

[0023] Step 102: Input the real-time video stream into the preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence data.

[0024] In this embodiment, the training and deployment of the deep learning object detection model are completed in advance. Specifically, microscopic image data containing normal solder joints and five typical PCB defects—such as bridging, insufficient solder, tombstoning, misalignment, and solder balls—are collected in advance. Tiny defects with a diameter of less than 20 micrometers are highlighted to construct a training dataset. Based on this dataset, the deep learning object detection model is trained, resulting in a model file with a single-frame inference time of less than 15ms at a set input resolution. The model file is then deployed to the edge computing unit after TensorRT acceleration. During detection, the real-time video stream is preprocessed frame-by-frame by extracting and standardizing the format. The preprocessed image frames are input into the pre-set deep learning object detection model for real-time inference processing. The model outputs the bounding boxes of all suspected defects in the current image frame, the corresponding defect categories, and confidence scores, while simultaneously outputting the material and feature attribute data of the currently detected area. In this embodiment, the deep learning object detection model is preferably the YOLO11s model, and the input resolution is preferably 640×640.

[0025] Step 103: Perform risk quantification processing on the bounding box, defect category, and confidence data of suspected defects to obtain the defect attention index.

[0026] In this embodiment, based on the bounding box of the suspected defect obtained through reasoning, the actual physical size of a single defect is calculated. Simultaneously, the number of all suspected defects within the current detection area and the distance between adjacent defects are statistically analyzed to obtain defect distribution density data for the current detection area. A pre-set type risk weight is matched according to the defect category, with higher type risk weights corresponding to defect types that have a greater impact on product performance. Inverse weighting is assigned based on the actual physical size of the defect; the smaller the actual physical size of the defect, the higher the corresponding size weight coefficient. A confidence correction coefficient corresponding to a pre-set confidence interval is matched based on the confidence data. The size weight coefficient, type risk weight, confidence correction coefficient, and defect distribution density data are weighted and fused to obtain the risk quantification value of a single defect. Finally, the risk quantification values ​​of all single defects within the current detection area are summarized and statistically analyzed to obtain the defect attention index for the current detection area.

[0027] Step 104: Compare the defect attention index with the preset risk threshold to obtain the risk level assessment result.

[0028] In this embodiment, a risk threshold for the defect attention index is pre-set according to the production line quality control requirements. The defect attention index calculated for the current inspection area is compared with the preset risk threshold. When the defect attention index is lower than the preset risk threshold, the current inspection area is determined to be a low-risk safe area, corresponding to a field of view containing only normal solder joints and substrate background. When the defect attention index is higher than the preset risk threshold, the current inspection area is determined to be a high-risk defect area, corresponding to a field of view containing suspected defects that need to be verified. Simultaneously, the judgment result is verified by combining the confidence data of suspected defects, eliminating the interference of invalid defect data with confidence levels lower than the preset qualified value, and finally obtaining the risk level assessment result of the current inspection area.

[0029] Step 105: Based on the risk level assessment results, adjust the scanning motion parameters, imaging acquisition and illumination control parameters to obtain the detection control parameters.

[0030] In this embodiment, for detection areas determined to be safe areas, the first speed of the current stage, imaging acquisition parameters, and illumination mode are kept unchanged, and detection control parameters for continuous cruising are generated. For detection areas determined to be defective areas, the bounding box of the suspected defect is first converted into the actual position in physical space through the affine transformation matrix between the pre-calibrated pixel coordinate system and the physical coordinate system of the stage. At the same time, the image frame acquisition timestamp of the detected defect is recorded, and the image transmission time, model inference time, and command communication time of the system are statistically analyzed to obtain the total system processing delay. Combined with the current running speed of the stage, the predicted position of the defect at the time of command execution is calculated. Then, the corresponding scanning motion adjustment parameters, imaging acquisition adjustment parameters, and illumination control adjustment parameters are generated based on the predicted position. The scanning motion adjustment parameters include the second speed of the stage, which in this embodiment is a reduced low speed, preferably 2.5 mm / s. In extreme cases, it may include the stage's reverse retraction control parameters to ensure the defect area is stably centered in the field of view. The imaging acquisition adjustment parameters include the multi-level micro-motion parameters of the Z-axis focal length adjustment mechanism and the camera's continuous shooting acquisition parameters. The illumination control adjustment parameters are generated according to the defect type and the material properties of the detection area. When highly reflective areas such as solder pads and gold fingers are identified, coaxial light activation parameters are generated; when silkscreen characters and surface foreign matter areas are identified, ring light enhancement parameters are generated; and when through holes, vias, and substrate crack areas are identified, bottom backlight activation parameters are generated. The above scanning motion adjustment parameters, imaging acquisition adjustment parameters, and illumination control adjustment parameters are spatiotemporally aligned and time-delay compensated for calibration, and after compliance verification, the final detection control parameters are obtained.

[0031] Step 106: Perform graded detection processing on the current detection area based on the detection control parameters to obtain defect detection data.

[0032] In this embodiment, for the safe area, based on the continuous cruise detection control parameters, the first speed of the stage is maintained to continuously perform image acquisition and model inference processing, without storing redundant image data without defects. For the defect area, the detection control parameters are sent to the motion controller and imaging unit in real time, controlling the stage to smoothly decelerate to the second speed when the defect area reaches the center of the field of view. Simultaneously, the Z-axis focal length adjustment mechanism is controlled to perform multi-level micro-motion above and below the focal plane, and the camera is controlled to capture multiple frames of images. Then, depth fusion processing is performed on the multiple frames of images to obtain a clear panoramic depth image of the defect area. The clear panoramic depth image is then input into the deep learning target detection model for secondary verification and inference to confirm the final category and attributes of the defect. The confirmed defect data, the corresponding acquisition location data, acquisition time data, and the clear panoramic depth image are uniquely identified and bound, and then format standardization and compliance verification processing is performed to obtain structured defect detection data, which is then encrypted and stored. After the verification acquisition and data storage of the defect area are completed, the stage is controlled to return to the first speed, and the remaining areas of the PCB board to be tested are scanned and detected until the detection process of all areas of the PCB board to be tested is completed.

[0033] This application's embodiments achieve full automation and real-time closed-loop control of the defect detection process through a complete workflow of image acquisition, model inference, risk quantification, risk level determination, multi-parameter adjustment, and graded detection. This effectively addresses the industry pain point of traditional defect detection, which either prioritizes detection efficiency at the expense of recognition accuracy or focuses on detection accuracy at the expense of production line cycle time. This application achieves dynamic detection of the area under test through real-time video stream acquisition, avoiding the lag of offline detection. Through risk quantification and graded detection strategies, it achieves a collaborative detection mode of "high-speed cruising in low-risk areas and precise verification in high-risk areas," adapting to the high-speed, high-precision detection needs of precision components such as PCBs in SMT production lines. This significantly reduces manual intervention, improves the automation level and stability of detection, and simultaneously balances detection efficiency and defect detection rate, reducing false positives and false negatives, providing efficient and reliable technical support for production line quality control.

[0034] In the above Figure 1 Based on the above, this application embodiment also provides a schematic diagram of the implementation process for obtaining a defect attention index. For example... Figure 2 As shown, steps 201 to 206 are included: Step 201: Perform size conversion and spatial distribution statistical processing on the bounding box of suspected defects to obtain actual physical size data and defect distribution density data.

[0035] In this embodiment, for each suspected defect within the current detection field of view, the pixel size of the defect bounding box is converted into actual physical size data in physical space using a pre-calibrated affine transformation matrix, accurately obtaining the actual length and width specifications of the defect. Simultaneously, the total number of all suspected defects within the current detection field of view and the physical distance between adjacent defects are statistically processed to calculate the defect distribution density within the current field of view, i.e., the defect distribution density data. This application focuses on accurately converting the size of tiny defects with a diameter less than 20 micrometers, ensuring that the size data of these tiny defects can be accurately incorporated into subsequent risk calculations.

[0036] Step 202: Perform preset risk level matching processing on the defect categories to obtain the type risk weight coefficient.

[0037] In this application embodiment, risk levels for different defect categories are pre-defined based on the impact of PCB defects on the electrical performance and safety of the product. A corresponding type risk weight coefficient is assigned to each defect category, with higher coefficients corresponding to greater impacts on product performance. Specifically, for the inferred defect categories, preset risk levels and weight coefficients are matched. For the five typical PCB defects—solder bridging, tombstoning, insufficient solder, misalignment, and solder balls—solder bridging and tombstoning are assigned the highest type risk weight coefficients according to the order of short circuit and functional failure risk from highest to lowest. Insufficient solder and misalignment are assigned the lowest, followed by insufficient solder and misalignment. Solder balls correspond to the basic type risk weight coefficients. After matching, the type risk weight coefficient for a single defect is obtained.

[0038] Step 203: Perform pre-set confidence interval matching processing on the confidence data to obtain the confidence correction coefficient.

[0039] In this embodiment, multiple confidence intervals are pre-defined based on the confidence level range of the model inference, and a corresponding confidence correction coefficient is matched for each interval. The higher the confidence level of a defect, the closer the corresponding confidence correction coefficient is to 1; the lower the confidence level, the lower the corresponding confidence correction coefficient. Defects with a confidence level below a preset acceptable threshold are considered invalid data and are not included in subsequent risk calculations. In this application, for the confidence level data of a single defect, its corresponding confidence interval is matched to obtain the corresponding confidence correction coefficient, which is used to reduce the interference of low-confidence suspected defects on the final risk assessment result.

[0040] Step 204: Perform reverse weighting on the actual physical size data to obtain the size weight coefficients.

[0041] In this embodiment, a reverse correspondence rule between size and weight is pre-defined based on the detection requirements of micro-defects in PCBs. That is, the smaller the actual physical size of the defect, the higher the corresponding size weight coefficient. This amplifies the risk weight of micro-defects, adapting to the high detection requirements of production lines for micron-level micro-defects. In this application, for the actual physical size data of a single defect, the size weight coefficient corresponding to the defect is calculated according to the preset reverse weight assignment rule. Specifically, micro-defects with an actual physical size of less than 20 micrometers are assigned a size weight coefficient significantly higher than that of defects with conventional sizes.

[0042] Step 205: Perform weighted fusion processing on the size weight coefficient, type risk weight coefficient, confidence correction coefficient, and defect distribution density data to obtain the risk quantification value.

[0043] In this application embodiment, according to the focus of production line quality control, corresponding fusion weights are set for the above four types of parameters. In this application, the size weight coefficient, type risk weight coefficient, and confidence correction coefficient corresponding to a single defect are combined with the defect distribution density data of the current field of view and weighted fusion calculation is performed according to the preset fusion weights to obtain the risk quantification value corresponding to a single suspected defect.

[0044] Step 206: Summarize and statistically process the risk quantification values ​​to obtain the defect attention index.

[0045] In this embodiment, the risk quantification values ​​corresponding to all valid suspected defects in the current detection field of view are summarized and accumulated to obtain a defect attention index that can characterize the overall defect risk level and observation value of the current detection field of view, thus completing the entire process of risk quantification.

[0046] This application's embodiments effectively address the technical problems of incomplete single-dimensional judgments and the easy neglect of micron-level minute defects in traditional risk assessments by using multi-dimensional weighted fusion quantification of the actual physical size, distribution density, defect category, and confidence level of defects. It accurately obtains the actual physical size of defects through size conversion, reflects the risk of defect clustering by combining spatial distribution density, distinguishes the degree of harm to product performance by using type risk weight coefficients, reduces the interference of low-confidence false detection data by using confidence level correction coefficients, and emphasizes the risk weight of minute defects by assigning reverse weights, thus adapting to the high detection requirements of precision components for minute defects. The weighted fusion and summary statistics of multi-dimensional parameters enable the defect attention index to comprehensively, objectively, and accurately characterize the overall defect risk of the detection area, providing a scientific and reliable quantitative basis for subsequent risk level assessments, avoiding assessment bias caused by single parameters, and improving the rationality and accuracy of risk assessments.

[0047] In some embodiments, comparing the defect attention index with a preset risk threshold to obtain a risk level assessment result includes: performing a numerical comparison between the defect attention index and the preset risk threshold to obtain comparison difference data.

[0048] Specifically, the preset risk threshold is a critical value determined in advance through testing and calibration of multiple batches of standard PCB samples and defective samples, combined with the acceptable false alarm rate and missed detection rate requirements of the production line. It is used to distinguish the risk level of the inspection area. The defect attention index calculated from the current inspection field of view is compared one-to-one with the preset risk threshold. The difference between the defect attention index and the preset risk threshold is calculated as the comparison difference data. This data can directly characterize the deviation of the risk level of the current inspection area from the critical threshold.

[0049] Furthermore, the difference data is compared and matched with a preset risk range to obtain the initial risk level classification results.

[0050] Specifically, based on the numerical range of the comparison difference data, two corresponding risk level intervals were pre-defined. The interval where the comparison difference data is less than or equal to 0 corresponds to a low-risk safe interval; the interval where the comparison difference data is greater than 0 corresponds to a high-risk defect interval. The comparison difference data obtained in the first step is matched with the two pre-defined risk intervals. When the comparison difference data falls into the low-risk safe interval, it is initially classified as a safe area, corresponding to no valid suspected defects or negligible defect risk within the field of view. When the comparison difference data falls into the high-risk defect interval, it is initially classified as a defect area, corresponding to the existence of suspected defects within the field of view that require verification and confirmation. This yields the initial risk level classification result.

[0051] Furthermore, the confidence data is subjected to threshold filtering to obtain effective defect feature data.

[0052] Specifically, a qualified threshold for defect confidence is pre-set based on the inference performance of the deep learning object detection model to eliminate low-confidence interference data generated by model false detections. The confidence data of all suspected defects within the current detection field of view are compared and filtered one by one with the preset qualified confidence threshold. Invalid suspected defect data with confidence levels below the qualified threshold are eliminated, while the bounding boxes, defect categories, and confidence data of suspected defects with confidence levels greater than or equal to the qualified threshold are retained and integrated into valid defect feature data. This eliminates the interference of low-confidence false detection data on the risk level determination results.

[0053] Furthermore, the consistency verification process is performed between the effective defect feature data and the initial risk level classification results to obtain the verified preliminary risk level assessment results.

[0054] Specifically, the verification logic is as follows: If the initial risk level classification result is a defect area, but no valid defect feature data is found after screening, it indicates that the initial classification is an erroneous judgment caused by low-confidence false detection data, and the initial classification result is corrected to a safe area; if the initial risk level classification result is a safe area, but valid defect feature data exists after screening, it indicates that the initial classification missed valid defect risks, and the initial classification result is corrected to a defect area; if the initial risk level classification result completely matches the existence status of valid defect feature data, the initial classification result is retained. After the above consistency verification and correction, the verified preliminary risk level assessment result is obtained.

[0055] Furthermore, risk weight adaptation processing is performed on the defect category data to obtain the level correction parameters.

[0056] Specifically, risk weights are pre-assigned to five typical PCB defects—solder bridging, tombstoning, insufficient solder, misalignment, and solder balls—based on their impact on product electrical performance and safety. The defect type with the greater impact on product performance receives a higher risk weight. Based on defect category data from the valid defect feature data, corresponding defect risk weights are matched, and the proportion of high-risk defects within the current inspection field of view is statistically analyzed. A grade correction parameter is calculated based on this proportion; the higher the proportion of high-risk defects, the larger the grade correction parameter value, which amplifies the impact weight of high-priority defects on the final risk grade.

[0057] Furthermore, the initial risk level assessment results after verification are calibrated and fused with the level correction parameters to obtain the risk level assessment results.

[0058] Specifically, for areas where the initial risk level assessment result after verification indicates a defect, the risk priority is further calibrated using the risk level correction parameters to determine whether it is a routine defect area or a high-priority re-inspection defect area. For areas where the initial risk level assessment result after verification indicates a safe area, a final review is completed using the risk level correction parameters. Once it is confirmed that there are no high-risk defects, it is ultimately determined to be a safe area. The final output risk level assessment result clearly indicates the risk level and risk priority of the current detection area, providing a basis for subsequent parameter adjustment and graded detection of the detection system.

[0059] This application's embodiments effectively address the problems of misjudgment, missed judgment, and mismatch between judgment results and actual defect hazards in traditional risk level determination through a multi-level processing flow including numerical comparison, interval division, confidence level screening, consistency verification, and defect category correction. Preliminary risk level division is achieved by comparing the defect attention index with a preset risk threshold; low-confidence invalid false detection data is eliminated through confidence level screening to reduce interference; consistency verification corrects the initial division deviation, avoiding situations where "no valid defects are judged as high-risk" or "valid defects are judged as low-risk"; and defect category risk weight adaptation amplifies the impact of high-hazard defects such as tin-stretching and tombstoning on risk levels. The calibrated and fused risk level assessment results accurately match the actual hazard level of defects, providing precise judgment basis for subsequent testing parameter adjustment and graded testing, ensuring the scientific and rational nature of testing decisions, and further reducing quality control risks caused by misjudgment and missed detection.

[0060] In some embodiments, the scanning motion parameters, imaging acquisition and illumination control parameters are adjusted based on the risk level assessment results to obtain detection control parameters, including: performing hierarchical analysis and strategy matching processing on the risk level assessment results to obtain a parameter adjustment benchmark strategy.

[0061] Specifically, the defect risk level assessment results are first analyzed to determine whether the current detection field of view corresponds to a low-risk safe area or a high-risk defect area. Then, the analyzed risk levels are matched one by one with pre-set parameter adjustment strategies. The low-risk safe area is matched with a baseline strategy that maintains high-speed cruising and does not change the basic acquisition and illumination parameters, with the core objective of ensuring detection efficiency. The high-risk defect area is matched with a closed-loop adjustment baseline strategy that uses deceleration and image stabilization, high-precision multi-layer imaging, and scene-adaptive illumination, with the core objective of ensuring the imaging quality and recognition accuracy of defect verification. After matching, the parameter adjustment baseline strategy corresponding to the current risk level is obtained.

[0062] Furthermore, the parameter adjustment baseline strategy and the current operating parameters are fused and planned to obtain the scanning motion adjustment parameters.

[0063] Specifically, the current operating parameters are the current scanning speed, direction of motion, and acceleration / deceleration curve parameters of the motorized stage. For low-risk safe areas, the parameter adjustment baseline strategy is integrated with the current operating parameters to directly generate scanning motion adjustment parameters that maintain the current high-speed cruising state. For high-risk defect areas, the parameter adjustment baseline strategy is combined with the current high-speed operating parameters to perform smooth deceleration planning, generating adjustment data that includes target low speed, smooth acceleration / deceleration curves, and fine-tuning parameters for the direction of motion. In extreme cases, reverse backtracking control parameters for defect repositioning are supplemented to ensure that the defect area can accurately and smoothly stop at the center of the field of view, avoiding motion blur caused by sudden acceleration / deceleration. Finally, scanning motion adjustment parameters that can be directly sent to the motion controller for execution are obtained.

[0064] Furthermore, the parameter adjustment baseline strategy is adapted to the operating state to obtain the imaging acquisition adjustment parameters.

[0065] Specifically, the operating status includes the current acquisition frame rate, exposure time, focus status, and depth-of-field range parameters of the industrial camera and microscope head. For low-risk safe areas, the parameter adjustment baseline strategy is adapted to the current operating status to generate imaging acquisition adjustment parameters that maintain the current normal acquisition frame rate and single-frame imaging mode. For high-risk defect areas, combined with the parameter adjustment baseline strategy, and based on the current focus status and the depth-of-field range of the microscope head, multi-level focal length micro-adjustment parameters, the number of continuous shots in the defect area, and optimized exposure time and gain parameters are generated to ensure that multiple clear images of different focal planes in the defect area can be acquired, providing a qualified data source for subsequent depth-of-field fusion, and ultimately obtaining imaging acquisition adjustment parameters adapted to the defect verification requirements.

[0066] Furthermore, the defect category data is processed by scene attribute matching to obtain lighting control adjustment parameters.

[0067] Specifically, based on the material properties of different areas of the PCB and the imaging requirements of different defect categories, corresponding lighting modes and parameters are pre-matched for each type of defect and detection area. Based on defect category data, the material and surface reflectivity of the detection area corresponding to the current defect are identified, and corresponding lighting control parameters are matched: when solder bridging or offset defects are detected in highly reflective areas such as solder pads and gold fingers, brightness and angle adjustment parameters for coaxial light are matched; when character defects or surface foreign matter defects are detected in silkscreen areas, zoned brightness adjustment parameters for ring light are matched; when substrate cracks or hole wall defects are detected in via and through-hole areas, on-state and brightness adjustment parameters for bottom backlight are matched, ultimately obtaining lighting control adjustment parameters adapted to the current defect detection scenario.

[0068] Furthermore, the scanning motion adjustment parameters, imaging acquisition adjustment parameters, and illumination control adjustment parameters are subjected to spatiotemporal alignment and time delay compensation calibration to obtain multi-parameter linkage control data.

[0069] Specifically, firstly, the acquisition timestamps corresponding to the image frames where defects are detected are recorded, and the system's image transmission time, model inference time, and command communication time are statistically analyzed to obtain the total system processing delay. Combined with the current operating speed of the stage, the predicted time and predicted position of the defect area reaching the center of the field of view are calculated. Then, the execution time of the scanning motion adjustment parameters, the trigger time of the imaging acquisition adjustment parameters, and the switching time of the illumination control adjustment parameters are spatiotemporally aligned to compensate for the positional deviation caused by the total system processing delay. This ensures that the three actions of stage deceleration, illumination mode switching, and camera multi-frame acquisition are executed synchronously at the same moment when the defect area reaches the center of the field of view, avoiding imaging blurring and defect out-of-frame problems caused by timing misalignment. Finally, multi-parameter linkage control data with unified timing and matching position are obtained.

[0070] Furthermore, compliance verification processing is performed on the multi-parameter linkage control data to obtain the detection control parameters.

[0071] Specifically, safe operating parameter ranges are pre-defined based on the equipment hardware specifications, including the maximum acceleration / deceleration threshold of the stage, the minimum operating speed, the maximum brightness threshold of the lighting unit, and the safe exposure time range of the camera. Each parameter in the multi-parameter linkage control data is individually verified to ensure it is within the equipment's permissible safe operating range. Simultaneously, it is checked for execution timing conflicts and action logic conflicts between multiple sets of parameters. Invalid parameters that exceed the range or conflict are eliminated, and non-compliant parameters are corrected for compliance. After successful verification, the multi-parameter linkage control data is encapsulated into a standardized instruction format recognizable by the motion controller, imaging unit, and lighting unit, ultimately yielding directly executable detection and control parameters.

[0072] This application's embodiments effectively address the shortcomings of traditional detection methods, such as fixed parameters that cannot be dynamically adjusted based on defect risk, leading to poor imaging quality, inaccurate motion control, and poor scene adaptability. By adaptively adjusting three core parameters—scanning motion, imaging acquisition, and illumination control—based on risk level assessment results, this approach achieves targeted parameter adjustment. The scanning motion parameters are fused with current operating parameters to avoid imaging blurring caused by sudden acceleration or deceleration, ensuring the stability and accuracy of the stage movement. Imaging acquisition parameters are adapted to the equipment's operating status to ensure imaging quality meets the detection needs of different risk areas. Illumination control parameters are matched with defect categories and detection scene attributes to address insufficient imaging contrast for different defects (such as bridging in highly reflective areas and tiny solder beads). Spatiotemporal alignment and delay compensation calibration prevent misalignment of multi-parameter execution timing, while compliance verification ensures control commands conform to equipment hardware specifications. The resulting detection control parameters achieve coordinated operation of motion, imaging, and illumination, improving the scene adaptability and control accuracy of the detection equipment and providing reliable parameter support for the smooth implementation of subsequent graded detection.

[0073] In some embodiments, a real-time video stream is input into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence data, including: frame extraction, format standardization, and image preprocessing of the real-time video stream to obtain the image data to be inferred.

[0074] Specifically, real-time video streams are acquired with low latency using shared memory technology, and the video streams are extracted frame by frame to avoid frame loss and misalignment during scanning. The extracted single-frame raw images are standardized, converting the raw Bayer format images output by the industrial camera into the RGB format supported by the preset target detection model, while uniformly scaling the image resolution to the model's preset 640×640 input specification. The standardized images are then preprocessed, including Gaussian filtering for noise reduction, adaptive contrast enhancement, and pixel value normalization, while simultaneously suppressing noise interference in highly reflective areas of the PCB pads. Finally, image data for inference that conforms to the model input specifications and is free of redundant interference is obtained.

[0075] Furthermore, the image data to be inferred is input into a preset target detection model for multi-scale feature extraction and defect feature recognition processing to obtain initial defect detection result data.

[0076] Specifically, the preset target detection model is a lightweight YOLO11s detection model accelerated and optimized by TensorRT. The model has a built-in multi-scale feature extraction backbone network adapted for the detection of small defects in PCBs and a dedicated detection head for small targets, meeting the real-time detection requirements of the production line. The image data to be inferred is input into the preset target detection model. The backbone network completes the multi-scale depth feature extraction of the image, and then the neck network completes feature fusion and enhancement. Finally, the detection head outputs the initial detection results of all potential defects in the current image frame, including the detection box coordinate offset, the probability distribution of various defect categories, and the target confidence score, which are integrated to obtain the initial defect detection result data.

[0077] Furthermore, nonmaximum suppression and redundant detection box removal are performed on the initial defect detection result data to obtain a defect candidate box dataset.

[0078] Specifically, to address the issue of multiple overlapping detection boxes corresponding to the same defect in the initial defect detection results data, the DIoU (Distance Intersection over Union) non-maximum suppression algorithm is used for deduplication. The DIoU threshold is preset to 0.5. First, all initial detection boxes are sorted from high to low confidence scores. The detection box with the highest confidence score is retained, and overlapping redundant boxes with an DIoU exceeding the preset threshold are removed. This process is repeated until all detection boxes are deduplicated. Invalid detection boxes that exceed the image boundary and abnormal detection boxes whose size exceeds the preset reasonable range for PCB solder joint defects are also removed. Finally, the deduplicated dataset of valid defect candidate boxes is obtained.

[0079] Furthermore, the defect candidate box dataset is subjected to category classification and confidence level calibration to obtain defect category data and confidence level data.

[0080] Specifically, for each candidate box in the defect candidate box dataset, the probability distribution of its corresponding defect categories is extracted. The defect category with the highest probability value is taken as the final category of the candidate box, thus completing the category classification determination and clearly distinguishing between normal solder joints and five types of defects: bridging, tombstoning, insufficient solder, offset, and solder balls. For the confidence deviation of different defect categories, the confidence values ​​are calibrated using pre-calibrated category calibration coefficients. For small solder balls and microcracks that are prone to low confidence, positive calibration is performed to avoid small defects being mistakenly filtered out. Finally, the defect category data and confidence data corresponding to each valid candidate box are obtained.

[0081] Furthermore, pixel coordinate extraction and boundary range normalization are performed on the defect candidate box dataset to obtain bounding box data.

[0082] Specifically, for each valid candidate box in the defect candidate box dataset, four core coordinate values ​​are extracted in the image pixel coordinate system: the top-left x-coordinate, the top-left y-coordinate, the bottom-right x-coordinate, and the bottom-right y-coordinate, to clarify the boundary range of the defect in the image. The extracted coordinate values ​​are then subjected to boundary range normalization processing, including integer normalization to eliminate subsequent calculation errors caused by decimal coordinates, cropping and correcting coordinates that exceed the effective boundary of the image, and appropriately expanding the boundary of tiny defect boxes with a size of less than 20 micrometers to ensure that the box completely covers the defect area. Finally, the normalized bounding box data corresponding to each defect is obtained.

[0083] Furthermore, compliance verification and invalid data removal are performed on the bounding box data, defect category data, and confidence level data to obtain the bounding boxes, defect categories, and confidence level data of suspected defects.

[0084] Specifically, firstly, the integrity of each data set is verified to ensure that the bounding box data, defect category data, and confidence score data correspond one-to-one and that no fields are missing. Secondly, the data is verified for compliance, and abnormal data sets with bounding box size of 0, invalid defect categories, and confidence scores of 0 are removed. Finally, based on the minimum detection threshold preset by the production line, low-confidence interference data with a confidence score below 0.3 are removed to eliminate invalid data interference caused by model false detections. After verification and removal, the bounding boxes, defect categories, and confidence scores corresponding to all valid suspected defects in the current image frame are finally obtained, providing an accurate data source for subsequent risk quantification processing.

[0085] This application's embodiments effectively address the technical problems of excessive image interference, redundant detection boxes, inaccurate defect identification, and non-standard results in traditional model inference through a full-process processing including image preprocessing, multi-scale feature extraction, redundant detection box removal, confidence calibration, and data compliance verification. Frame extraction and format standardization ensure the standardization of real-time video stream data; image preprocessing (denoising and contrast enhancement) effectively suppresses interference such as high reflectivity of PCB pads and environmental noise, improving image quality; multi-scale feature extraction accurately captures the features of tiny defects of different sizes (especially those within 20 micrometers in diameter), avoiding missed detections of minute defects; non-maximum suppression algorithms remove redundant detection boxes, reducing invalid data; category classification and confidence calibration ensure accurate defect category identification and reliable confidence data, avoiding misclassification; bounding box regularization and compliance verification standardize the detection result format and remove invalid data. The resulting suspected defect bounding boxes, defect categories, and confidence data provide accurate and effective data sources for subsequent risk quantification, improving the accuracy and stability of the entire defect detection process and laying the foundation for the smooth implementation of subsequent steps.

[0086] In some embodiments, the current detection area is subjected to graded detection processing based on detection control parameters to obtain defect detection data, including: graded determination of risk level assessment results and matching processing with execution strategies to obtain a graded detection execution plan.

[0087] Specifically, the risk level assessment results are first clearly classified and determined to accurately distinguish whether the current detection field of view is a low-risk safe area or a high-risk defect area. Then, the determined risk level is matched one by one with the pre-set graded detection execution strategy. The low-risk safe area is matched with a continuous cruise detection scheme with detection efficiency as the core. The core of the scheme is to maintain a high-speed scanning state, continuously perform real-time inference, and not store defect-free redundant image data. The high-risk defect area is matched with a high-precision verification detection scheme with detection accuracy as the core. The core of the scheme is to reduce the speed and stabilize the image, optimize multi-dimensional imaging, perform secondary defect verification, and store the full data in a structured manner. After the matching is completed, a graded detection execution scheme that is fully adapted to the risk level of the current detection area is obtained.

[0088] Furthermore, the hierarchical detection execution scheme is adapted to the basic operating parameters to obtain continuous detection control instructions.

[0089] Specifically, the basic operating parameters are the high-speed cruise baseline parameters pre-set by the detection system, including the standard scanning speed of the motorized stage, the conventional acquisition frame rate of the industrial camera, the baseline focusing parameters of the microscope lens, and the basic illumination mode of the illumination unit. For the low-risk safety zone where the graded detection execution scheme is a continuous cruise detection scheme, the execution scheme is adapted and integrated with the basic operating parameters without changing or adjusting the basic operating parameters. This generates continuous detection control commands that can be directly issued and executed. The commands maintain the high-speed scanning state of the stage, continuously perform image acquisition and model inference, and do not trigger verification acquisition and redundant data storage operations, ensuring that the production line detection cycle is not affected.

[0090] Furthermore, the graded detection execution plan is correlated and adapted with defect categories and confidence level data to obtain high-precision verification control instructions.

[0091] Specifically, for high-risk defect areas where the graded inspection execution plan is a high-precision verification inspection plan, differentiated adaptation is performed by combining defect category data and confidence level data: For high-risk defect categories such as tin bonding and tombstoning, which have a significant impact on the electrical performance of the product, verification control parameters with multi-level focal length acquisition, high-contrast lighting enhancement, and multi-frame continuous shooting are adapted and generated; for suspected defects with low confidence levels, verification control parameters with exposure optimization and multi-mode lighting switching are adapted and generated to improve defect imaging contrast and reduce the probability of false detection; for tiny defects with a size of less than 20 micrometers, verification control parameters with fine-grained focal plane micro-motion and ultra-depth-of-field acquisition are adapted and generated, ultimately obtaining a high-precision verification control command that perfectly matches the current defect attributes.

[0092] Furthermore, the high-precision verification control commands and detection control parameters are fused and analyzed to obtain the linkage execution commands.

[0093] Specifically, the detection control parameters are multi-parameter linkage data including scanning motion adjustment parameters, imaging acquisition adjustment parameters, and lighting control adjustment parameters. High-precision verification control commands are deeply integrated and analyzed with the detection control parameters to unify and align the execution timing and logic of stage motion control, lighting mode switching, and camera acquisition triggering. This compensates for time delay deviations caused by system transmission and inference, ensuring complete synchronization of the execution timing of the three types of actions. Simultaneously, it verifies that the execution logic of the commands is conflict-free. Finally, it generates linkage execution commands that can be synchronously recognized and executed by the motion controller, industrial camera, and lighting unit.

[0094] Furthermore, based on the linkage execution instructions, multi-focal-length image acquisition, depth-of-field fusion, and secondary defect verification and reasoning processing are performed on the sample under test to obtain a clear full-depth image and defect confirmation data.

[0095] Specifically, following the linkage execution command, the electric stage is first controlled to smoothly decelerate to a preset low speed when the defect area reaches the center of the field of view. Simultaneously, the Z-axis focal length adjustment mechanism is controlled to perform multi-level fine micro-motion above and below the reference focal plane. The industrial camera is controlled to acquire corresponding images at each focal length level, resulting in a multi-focal length image sequence of different focal planes in the defect area. A wavelet transform depth-of-field fusion algorithm is used to fuse the multi-focal length image sequence, extracting the clear areas from each frame and integrating them to obtain a full-view depth-of-field clear image covering the entire defect area, solving the problem of blurred imaging of small defects caused by insufficient depth of field of the microscope lens. The full-view depth-of-field clear image is then input into a preset target detection model for secondary defect verification and inference processing to accurately confirm the final category, actual physical size, confidence level, and location information of the defect. False detection data is simultaneously eliminated, and finally, the verified defect confirmation data is obtained.

[0096] Furthermore, the defect confirmation data, acquisition location data, acquisition time data, and panoramic depth-of-view clear images are correlated, bound, and structured for verification to obtain defect detection data.

[0097] Specifically, a unique identifier ID is generated for each defect. Defect confirmation data, corresponding acquisition location data, acquisition time data, and a clear panoramic image are linked and bound together using this unique identifier ID to ensure traceability and location of the defect data. All linked data undergoes structured validation, including field integrity checks, data compliance checks, and location and time consistency checks, eliminating invalid data with missing fields or data anomalies. The validated data is then standardized according to the interface specifications of the production line's MES (Manufacturing Execution System), packaged into a structured dataset, and encrypted for storage, ultimately yielding compliant, complete, and traceable defect detection data. After the defect detection data is generated and stored, the system automatically restores to its basic operating parameters and continues scanning and detecting the remaining areas of the PCB board under test until the entire inspection process is completed.

[0098] This application's embodiments effectively address the problems of low efficiency, blurred imaging of minor defects, insufficient defect confirmation accuracy, and untraceable inspection data caused by traditional "indiscriminate full-area inspection" by employing a graded inspection strategy combined with multi-focal imaging, depth-of-field fusion, and secondary verification reasoning. By matching differentiated inspection schemes to risk level determination, low-risk areas maintain high-speed continuous inspection, ensuring production line inspection cycle time; high-risk areas undergo high-precision verification, using multi-focal image acquisition and depth-of-field fusion to solve the problem of blurred defect imaging caused by insufficient depth of field of the microscope lens, obtaining clear images of defects with full depth of field; secondary defect verification reasoning further confirms defect attributes, reducing false positives and false negatives; by associating defect confirmation data, acquisition location, acquisition time, and full depth-of-field images, combined with structured verification, the inspection data is ensured to be complete, standardized, and traceable, meeting the needs of production line quality control and data management. Simultaneously, the structured data can be directly connected to the MES system, improving the level of intelligent management of the production line, reducing manual data processing costs, and providing reliable data support for product quality traceability and inspection process optimization.

[0099] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0100] like Figure 3 As shown in the illustration, this application also provides a defect detection device 300. The device includes: The processing module 301 is used to perform image acquisition and processing on the sample area of ​​the sample to be tested to obtain a real-time video stream.

[0101] The processing module 301 is also used to input the real-time video stream into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category and the confidence data.

[0102] The processing module 301 is also used to perform risk quantification processing on the bounding box, defect category and confidence data of suspected defects to obtain the defect attention index.

[0103] The assessment module 302 is used to compare the defect attention index with the preset risk threshold to obtain the risk level assessment result.

[0104] The processing module 301 is also used to adjust the scanning motion parameters, imaging acquisition and illumination control parameters based on the risk level assessment results to obtain the detection control parameters.

[0105] The grading module 303 is used to perform grading detection processing on the current detection area based on the detection control parameters to obtain defect detection data.

[0106] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0107] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0108] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0109] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0110] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.

[0111] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0112] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0113] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0114] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0115] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A defect detection method, characterized in that, include: Image acquisition and processing are performed on the sample area to be tested to obtain a real-time video stream; The real-time video stream is input into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence data. The bounding box, defect category, and confidence level data of the suspected defects are subjected to risk quantification processing to obtain the defect attention index; The defect attention index is compared with a preset risk threshold to obtain a risk level assessment result. Based on the risk level assessment results, the scanning motion parameters, imaging acquisition and illumination control parameters are adjusted to obtain the detection control parameters; Based on the aforementioned detection control parameters, the current detection area is subjected to graded detection processing to obtain defect detection data.

2. The method according to claim 1, characterized in that, The risk quantification process is performed on the bounding box, defect category, and confidence data of the suspected defects to obtain a defect attention index, including: The bounding box of the suspected defect is subjected to size conversion and spatial distribution statistical processing to obtain actual physical size data and defect distribution density data; The defect categories are subjected to a preset risk level matching process to obtain the type risk weight coefficient; The confidence data is subjected to pre-set confidence interval matching processing to obtain the confidence correction coefficient; The actual physical size data is subjected to reverse weighting to obtain size weight coefficients; The size weighting coefficient, the type risk weighting coefficient, the confidence correction coefficient, and the defect distribution density data are weighted and fused to obtain a risk quantification value. The risk quantification values ​​are summarized and statistically processed to obtain the defect attention index.

3. The method according to claim 1, characterized in that, The process of comparing the defect attention index with a preset risk threshold to obtain a risk level assessment result includes: The defect attention index is compared with a preset risk threshold to obtain the comparison difference data. The comparison difference data is subjected to preset risk interval matching processing to obtain the initial risk level classification result; The confidence data is subjected to threshold filtering to obtain effective defect feature data; The valid defect feature data and the initial risk level classification result are subjected to consistency verification processing to obtain the verified initial risk level assessment result. The defect category data is subjected to risk weight adaptation processing to obtain the level correction parameters; The initial risk level assessment result after verification is calibrated and fused with the level correction parameters to obtain the risk level assessment result.

4. The method according to claim 1, characterized in that, The process of adjusting the scanning motion parameters, imaging acquisition and illumination control parameters based on the risk level assessment results yields the detection control parameters, including: The risk level assessment results are subjected to hierarchical analysis and strategy matching to obtain the parameter adjustment benchmark strategy; The parameter adjustment baseline strategy and the current operating parameters are fused and planned to obtain the scanning motion adjustment parameters; The parameter adjustment benchmark strategy is adapted to the operating state to obtain the imaging acquisition adjustment parameters; The defect category data is subjected to scene attribute matching processing to obtain lighting control adjustment parameters; The scanning motion adjustment parameters, the imaging acquisition adjustment parameters, and the illumination control adjustment parameters are subjected to spatiotemporal alignment and time delay compensation calibration to obtain multi-parameter linkage control data. The multi-parameter linkage control data is subjected to compliance verification processing to obtain the detection control parameters.

5. The method according to claim 1, characterized in that, The step of inputting the real-time video stream into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category, and the confidence score data includes: The real-time video stream is subjected to frame extraction, format standardization, and image preprocessing to obtain the image data to be inferred; The image data to be inferred is input into a preset target detection model for multi-scale feature extraction and defect feature recognition to obtain initial defect detection result data. The initial defect detection result data is subjected to non-maximum suppression and redundant detection box removal to obtain a defect candidate box dataset. The defect candidate box dataset is subjected to category classification and confidence level calibration to obtain defect category data and confidence level data; The defect candidate box dataset is subjected to pixel coordinate extraction and boundary range normalization to obtain bounding box data; The bounding box data, defect category data, and confidence score data are subjected to compliance verification and invalid data removal to obtain the bounding box, defect category, and confidence score data of suspected defects.

6. The method according to claim 1, characterized in that, The step of performing graded detection processing on the current detection area based on the detection control parameters to obtain defect detection data includes: The risk level assessment results are graded and matched with execution strategies to obtain a graded detection execution plan; The hierarchical detection execution scheme is adapted to the basic operating parameters to obtain continuous detection control instructions; The graded detection execution scheme is associated and adapted with the defect category and the confidence data to obtain a high-precision review control instruction. The high-precision verification control command and the detection control parameters are fused and analyzed to obtain the linkage execution command; Based on the linked execution command, the sample to be tested is subjected to multi-focal length image acquisition, depth-of-field fusion and secondary defect verification reasoning to obtain a clear full-depth-of-field image and defect confirmation data. The defect confirmation data, acquisition location data, acquisition time data, and the full-view depth-of-field clear image are associated, bound, and structured for verification to obtain defect detection data.

7. A defect detection device, characterized in that, Applied to the method according to any one of claims 1 to 6, the apparatus comprises: The processing module is used to acquire and process images of the sample area to be tested, and obtain a real-time video stream. The processing module is also used to input the real-time video stream into a preset target detection model for inference processing to obtain the bounding box of the suspected defect, the defect category and the confidence data; The processing module is also used to perform risk quantification processing on the bounding box, defect category and confidence data of the suspected defect to obtain the defect attention index. The assessment module is used to compare the defect attention index with a preset risk threshold to obtain a risk level assessment result. The processing module is also used to adjust the scanning motion parameters, imaging acquisition and illumination control parameters based on the risk level assessment results to obtain detection control parameters; The grading module is used to perform grading detection processing on the current detection area based on the detection control parameters to obtain defect detection data.

8. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images