YOLO improvement-based two-stage automatic probe card print detection method

Abstract

The invention relates to the technical field of image processing, and provides a two-stage probe card pin print automatic detection method based on YOLO improvement, and the method comprises the steps: constructing a welding spot sample data set, and training a pin print detection model based on a YOLOv8 frame; obtaining a welding spot positioning result based on the trained needle print detection model; based on a welding spot positioning result, extracting potential needle print features by adopting an image processing method; and based on the potential needle print features, recognizing and positioning the needle print defects by adopting a geometrical shape feature judgment method. According to the method, a grading detection strategy is adopted, the recall rate and the positioning precision of needle print recognition can be remarkably improved, and high-precision and robust detection under the complex background is achieved. In addition, the method has good expandability and can be migrated and applied to automatic detection tasks of other types of tiny surface features or defects.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an automatic detection method for two-stage probe card imprints based on YOLO. Background Technology

[0002] Chip pin marks are a common type of tiny surface feature in the manufacturing process, typically generated by bed-of-nails testing or mechanical contact processes. While pin marks do not directly affect product functionality in most cases, their distribution, size, and morphology can reflect stress conditions and equipment status during production, serving as important reference indicators for process control, equipment condition monitoring, and quality traceability. By detecting and recording pin marks, potential process deviations or equipment anomalies can be identified early, indirectly improving the long-term reliability of electronic products and providing data support for quality traceability and process control.

[0003] However, pin mark detection still faces many technical challenges. First, from a spatial perspective, the diameter of a pin mark is typically only 5% to 20% of the solder joint diameter, corresponding to only a very small number of pixels in the overall board image, making it easily overwhelmed by imaging noise. Second, from a visual perspective, the grayscale difference between the pin mark and the surrounding solder surface is weak, and its surface reflectivity easily creates highlight areas in the image, making the gradient distribution between the background and the target unstable, thus increasing the difficulty of boundary feature extraction. In addition, pin mark morphology is highly diverse, ranging from regular circular indentations to irregular depressions, with their boundaries often being smooth transitions and lacking significant high-frequency details. Finally, the PCB surface has complex background interference such as silkscreen characters, solder pad reflections, and flux residues. These non-target features easily produce similar responses to pin marks, further increasing the probability of false detection.

[0004] Existing pinprint detection methods mainly fall into three categories. The first category is manual detection methods, relying on visual judgment by inspectors under high-powered microscopes or magnified images. While offering high flexibility, these methods suffer from low efficiency, depend on subjective experience, and struggle to guarantee consistency and repeatability in mass production. The second category comprises automated detection methods based on traditional image processing, such as threshold segmentation, edge detection, morphological operations, and template matching. These methods are low-cost and easy to deploy, but are highly sensitive to changes in imaging lighting, background texture, and target scale, resulting in insufficient generalization ability across different production environments. The third category consists of detection methods based on single-stage deep learning, such as Faster R-CNN, which enables end-to-end automated pinprint recognition. However, because pinprints are small targets with weak features, direct detection in the entire image is easily affected by complex backgrounds, leading to decreased recall and localization accuracy.

[0005] In summary, existing technologies still have the following shortcomings in pinprint detection: (1) small target details are easily lost during downsampling, resulting in missed detections; (2) the global detection strategy has insufficient localization accuracy, and the bounding box is prone to containing redundant background; (3) the model has limited generalization ability under different board types and lighting conditions, and the detection performance is unstable. Therefore, this invention proposes a two-stage probe card pinprint automatic detection method based on YOLO improvement to achieve automated detection of chip pinprint defects. Summary of the Invention

[0006] In view of this, the present invention proposes an automatic detection method for probe card imprints based on a two-stage improved YOLO framework. By combining the efficiency of the YOLOv8 framework with a two-stage detection strategy, it achieves good task portability and application expansion potential, thereby solving the problems existing in the prior art.

[0007] To achieve the above objectives, this invention proposes an automatic detection method for two-stage probe card imprints based on YOLO improvement, characterized by the following steps: Construct a solder joint sample dataset and train a pin mark detection model based on the YOLOv8 framework; The solder joint location results are obtained based on the trained needle mark detection model. Based on the solder joint location results, potential pin mark features are extracted using image processing methods; Based on the potential needle mark characteristics, a geometric shape feature determination method is used to identify and locate needle mark defects.

[0008] Furthermore, the solder joint sample dataset includes high-resolution chip images with different light intensities, contrast levels, resolutions, and surface contamination conditions.

[0009] Furthermore, the YOLOv8 framework includes a Backbone, a Neck, and a Head. The Backbone uses an improved C2f module for multi-scale feature extraction. The Neck uses a combination of a feature pyramid network and a path aggregation network for multi-scale feature fusion. The Head uses an Anchor-Free prediction mechanism to directly regress the target center point and bounding box parameters, and outputs the class probability and target confidence.

[0010] Furthermore, during the training process of the needle print detection model, data augmentation strategies such as Mosaic splicing, random flipping, color jittering, and scale scaling are employed, and the learning rate is dynamically adjusted in conjunction with a cosine annealing strategy.

[0011] Furthermore, the process of extracting latent needleprint features using image processing methods includes: A fixed threshold segmentation method is used to separate the foreground and background of the solder joint region in the solder joint location result to obtain a binarized image. Connectivity analysis is used to label and extract features from all independent regions in the binarized image.

[0012] Furthermore, the process of identifying and locating needle mark defects using geometric shape feature determination methods includes: A roundness threshold is set, and the roundness value of the potential needle mark feature is calculated. When the calculated roundness value exceeds the roundness threshold, it is determined to be a needle mark defect.

[0013] Furthermore, the method for calculating the roundness value is as follows:

[0014] Where C represents the roundness value, A is the area of ​​the connected region, and P is the perimeter.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes an image-based automatic pinprint recognition method. The first stage employs the YOLOv8 framework, combined with an improved C2f module, feature pyramid network, path aggregation network, and an anchor-free prediction mechanism, to achieve high-precision localization of solder joint regions in a full-board chip image. This effectively suppresses interference from complex backgrounds and provides high-quality regions of interest (ROIs) for subsequent detection. The second stage, within the solder joint ROI, accurately identifies and locates pinprint defects through fixed-threshold segmentation, connected component analysis, and roundness index filtering. This method preserves detailed information of small targets, enhances defect boundary characterization, and significantly improves the recall and localization accuracy of pinprint recognition.

[0016] This invention constructs a sample dataset of solder joints containing various shooting conditions and employs data augmentation strategies such as Mosaic stitching, random flipping, color jittering, and scale scaling. The generalization ability of the model is significantly enhanced under different lighting intensities, contrast levels, resolutions, and surface contamination conditions.

[0017] This invention combines the efficiency of the YOLOv8 framework with a two-stage detection strategy to ensure that the detection process maintains high accuracy while still achieving millisecond-level inference speed, making it suitable for real-time detection needs in large-scale production environments. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1This is a schematic diagram of the overall process of the two-stage probe card imprint automatic detection method based on YOLO of the present invention; Figure 2 This is a schematic diagram of a chip image in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the precise identification and positioning of the solder joint area in an embodiment of the present invention; Figure 4 This is a schematic diagram of needleprint candidate region screening in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] This embodiment proposes an automatic detection method for probe card imprints based on a two-stage YOLO-improved approach, such as... Figure 1 As shown, the first stage employs a YOLOv8-based deep learning object detection network to perform multi-scale solder joint region detection and precise localization on the entire chip board image, suppressing background interference at the source and providing high-quality regions of interest (ROIs) for subsequent processing. The second stage, targeting the extracted solder joint ROIs, introduces a refined detection process based on image binarization segmentation, connected component labeling, and feature extraction. Roundness index filtering enables accurate identification of small circular defects such as pin marks, effectively preserving the detailed features of small targets and enhancing the ability to delineate defect boundaries. This method not only outperforms existing solutions in terms of detection accuracy and robustness but also possesses good task transferability and application expansion potential.

[0021] (1) Solder joint positioning based on YOLOv8 The core objective of this stage is to quickly and accurately locate all solder joint areas in the entire chip board image, providing high-quality ROI input for subsequent fine-grained defect detection. Considering the complex structure of the chip board and the large number and dense distribution of solder joints, this embodiment selects the latest generation YOLOv8 as the detection framework. While maintaining the high efficiency of single-stage detection, YOLOv8 introduces an improved feature extraction network and an adaptive anchor box mechanism, demonstrating significant advantages in multi-scale target detection accuracy, inference speed, and generalization ability, effectively meeting the dual requirements of detection accuracy and real-time performance in this task.

[0022] YOLOv8 mainly consists of three parts: Backbone, Neck, and Head. The Backbone is responsible for multi-scale feature extraction, employing an improved C2f module to reduce the network's computational burden while ensuring smooth gradient flow. The Neck uses a combination of feature pyramid and path aggregation networks to achieve top-down and bottom-up multi-scale feature fusion, enhancing the ability to simultaneously perceive small and large targets. The Head uses an anchor-free prediction mechanism to directly regress the target center point and bounding box parameters, outputting class probabilities and target confidence scores, thus avoiding the computational and hyperparameter sensitivity issues caused by anchor-box matching.

[0023] During the training phase, this embodiment first constructs a dedicated dataset containing solder joint samples under various shooting conditions, covering variations in lighting intensity, contrast levels, resolution, and surface contamination to enhance the model's generalization ability. This dataset contains 580 high-resolution chip images, with approximately 100,000 pin-print defect samples, ensuring the model's robustness in diverse scenarios. The chip images include... Figure 2 As shown.

[0024] For data augmentation, strategies such as Mosaic stitching, random flipping, color dithering, and scale scaling were combined to further enhance the model's adaptability to different scenes and noise levels. The final trained YOLOv8 model can provide high-precision solder joint localization results while maintaining inference speed, providing reliable ROI input for subsequent fine-grained needle mark detection.

[0025] Model training is based on the official YOLOv8 framework, employing the AdamW optimizer with an initial learning rate of 1×10⁻⁴. A cosine annealing strategy is used to dynamically adjust the learning rate to accelerate convergence and prevent overfitting. The number of training epochs is set to 500, and the batch size is 4. The loss function uses the default composite form of YOLOv8.

[0026] in, Used for bounding box regression Used for classification and target confidence prediction. Discrete distribution used to refine the regression bounding box.

[0027] The trained YOLOv8 detection model is used for accurate identification and localization of solder joint areas, such as... Figure 3 As shown, the experimental results demonstrate that the model in this stage can achieve high-precision positioning of dense, small-sized solder joints while maintaining millisecond-level inference speed, providing a reliable guarantee for fine defect detection in the second stage.

[0028] (2) Needleprint detection The system further employs traditional image processing methods to accurately identify and locate pin marks in the solder joint ROI region provided by the YOLO model in the first stage. The main task of this stage is to extract potential pin mark features from the high-precision solder joint region and effectively distinguish them from normal solder joints, thereby achieving high-reliability defect detection.

[0029] The specific process is as follows: First, for the solder joint ROI obtained from the first stage of cropping, a fixed threshold segmentation method is used to separate the foreground and background of the solder joint region, resulting in a binarized image. Next, connected component analysis is used to label and extract features from all independent regions in the binary image. During the candidate region selection process, geometric shape features are considered, especially the circularity index. Circularity is defined as:

[0030] Where A is the area of ​​the connected region and P is the perimeter. For an ideal circle, the roundness value is close to 1, while needle marks usually present as indentations or stains that are nearly circular. Therefore, a roundness threshold (set to 0.5 in this invention) is used to screen potential needle mark candidate regions. The screening results are as follows: Figure 4 As shown.

[0031] In summary, this invention proposes an image-based automatic pinprint recognition method. The first stage utilizes a deep learning object detection network to accurately locate solder joint regions in the entire board image, suppressing background interference at the source and narrowing the search space for subsequent detection. The second stage introduces a local pinprint detection mechanism based on a segmentation model within the solder joint ROI region to maximize the preservation of detailed information of small targets and improve boundary delineation accuracy. Through this hierarchical detection strategy, this invention can significantly improve the recall and localization accuracy of pinprint recognition, achieving high-precision and robust detection in complex backgrounds. Furthermore, this method has good scalability and can be transferred to automated detection tasks for other types of small surface features or defects.

[0032] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. All should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An automatic detection method for two-stage probe card imprints based on YOLO improvement, characterized in that, Includes the following steps: Construct a solder joint sample dataset and train a pin mark detection model based on the YOLOv8 framework; The solder joint location results are obtained based on the trained needle mark detection model. Based on the solder joint location results, potential pin mark features are extracted using image processing methods; Based on the potential needle mark characteristics, a geometric shape feature determination method is used to identify and locate needle mark defects.

2. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 1, characterized in that, The solder joint sample dataset includes high-resolution chip images with different light intensities, contrast levels, resolutions, and surface contamination conditions.

3. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 1, characterized in that, The YOLOv8 framework includes a Backbone, a Neck, and a Head. The Backbone uses an improved C2f module for multi-scale feature extraction. The Neck uses a combination of a feature pyramid network and a path aggregation network for multi-scale feature fusion. The Head uses an Anchor-Free prediction mechanism to directly regress the target center point and bounding box parameters, and outputs the class probability and target confidence.

4. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 1, characterized in that, During the training process of the needle print detection model, data augmentation strategies such as Mosaic stitching, random flipping, color jittering, and scale scaling are used, and the learning rate is dynamically adjusted in conjunction with a cosine annealing strategy.

5. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 1, characterized in that, The process of extracting latent needleprint features using image processing methods includes: A fixed threshold segmentation method is used to separate the foreground and background of the solder joint region in the solder joint location result to obtain a binarized image. Connectivity analysis is used to label and extract features from all independent regions in the binarized image.

6. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 1, characterized in that, The process of identifying and locating pinprint defects using geometric shape feature determination methods includes: A roundness threshold is set, and the roundness value of the potential needle mark feature is calculated. When the calculated roundness value exceeds the roundness threshold, it is determined to be a needle mark defect.

7. The automatic detection method for two-stage probe card imprints based on YOLO improvement according to claim 6, characterized in that, The method for calculating the roundness value is as follows: ， Where C represents the roundness value, A is the area of ​​the connected region, and P is the perimeter.

Images