A multi-type chip pin positioning method based on binocular vision and related device

Abstract

The application discloses a multi-type chip pin positioning method based on binocular vision and a related device, which is applied to the alignment of probes and chip pins in electrostatic discharge testing, and the method comprises the following steps: acquiring a binocular vision image of a chip to be tested in CDM discharge testing; inputting the binocular vision image of the chip to be tested into a pre-trained chip recognition model to obtain a chip recognition result; performing image processing on a chip contour according to a chip packaging type to obtain a chip pin contour and acquire a pin center point; obtaining three-dimensional coordinates of corresponding pin center points based on the distance between the same pin center points in the binocular vision image; wherein the chip recognition model is obtained by improving a YOLOv8n model, and the improvement method comprises the following steps: replacing a Bottleneck structure of a C2f module in the YOLOv8n model with a FasterBlock, so that accurate and unmanned chip pin positioning is realized.

Description

Technical Field

[0001] This invention belongs to the field of electrostatic measurement technology, and in particular relates to a method and related device for locating pins of multiple types of chips based on binocular vision. Background Technology

[0002] Traditional CDM ESD (Charged Device Model-Electro-Static discharge) testing methods are mainly manual, which has revealed many drawbacks in practical applications: On the one hand, the testing process is highly dependent on the experience and proficiency of the operators. Manually operating the CDM ESD testing equipment and completing the high-precision alignment of the probes with the package pads is not only time-consuming, but also prone to inaccurate test data due to operational jitter or positioning deviations, which greatly affects the efficiency and reliability of the test; on the other hand, manual testing is difficult to adapt to the diverse needs of modern complex device structures, and its testing accuracy and efficiency can no longer meet the quality control requirements of large-scale integrated circuit production.

[0003] Automated CDM ESD testing systems are being widely researched and applied. Through high-precision mechanical systems and computer control technology, they improve the alignment speed of test probes and package pins, reduce human error, and enhance alignment accuracy. However, despite the significant advantages of automated CDM ESD testing systems in terms of accuracy and efficiency, they still face some technical challenges. Especially with the rapid development of chip packaging technology, there are numerous types of chip package pins, including common ones such as Transistor Outline (TO), Small Outline Package (SOP), Ball Grid Array (BGA), Quad Flat Package (QFP), and Quad Flat No-Leads Package (QFN). The differences between the pins of different packages are also significant; for example, QFP and BGA pins differ considerably, while QFP and QFN pins differ only slightly. Package pin sizes span multiple orders of magnitude, from micrometers (µm) for small BGA packages to centimeter-sized TO packages. The number of pins for the same type of chip package also varies considerably, and the height of each pin also differs. These challenges constitute the technical bottleneck for accurate identification and positioning of chip package pins and the acquisition of their three-dimensional coordinates in CDM ESD testing.

[0004] Meanwhile, chip package pins, as the key interface connecting the chip to external circuits, pose a huge challenge to traditional detection and identification technologies due to their small size, high density, and diverse types. Summary of the Invention

[0005] The purpose of this invention is to provide a method and related device for locating pins of multiple types of chips based on binocular vision. The method uses an improved lightweight YoLov8n to perform contour recognition on the chip. After dividing the recognized contour into regions of interest, a series of image processing operations are performed to locate the pixel point of the pin center point to be aligned in the binocular vision image. The three-dimensional coordinates of the pixel point are obtained through the triangulation principle of binocular vision, thereby achieving high-precision pin positioning.

[0006] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0007] In a first aspect, the present invention provides a method for locating multi-type chip pins based on binocular vision, the method being used to align probes with chip pins during electrostatic discharge testing, comprising:

[0008] Acquire binocular visual images of the chip under test during CDM discharge testing;

[0009] The binocular vision image of the chip under test is input into a pre-trained chip recognition model to obtain the chip recognition result; the chip recognition result includes the chip outline and the chip package type.

[0010] The chip outline is image-processed according to the chip package type to obtain the chip pin outline and pin center point;

[0011] Based on the distance between the center points of the same pin in the binocular vision image, the three-dimensional coordinates of the corresponding pin center point are obtained;

[0012] The chip identification model is obtained by improving the YOLOv8n model. The improvement method includes replacing the Bottleneck structure of the C2f module in the YOLOv8n model with FasterBlock.

[0013] Optionally, the method for improving the YOLOv8n model using the chip recognition model further includes: replacing the original convolutional layers in the backbone and neck networks of the YOLOv8n model with depth-separable convolutional layers, and simplifying the downsampling operation and detection head structure in the original network.

[0014] Optionally, the training method for the chip recognition model includes:

[0015] Acquire historical chip images of various package types, including ball grid array (BGA) packages, small outline packages, and rectangular flat packages.

[0016] The chip outline and chip package type are labeled for each historical chip image to obtain a sample set;

[0017] The sample set is input into the pre-built chip recognition model to obtain the trained chip recognition model.

[0018] Optionally, the backbone network is used to extract features from the input chip image under test to obtain multi-scale global semantic features of the chip image under test; wherein, the backbone network includes a series of linearly connected deep separable convolutional layers, a three-level feature extraction module and a fast spatial pyramid pooling module.

[0019] Cross-scale fused features are obtained by fusing multi-scale global semantic features through a neck network;

[0020] The chip recognition result is obtained by the detection head based on cross-scale fusion feature mapping.

[0021] Optionally, the step of extracting features from the input chip image under test through the backbone network to obtain multi-scale global semantic features of the chip image under test includes:

[0022] The image of the chip under test is downsampled by multiple deep separable convolutional layers, and a compressed primary feature map is output.

[0023] The compressed primary feature map is processed by a three-level feature extraction module to extract features and output multi-scale features. Each feature extraction module includes a cascaded C2f_Faster module and a depthwise separable convolutional layer. The C2f_Faster module is the C2f module with Bottleneck replaced by FasterBlock.

[0024] Multi-scale features are pooled in parallel using a pyramid pooling layer to output multi-scale global semantic features.

[0025] Optionally, the process of fusing multi-scale global semantic features through the neck network to obtain cross-scale fused features includes:

[0026] In the top-down path, the deep semantic features in the multi-scale global semantic features are upsampled through the upsampling layer, and the upsampled features are concatenated with the corresponding scale features in the multi-scale global semantic features. Then, the concatenated features are enhanced by the C2f_Faster module.

[0027] In the bottom-up path, the low-level semantic features in the multi-scale global semantic features are downsampled through depthwise separable convolutional layers. The downsampled features are then concatenated with the corresponding scale features in the top-down path. The concatenated features are then enhanced by the C2f_Faster module to output cross-scale fused features.

[0028] Optionally, obtaining the chip recognition result by the detection head based on the cross-scale fusion feature vector mapping includes:

[0029] Each detection head maps the feature map of the corresponding scale in the cross-scale fusion feature vector through anchor frames of different sizes to obtain multiple prediction vectors; wherein, the prediction vector includes chip package type, chip contour bounding box and confidence level;

[0030] Filter out prediction vectors with low confidence, use non-maximum suppression to merge overlapping chip contour bounding boxes in the remaining prediction vectors, and output the final chip contour and chip package type.

[0031] Optionally, the step of performing image processing on the chip contour recognition result according to the chip package type to obtain the chip pin contour and acquire the pin center point includes:

[0032] The chip contour recognition results are compared with the image background to perform ROI region segmentation;

[0033] Image processing is performed on the segmented ROI region according to the chip package type to extract the chip pin contours; wherein, the image processing operations include image enhancement, grayscale conversion, filtering and noise reduction, adaptive threshold binarization, and Canny edge detection; multiple image processing operations are selected from each image processing operation according to the chip package type and executed sequentially;

[0034] The pixel point at the center of the pin is obtained based on the chip pin profile using the generalized Hough transform operation.

[0035] Optionally, obtaining the three-dimensional coordinates of the corresponding pin center point based on the distance between the center points of the same pin in the binocular vision image includes:

[0036] The corresponding pixel points of the center point of the same pin on the left and right images of the stereo camera are obtained by stereo matching.

[0037] Based on the baseline distance and focal length of the binocular camera, the parallax between corresponding pixels is calculated using the principle of triangulation, and the three-dimensional coordinates of the corresponding pin center point are obtained.

[0038] Secondly, the present invention provides a multi-type chip pin positioning device based on binocular vision, comprising:

[0039] Chip binocular image acquisition module: used to acquire binocular visual images of the chip under test during CDM discharge testing;

[0040] Chip recognition result acquisition module: used to input the binocular vision image of the chip under test into the pre-trained chip recognition model to obtain the chip recognition result; wherein, the chip recognition result includes the chip outline and the chip package type;

[0041] Pin center point acquisition module: used to perform image processing on the chip outline according to the chip package type, obtain the chip pin outline and acquire the pin center point;

[0042] Pin coordinate acquisition module: used to obtain the three-dimensional coordinates of the corresponding pin center point based on the distance between the center points of the same pin in the binocular vision image;

[0043] The chip identification model is obtained by improving the YOLOv8n model. The improvement method includes replacing the Bottleneck structure of the C2f module in the YOLOv8n model with FasterBlock.

[0044] Thirdly, the present invention provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-type chip pin positioning method based on binocular vision as described in any of the first aspects.

[0045] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: By replacing the Bottleneck structure of the C2f module in the YOLOv8n model with FasterBlock to obtain the chip recognition model, the number of model parameters and computational load are reduced while maintaining strong detection performance. This makes the chip recognition model more lightweight. Using the lightweight chip recognition model as the front-end for identifying the chip under test is more adaptable to the positioning speed required for probe alignment pins in electrostatic testing. Based on the identified chip contour and chip package type, customized identification of pins for different package types of chips is then performed, enabling accurate chip pin contours to be obtained for various chip types. This improves the accuracy of chip pin identification. It uses corresponding image processing technology to segment and locate chip pins based on their package type, and accurately obtains the three-dimensional coordinate information of the pins based on the visual difference of the binocular camera and the stereo matching of pixels. No manual intervention is required in the chip identification and pin positioning process. It can automatically adapt to multiple chip package types, is simple to deploy, and has strong environmental adaptability. With just a binocular camera and identification algorithm, it can complete the identification of multiple chip pins in multiple scenarios and obtain sub-millimeter-level three-dimensional coordinates of the pins. It is suitable for electrostatic discharge testing scenarios of charged device models, meeting the requirements of large-scale integrated circuit production quality control for testing efficiency and accuracy. Attached Figure Description

[0046] Figure 1 The diagram shown is a flowchart of a multi-type chip pin positioning method based on binocular vision in one embodiment of the present invention.

[0047] Figure 2 The diagram shown is a schematic diagram of the C2f_Faster module structure in one embodiment of the present invention;

[0048] Figure 3The diagram shown is a schematic diagram of the chip recognition model structure in one embodiment of the present invention;

[0049] Figure 4 The diagram shows the step-by-step results of the pin positioning process for the chip under test in one embodiment of the present invention; wherein, (a) is the image of the chip under test captured by the camera, (b) is the chip outline image obtained by the chip recognition model from the image of the chip under test, (c) is the grayscale chip outline image, and (d) is the chip pin positioning image extracted from the grayscale chip outline image.

[0050] Figure 5 The diagram shown is a schematic representation of the binocular vision measurement principle in one embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0052] Example 1

[0053] With the continuous development of computer vision technology, the technology of using cameras to simulate human eye perception has shown significant advantages in industrial production and testing. Compared with traditional manual inspection, vision systems can process large amounts of image data at a higher speed, reduce human error, and still operate stably in complex environments or under difficult operating conditions. At the same time, computer vision technology is highly flexible and can be customized and optimized according to different needs to adapt to different industrial scenarios. Furthermore, with the help of advanced deep learning algorithms such as Region-based Convolutional Neural Network (RCNN) and YOLO (You Only Look Once), computer vision systems can achieve high-precision, automated detection and recognition, greatly improving the accuracy and efficiency of the production process.

[0054] like Figure 1 As shown, this embodiment provides a multi-type chip pin positioning method based on binocular vision, applied to the field of electrostatic discharge testing. Addressing the difficulty of probe and chip pin positioning in CDM ESD testing, this method introduces computer vision into CDM ESD testing, proposing a binocular vision-based multi-type chip identification and sub-millimeter-level pin 3D coordinate testing system. In the CDM ESD testing system, by deploying a binocular camera and a recognition algorithm and image processing module, various packaged chips can be quickly identified, each pin can be accurately located, and finally, the 3D coordinate information of each pin is output. The specific steps are as follows:

[0055] S1: Constructing a sample set for training the chip recognition model

[0056] Currently, the CDM ESD testing field generally uses traditional manual alignment methods and lacks publicly available datasets for different chip package types. To address this issue, this study independently constructed a chip package image dataset. The specific construction process is as follows:

[0057] During the data acquisition phase, both a binocular camera and a smartphone were used to capture images of semiconductor chips commonly found in industrial scenarios, such as BGA, SOP, and QFN. In this embodiment, to balance the characteristics of small sample size, recognition accuracy, and training cost, and to adapt to the complexity of engineering applications, a total of 236 images were captured. Images captured by the smartphone were used as training samples, while images captured by the binocular camera used for testing were used as validation samples.

[0058] The acquired raw images were processed by adjusting brightness, cropping, translating, and rotating to obtain standardized sample images. The semi-automatic annotation software labelimg was used to annotate the standardized sample images with chip outlines and chip package types. The annotation results were saved in txt file format.

[0059] S2: Construct a lightweight L-YoLov8n model as a chip recognition model.

[0060] YOLOv8n mainly consists of three parts: the backbone network, the neck network, and the head network, which are used for feature extraction, feature fusion, and prediction output, respectively.

[0061] To achieve the research goal of high-precision identification of multiple chip package types and meet the real-time and lightweight requirements of CDM ESD testing scenarios, a lightweight design of the YOLOv8n algorithm model was implemented. Firstly, the Bottleneck structure in the C2f module was replaced with the FasterBlock acceleration block, constructing the C2f_Faster acceleration module. FasterBlock is a high-efficiency network module that reduces the number of parameters and computational load while maintaining strong performance by optimizing the arrangement and connection of convolutional layers.

[0062] like Figure 2As shown, the overall structure of the C2f_Faster module is a lightweight feature fusion structure optimized from the C2f module. The C2f_Faster module processes the input features in the following steps: the input features first pass through a convolutional layer to complete the initial feature dimension adjustment and information extraction; the preprocessed features are split into two parallel branches; one branch passes through n acceleration blocks in sequence to complete the depth transformation of the features with lightweight operations; the branch that has passed through the acceleration blocks is concatenated with the unprocessed original branch to fuse the feature information from different paths; finally, a convolutional layer is used to integrate the concatenated features and output the final features.

[0063] Secondly, depthwise separable convolutions (DSConv) are used to replace the original convolutional layers in the backbone and neck of the original network. The main advantage of DSConv is that it significantly reduces the number of parameters and computational complexity, and can significantly reduce computational costs compared with ordinary convolutional layers while maintaining similar performance.

[0064] The original YOLOv8n backbone network includes multiple downsampling operations, which improves the feature representation of medium and large targets but reduces the feature representation of small targets. Furthermore, there are three downsampling operations in the neck network, and three prediction heads in the original detection head, corresponding to the detection of small, medium, and large targets. However, as the feature scale decreases, the contained detailed information is lost, resulting in redundancy in small target detection. Therefore, the YOLOv8n network structure was optimized for the small target detection task in chip recognition. The network structure of the lightweight L-YOLOv8n model is as follows: Figure 3 As shown, it comprises the following three parts: Backbone network: consisting of sequentially connected depthwise separable convolutional layers, a three-level feature extraction module, and a fast spatial pyramid pooling module; each feature extraction module includes a cascaded C2f_Faster module and a depthwise separable convolutional layer; Neck network: comprising a top-down path and a bottom-up path. The top-down path consists of upsampling, concatenation layers, and a C2f_Faster module, while the bottom-up path consists of depthwise separable convolutional layers, a concatenation layer, and a C2f_Faster module; Detection head: comprising two prediction heads, each including two cascaded convolutional layers and one two-dimensional convolutional layer.

[0065] S3: Train the chip recognition model using a sample set.

[0066] In the backbone network: the input image is downsampled and initially channel augmented through multiple deep separable convolutional layers, and a compressed primary feature map is output; a cascaded three-level feature extraction module is used to extract features from the compressed primary feature map, and multi-scale features are output; a fast spatial pyramid pooling module is used to perform parallel pooling of the multi-scale features, and multi-scale global semantic features are output.

[0067] In the neck network: In the top-down path, the deep semantic features in the multi-scale global semantic features are upsampled through an upsampling layer, and the upsampled features are concatenated with the corresponding scale features in the multi-scale global semantic features. Then, the concatenated features are refined by the C2f_Faster module. In the bottom-up path, the low-level semantic features in the multi-scale global semantic features are downsampled through a depthwise separable convolutional layer, and the downsampled features are concatenated with the corresponding scale features in the upsampling path. Then, the concatenated features are refined by the C2f_Faster module, and a multi-level cross-scale fused feature vector is output.

[0068] In the detection head: the simplified detection head obtains the chip contour recognition result and the chip package type recognition result based on the cross-scale fusion feature vector mapping; the parameters of the chip recognition model are adjusted according to the loss between the chip contour recognition result and the corresponding real chip contour, and the loss between the chip package type recognition result and the corresponding real chip package type, to obtain the trained chip recognition model.

[0069] Compared to the original YOLOv8n model, the L-YOLOv8n model shows the variation of Mean Average Precision (mAP) when the Intersection over Union (IoU) ratio is 0.5-0.95, as shown in Table 1. Its mAP50 remains stable above 99%, and the mAP50-95 remains essentially unchanged, meeting the requirements for accurate identification of the chip under test during CDM ESD testing. Simultaneously, training time is reduced by 56%, detection speed is increased by 30%, and model size is compressed by 54%. While ensuring accuracy and stability in identification and detection, it significantly reduces the system's resource requirements, thereby improving its deployment efficiency and feasibility in CDM ESD testing applications.

[0070] Table 1

[0071]

[0072] S4: As Figure 4 As shown, contour recognition and image processing are performed on the chip under test in CDM electrostatic testing.

[0073] Identification and positioning of package pins is a crucial step in this research. Through study and analysis, it was found that high similarity exists between different pins, leading to overgeneralization and low identification accuracy. Therefore, the improved L-YOLOv8n model cannot be directly used for pin identification and detection. To address this, a lightweight L-YOLOv8n model is used as the front end to first identify the entire chip. Then, different image processing modules are designed for different chip types to identify and position all their pins. Taking a BGA chip with many pins and a small package as an example, the pin positioning process involves most of the image processing steps, as detailed below:

[0074] (1) Using a binocular camera, the L-YOLOv8n model is used to identify the chip under test, determine the type of the chip, and use the identified chip as a whole as the ROI (Region of Interest). Image processing technology is then used to achieve precise segmentation and positioning of the pins. The binocular camera is used to identify the chip under test, obtain the chip type, and use the whole chip as the ROI.

[0075] (2) Due to the low resolution of the camera used, in order to solve the problem of insufficient accuracy caused by the low resolution of the camera, the ROI area is magnified by 4 times interpolation (here, 4 times refers to the magnification factor of the side length). The derivation of the change in the number of pixels is as follows:

[0076] Let the length of the original ROI image be... Width The total number of original pixels for:

[0077] ,

[0078] After being magnified 4 times, the length of the image becomes Width becomes Total number of pixels after magnification for:

[0079] ,

[0080] As the above derivation shows, after a 4x interpolation magnification, the total number of image pixels becomes 16 times the original, and the detail accuracy is improved through pixel filling. The magnification process uses a bilinear interpolation algorithm to achieve accurate pixel filling, and its core calculation formula is as follows:

[0081] ,

[0082] In the formula, For the target interpolation point coordinates, , , , These are the coordinates of the four original pixels surrounding the interpolation point; , These are the interpolation points relative to the original pixel points. axis, Axis offset, This is a function for pixel grayscale values.

[0083] (3) Simultaneously, in order to adapt to the processing of digital computers, the spatial and amplitude values ​​of continuous image functions must be digitized. Spatial coordinates The digitization of image values ​​is called image sampling, while the digitization of amplitude is called grayscale quantization. The digitized image is called a digital image (or discrete image). A black and white image can be represented using a two-dimensional function. It means that, among them , It is a two-dimensional coordinate system on a plane. Point The brightness value (grayscale value). If it's a color image, the values ​​at each point should also reflect color changes, which can be expressed using... It means that, among them The wavelength is used. Therefore, image enhancement and grayscale processing are performed on the ROI to reduce computational complexity and highlight the contour features of the pin area.

[0084] (4) A filtering algorithm is used to smooth the image to suppress noise interference and preserve the edge information of the pins. Next, an adaptive thresholding algorithm is used to binarize the image to address the problems of uneven illumination and high background complexity. For any pixel in the image... Its adaptive threshold The calculation formula is:

[0085] ,

[0086] In the formula, For The average grayscale value of pixels within a local neighborhood centered on the center. This is the set constant offset.

[0087] The image is converted to a black-and-white binary image based on an adaptive threshold, and the determination rule is as follows:

[0088] ,

[0089] In the formula, These are the binarized pixel values, where 255 represents the pin area and 0 represents the background area. for The pixel grayscale value.

[0090] (5) Subsequently, the pin contour information is extracted using the Canny edge detection algorithm. This algorithm determines the edge by calculating the gradient magnitude and direction of the image. and direction The calculation formula is:

[0091] ,

[0092] ,

[0093] In the formula, and The images are respectively in direction and The gradient value in the direction.

[0094] (6) The image after edge detection is processed using the generalized Hough transform algorithm to extract the coordinates of the pin center point. For the set of edge points of the pin... Its corresponding Hough space accumulator The calculation formula is:

[0095] ,

[0096] In the formula, For edge point indexing, The number of edge points. Let be the parameters of the Hough space. This represents the perpendicular distance from the origin to the line. Represents the perpendicular line and The included angle of the axis, This is the Dirac function, which takes the value 1 when the parameter satisfies the equation of the straight line, and 0 otherwise.

[0097] By finding the accumulator peak in Hough space, the geometric center position of the pin is determined, and finally the coordinates of the pin center point are obtained.

[0098] (7) Map the pin coordinates obtained from the ROI image back to the original image to improve positioning accuracy. Let the magnification factor of the ROI image be... (Side length magnification factor), then the coordinates of any point in the ROI image. Coordinates mapped to the original image The conversion formula is:

[0099] ,

[0100] ,

[0101] Based on the pixel count derivation results above, the pixel density of the ROI after 4x magnification is 16 times that of the original image. Therefore, 16 pixels in the ROI image correspond to 1 pixel in the original image. This mapping relationship can significantly reduce positioning errors and improve pin positioning accuracy.

[0102] S5: Obtain the three-dimensional coordinates of the pin center point based on the ranging principle of binocular vision technology.

[0103] After obtaining the two-dimensional coordinate information of the pin center point by calculating the centroid of the pin profile, and realizing the visual recognition and positioning of the chip package pin, the parallax of the pin in the binocular camera is calculated based on the obtained two-dimensional coordinate information of the pin. Then, the pixel position of the feature point in the left and right cameras is determined by the stereo matching algorithm. Finally, the depth information of the pin can be obtained by perspective transformation and triangulation. The specific steps are as follows:

[0104] In binocular stereo vision, the three-dimensional information of the target point is obtained by establishing a mapping relationship between spatial points and image points based on four coordinate systems. Figure 5 Taking the left camera in the image as an example, a point in space The coordinates in the two images are respectively and Based on the camera imaging principle, an image coordinate system can be established. With the physical coordinate system of the image The relationship between them:

[0105] ,

[0106] In the formula, and Representing points respectively Coordinates in the left image coordinate system and Representing points respectively Coordinates in the physical coordinate system of the left image. and These represent the optical center points of the left camera. Coordinates in the left image coordinate system and They represent in , The distance between pixels in the direction.

[0107] World coordinate system With camera coordinate system The transformation relationship between them is as follows:

[0108] ,

[0109] In the formula, For rotation matrix, The translation vector reflects the positional transformation relationship between the world coordinate system and the camera coordinate system; it is called the extrinsic parameter. , , Representing points respectively Coordinates in the left camera coordinate system , , Representing points respectively Coordinates in the world coordinate system.

[0110] If the optical center of the left camera is known. The distance to the image plane, i.e., the effective focal length. Then the camera coordinate system can be obtained. With the physical coordinate system of the image The relationship between them:

[0111] ,

[0112] ,

[0113] Based on the calculation formula above in this step, the pixel coordinate system can be obtained. and world coordinate system The relationship between them:

[0114] ,

[0115] in, It is a zero vector.

[0116] If line segment and Let be the distances from the image points of the left and right cameras to the left image plane, respectively. Then, the parallax of this point on the left and right cameras is... It can be defined as:

[0117] Based on the transformation relationship between coordinate systems and the principle of triangulation, combined with the baseline distance between the two parallel cameras... and the camera's focal length The two-dimensional spatial coordinates of the target point can be obtained. and depth information :

[0118] ,

[0119] Based on two-dimensional spatial coordinate information and depth information The three-dimensional coordinates of the center point of the target pin are obtained. .

[0120] Example 2

[0121] This embodiment provides a multi-type chip pin positioning device based on binocular vision, including:

[0122] Chip binocular image acquisition module: used to acquire binocular visual images of the chip under test during CDM discharge testing;

[0123] Chip recognition result acquisition module: used to input the binocular vision image of the chip under test into the pre-trained chip recognition model to obtain the chip recognition result; wherein, the chip recognition result includes the chip outline and the chip package type;

[0124] Pin center point acquisition module: used to perform image processing on the chip outline according to the chip package type, obtain the chip pin outline and acquire the pin center point;

[0125] Pin coordinate acquisition module: used to obtain the three-dimensional coordinates of the corresponding pin center point based on the distance between the center points of the same pin in the binocular vision image;

[0126] The chip identification model is obtained by improving the YOLOv8n model. The improvement method includes replacing the Bottleneck structure of the C2f module in the YOLOv8n model with FasterBlock.

[0127] The device provided in this embodiment can execute the multi-type chip pin positioning method based on binocular vision provided in any step of Embodiment 1, and has the corresponding functional modules and beneficial effects of the execution method.

[0128] Example 3

[0129] This embodiment provides a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the multi-type chip pin positioning method based on binocular vision provided in any step of Embodiment 1.

[0130] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0132] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0133] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0134] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A multi-type chip pin positioning method based on binocular vision, the method being used for aligning a probe with a chip pin in an electrostatic discharge test, characterized in that, The application relates to a chip recognition method and device. The method comprises the following steps: acquiring binocular vision images of a chip to be tested in a CDM discharge test; inputting the binocular vision images of the chip to be tested into a pre-trained chip recognition model to obtain a chip recognition result; wherein the chip recognition result comprises a chip contour and a chip packaging type; performing image processing on the chip contour according to the chip packaging type to obtain a chip pin contour and acquire pin center points; based on the distance between the same pin center points in the binocular vision images, three-dimensional coordinates of the corresponding pin center points are obtained.

2. The binocular vision-based multi-type chip pin positioning method according to claim 1, wherein, The chip recognition model is obtained by improving a YOLOv8n model, and the improvement method comprises replacing a Bottleneck structure of a C2f module in the YOLOv8n model with a FasterBlock.

3. The binocular vision-based multi-type chip pin positioning method according to claim 2, wherein, The method for improving the YOLOv8n model by using the chip recognition model further comprises replacing original convolution layers in a backbone network and a neck network of the YOLOv8n model with depth separable convolution layers, and reducing the number of down-sampling operations and the number of detection heads in the neck network. The method for inputting the binocular vision images of the chip to be tested into the pre-trained chip recognition model to obtain the chip recognition result comprises the following steps: extracting features of the input chip image to be tested by using a backbone network to obtain multi-scale global semantic features of the chip image to be tested; wherein the backbone network comprises a plurality of layers of depth separable convolution layers, a three-level feature extraction module and a fast spatial pyramid pooling module which are linearly connected in sequence; fusing the multi-scale global semantic features by using a neck network to obtain cross-scale fusion features; 4. The binocular vision-based multi-type chip pin positioning method according to claim 3, wherein, mapping the cross-scale fusion features to obtain the chip recognition result by using a detection head. The method for extracting features of the input chip image to be tested by using the backbone network to obtain multi-scale global semantic features of the chip image to be tested comprises the following steps: down-sampling the chip image to be tested by using a plurality of layers of depth separable convolution layers to output compressed primary feature maps; extracting features of the compressed primary feature maps by using a three-level feature extraction module to output multi-scale features; wherein each level of the feature extraction module comprises a C2f_Faster module and a depth separable convolution layer which are connected in series, and the C2f_Faster module is a C2f module after the Bottleneck is replaced by the FasterBlock; 5. The binocular vision-based multi-type chip pin positioning method according to claim 3, wherein, performing parallel pooling on the multi-scale features by using a pyramid pooling layer to output multi-scale global semantic features. The method for fusing the multi-scale global semantic features by using the neck network to obtain cross-scale fusion features comprises the following steps: in a top-down path, up-sampling deep semantic features in the multi-scale global semantic features by using an up-sampling layer, splicing the up-sampled features and corresponding scale features in the multi-scale global semantic features, and then enhancing the spliced features by using a C2f_Faster module; in a bottom-up path, down-sampling low semantic features in the multi-scale global semantic features by using a depth separable convolution layer, splicing the down-sampled features and corresponding scale features in the top-down path, and then enhancing the spliced features by using a C2f_Faster module to output cross-scale fusion features.

6. The binocular vision-based multi-type chip pin positioning method according to claim 3, wherein, The chip recognition result is obtained by detecting the head according to the cross-scale fusion feature vector mapping, comprising: Each detection head maps the feature map of the corresponding scale in the cross-scale fusion feature vector through different size anchor boxes to obtain a plurality of prediction vectors; wherein the prediction vector includes a chip package type, a chip contour bounding box and a confidence level; Filtering out low-confidence prediction vectors, using non-maximum suppression to merge overlapping chip contour bounding boxes in the remaining prediction vectors, and outputting the final chip contour and chip package type.

7. The binocular vision-based multi-type chip pin positioning method according to claim 1, wherein, The chip pin contour is obtained by image processing the chip contour recognition result according to the chip package type, and the pin center point is obtained, comprising: The chip contour recognition result is ROI region segmented with the image background; According to the chip package type, the ROI region after segmentation is image processed to extract the chip pin contour; Through the generalized Hough transform operation, the pixel point of the pin center point is obtained according to the chip pin contour.

8. The binocular vision based multi-type chip pin positioning method according to claim 1, wherein, The three-dimensional coordinates of the corresponding pin center points are obtained based on the distance between the same pin center points in the binocular vision image, comprising: The corresponding pixel points of the same pin center points in the binocular vision image are obtained through stereo matching; According to the baseline distance and focal length of the binocular camera, the parallax between the corresponding pixel points is calculated through the principle of triangulation to obtain the three-dimensional coordinates of the corresponding pin center points.

9. A multi-type chip pin positioning device based on binocular vision, characterized in that, It comprises: A chip binocular image acquisition module is used to acquire the binocular vision image of the chip to be tested in the CDM discharge test; A chip recognition result acquisition module is used to input the binocular vision image of the chip to be tested into a pre-trained chip recognition model to obtain a chip recognition result; wherein the chip recognition result includes a chip contour and a chip package type; A pin center point acquisition module is used to image process the chip contour according to the chip package type to obtain the chip pin contour and the pin center point; A pin coordinate acquisition module is used to obtain the three-dimensional coordinates of the corresponding pin center points based on the distance between the same pin center points in the binocular vision image. The chip recognition model is obtained by improving the YOLOv8n model, and the improvement method comprises replacing the Bottleneck structure of the C2f module in the YOLOv8n model with a FasterBlock.

10. A computer storage medium having stored thereon a computer program, characterized in that When the computer program is executed by the processor, the multi-type chip pin positioning method based on binocular vision is realized as claimed in any one of claims 1-8.

Images