Pin position detection method and system

Abstract

The application discloses a PIN needle position degree detection method and system, the method comprises the following steps: collecting a front image and a side image of the PIN needle, establishing a unified pixel coordinate and physical coordinate mapping relationship through joint calibration of the front camera and the side camera, and realizing spatial alignment of the front image and the side image; carrying out denoising and illumination equalization processing on the front image and the side image, and separating the PIN needle foreground area and the background area in the front image and the side image through an image segmentation algorithm; carrying out edge detection and sub-pixel interpolation processing on the separated image, and extracting the center line and the end point sub-pixel level coordinates of the PIN needle in the front image; calculating the position degree parameters of the PIN needle based on the extracted sub-pixel level features; comparing the calculated position degree parameters with the respective corresponding preset tolerance threshold, judging whether the PIN needle or the product corresponding thereto is qualified or not, outputting a detection result, and controlling a production line to perform corresponding actions according to the judgment result.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection technology, and in particular relates to a method and system for detecting the position of a PIN pin. Background Technology

[0002] Pins are the core conductive structures in electronic connectors, semiconductor packages, and other products. Their positional accuracy (including planar offset, pin spacing, perpendicularity, and coplanarity) directly affects the electrical connection reliability, signal transmission integrity, and subsequent assembly compatibility of the product. As electronic components develop towards miniaturization, high density, and high reliability, increasingly higher requirements are being placed on the accuracy and efficiency of pin positional detection.

[0003] Currently, PIN pin position accuracy is primarily assessed using two methods: manual inspection and traditional machine vision inspection. Manual inspection typically involves using an optical microscope, tool microscope, or calipers, with inspectors measuring the position parameters of each PIN pin individually. Traditional machine vision inspection, on the other hand, uses industrial cameras to capture images of the PIN pins. Through image processing algorithms such as edge detection, template matching, and Hough transform, the planar position information of the PIN pins is extracted and compared with preset theoretical positions to determine if there is any positional deviation.

[0004] However, both of the above methods have significant technical drawbacks in practical applications. Manual inspection is inefficient, time-consuming per product, and its results are heavily influenced by operator subjectivity, leading to poor consistency between measurements. Furthermore, it is limited by the human eye's ability to discern minute positional deviations down to the micrometer level, and repetitive work over a long period can easily result in missed detections due to visual fatigue. Traditional machine vision inspection methods suffer from insufficient stability in image feature extraction when faced with metallic reflections on PIN pin surfaces, complex textures in plastic casings, and variations in production line lighting, making them prone to edge detection errors or feature loss. Their pixel-based processing has limited ability to resolve minute offsets, failing to meet high-precision inspection requirements. When product models or PIN pin arrangements change, algorithm parameters or matching templates often need to be readjusted, resulting in a cumbersome debugging process. In addition, existing methods are mostly limited to planar position detection, lacking effective means to detect perpendicularity and coplanarity deviations of PIN pins, resulting in a lack of detection dimensions. Summary of the Invention

[0005] The purpose of this invention is to provide a PIN pin position detection method and system to solve the above-mentioned problems existing in the prior art.

[0006] To achieve this objective, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for detecting the position of a PIN pin, comprising: A front camera, a side camera, a telecentric lens, and a ring low-angle light source are used to acquire front and side images of the PIN pin, respectively. A unified mapping relationship between pixel coordinates and physical coordinates is established through joint calibration of the front camera and the side camera, and spatial alignment of the front image and the side image is achieved. The acquired front and side images are subjected to denoising and illumination equalization processing, and the foreground region of the PIN needle in the front and side images is separated from the background region by an image segmentation algorithm. Edge detection and subpixel interpolation are performed on the separated front and side images respectively to extract the subpixel level coordinates of the center line and endpoints of the PIN in the front image, and the subpixel level coordinates of the center line and top point in the side image. Based on the extracted sub-pixel level features, the following positional parameters of the PIN pins are calculated: planar positional degree, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the height deviation range of all PIN pin top points relative to the reference plane obtained by least squares fitting. The calculated position parameters are compared with their respective preset tolerance thresholds to determine whether the PIN or its corresponding product is qualified. The test results are output, and the production line is controlled to perform corresponding actions based on the determination results.

[0007] Secondly, the present invention provides a PIN pin position detection system, comprising: The image acquisition module is used to acquire front and side images of the PIN pin using a front camera, a side camera, a telecentric lens, and a ring low-angle light source, and to establish a unified mapping relationship between pixel coordinates and physical coordinates through joint calibration of the front camera and the side camera, and to achieve spatial alignment between the front image and the side image. The image processing module is used to perform noise reduction and illumination equalization processing on the acquired front and side images, and to separate the foreground region of the PIN needle from the background region in the front and side images using an image segmentation algorithm. The image preprocessing module is used to perform edge detection and subpixel interpolation on the separated front and side images respectively, and extract the subpixel level coordinates of the center line and endpoint of the PIN in the front image, and the subpixel level coordinates of the center line and top point in the side image. The positional calculation module is used to calculate the following positional parameters of the PIN pins based on the extracted sub-pixel level features: planar positional, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the height deviation range of all PIN pin top points relative to the reference plane obtained by least squares fitting. The result determination and output module is used to compare the calculated position parameters with their respective preset tolerance thresholds to determine whether the PIN or its corresponding product is qualified, output the test results, and control the production line to perform corresponding actions based on the determination results.

[0008] The PIN pin position detection method provided in this application employs a front camera and a side camera to acquire front and side images respectively, and combines dual-camera joint calibration to achieve image spatial alignment. This allows for the acquisition of complete spatial pose information of the PIN pin from two orthogonal perspectives, laying a data foundation for 3D parameter calculation. The method effectively improves imaging quality and anti-interference capability by eliminating perspective distortion with a telecentric lens and suppressing metallic reflections with a ring-shaped low-angle light source. Furthermore, by performing denoising and illumination equalization on the images, and combining this with an image segmentation algorithm to separate the PIN pin foreground and background, the method can stably extract the PIN pin region, avoiding the impact of complex backgrounds on subsequent measurements. Interference is eliminated; by performing edge detection and subpixel interpolation on the separated image, subpixel-level coordinates of the center line, endpoints, and top point are extracted, improving the feature localization accuracy to the subpixel level, thereby significantly improving the resolution of geometric measurements. Based on this, planar position (offset and spacing deviation), perpendicularity (angle between the center line and the base normal), and coplanarity (height deviation range of the top point relative to the fitted plane) are calculated based on subpixel-level features, achieving full-dimensional parameter measurement from two-dimensional plane to three-dimensional space. Finally, by comparing with preset tolerance thresholds and outputting the judgment result, automatic judgment of the detection result and production line linkage are achieved. Therefore, this invention, through the synergistic effect of the imaging end, algorithm end, and parameter calculation end, achieves micron-level high-precision, full-dimensional, and automated detection of PIN pin position. Attached Figure Description

[0009] To more clearly illustrate and understand the technical solutions in the embodiments of the present invention, the accompanying drawings used in the background technology and embodiment descriptions of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the content of the embodiments of the present invention and these drawings without creative effort.

[0010] Figure 1 This is a simplified schematic diagram of a PIN pin position detection method provided in an embodiment of this application; Figure 2 This is a schematic diagram of a PIN pin position detection system provided in an embodiment of this application. Detailed Implementation

[0011] To make the technical problems solved by the present invention, the technical solutions adopted, and the technical effects achieved clearer, the technical solutions of the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] Figure 1 A schematic diagram illustrating the implementation process of a PIN pin position detection method provided in this application embodiment includes: Step 110: Using a front camera, a side camera, a telecentric lens, and a ring low-angle light source, front and side images of the PIN pin are acquired respectively. Through joint calibration of the front and side cameras, a unified mapping relationship between pixel coordinates and physical coordinates is established, and spatial alignment between the front and side images is achieved.

[0013] The front camera is mounted vertically downwards, with its optical axis perpendicular to the product base plane, to capture a top-view image of the PIN pins and obtain planar position information. The side camera is mounted horizontally, with its optical axis parallel to the product base plane, to capture a side-view image of the PIN pins and obtain height and tilt information. The two cameras are at a 90-degree angle to each other, forming an orthogonal binocular vision system.

[0014] Joint calibration refers to the process of mathematically calculating the spatial relationship between the frontal and side cameras (including the rotation matrix R and translation vector T) to unify the pixel coordinate systems of the two cameras into the same world coordinate system. Spatial alignment refers to establishing a one-to-one correspondence between the pin coordinates in the frontal image and the height / tilt information of the same pin in the side image, ensuring the accuracy of subsequent 3D parameter calculations.

[0015] In step 110, an image acquisition system is first constructed, consisting of a front industrial camera, a side industrial camera, a telecentric lens, and a ring-shaped low-angle light source. The telecentric lens is used to eliminate perspective distortion and ensure that objects at different distances are imaged at the same size; the ring-shaped low-angle light source illuminates the surface of the PIN pin at a low angle, effectively suppressing metal reflections. During acquisition, the front camera and the side camera are triggered simultaneously to acquire the front and side images of the PIN pin, respectively.

[0016] Based on this, the Zhang Zhengyou calibration method can be used to jointly calibrate the two cameras. The specific operation is as follows: A checkerboard calibration board is simultaneously placed in the field of view of both cameras (this can be achieved using a double-sided calibration board or a reflector). Multiple sets of calibration board images are acquired, the corner coordinates on the calibration board are detected, and the intrinsic parameters (focal length, principal point, distortion coefficient) of each camera and the extrinsic parameters (rotation matrix and translation vector) between the two cameras are calculated. Through joint calibration, a unified mapping relationship between pixel coordinates and physical coordinates is established (e.g., 1 pixel corresponds to 0.008 mm), and spatial alignment between the front and side images is achieved, ensuring that the position coordinates of any pin in the front image accurately correspond to the height and tilt information of the same pin in the side image.

[0017] In some exemplary embodiments, the optical axis of the front camera is perpendicular to the product base plane, and the optical axis of the side camera is parallel to the product base plane; the joint calibration adopts the Zhang Zhengyou calibration method, combined with the checkerboard calibration board that appears in the field of view of both cameras at the same time, to solve the extrinsic parameter matrix between the two cameras.

[0018] The optical axis of the side camera is parallel to the plane of the product base, with an angle of 90 degrees between them, forming an orthogonal imaging layout. The joint calibration adopts the Zhang Zhengyou calibration method, combined with a checkerboard calibration board that appears in the field of view of both cameras simultaneously. By detecting the corresponding corner points on the calibration board, the extrinsic parameter matrix (including the rotation matrix R and the translation vector T) between the two cameras is calculated, thereby achieving the unification of the coordinate systems of the two cameras.

[0019] In some exemplary embodiments, a stop-shooting mode or a fly-shooting mode is used when acquiring front and side images. The stop-shooting mode is: the product is moved to the shooting station and then stops moving, triggering the camera to acquire images. The fly-shooting mode is: the product moves at a constant speed, and the encoder feeds back the position to trigger the camera to acquire images at a predetermined position.

[0020] The stop-shoot mode is suitable for production lines with regular cycle times: the motion platform moves the product to the shooting station and then stops moving, waiting for the platform to stabilize (approximately 30ms). Then, the camera is triggered to capture an image. After the image capture is complete, the platform moves the product away. Image processing is performed in parallel with the platform movement. The fly-shoot mode is suitable for ultra-high-speed production lines (cycle time requirement ≤50ms): the product moves at a constant speed on the motion platform, the encoder provides real-time position feedback, and when the product reaches the predetermined position, the camera is triggered to capture an image, using short exposure (≤10μs) to suppress motion blur.

[0021] As an example, a 6-megapixel industrial camera with a 0.5x telecentric lens was used, the brightness of the ring low-angle light source was set to 2000 lux, and the motion platform accuracy was ±0.001 mm. The front camera was mounted vertically, and the side camera was mounted horizontally, with an angle of 90 degrees between them. The Zhang Zhengyou calibration method was used for joint calibration of the two cameras: a checkerboard calibration board was placed simultaneously in the field of view of both cameras, 15 sets of images of the calibration board in different poses were acquired, corner coordinates were detected, and the extrinsic parameter matrix between the front and side cameras was calculated, establishing a unified pixel-physical coordinate mapping relationship of 1 pixel = 0.008 mm (i.e., 8 micrometers / pixel). A stop-shoot mode was used during acquisition: after the product moved to the shooting position and stopped for 30 milliseconds, the camera was triggered to acquire an image with a resolution of 2448×2048 pixels.

[0022] By orthogonally arranging and jointly calibrating the front and side cameras, complete spatial pose information of the PIN pins can be obtained from two orthogonal perspectives, laying a data foundation for subsequent three-dimensional parameter calculations (perpendicularity, coplanarity); perspective distortion is eliminated by the telecentric lens, ensuring consistent imaging of PIN pins at different positions; metal reflection is suppressed by the ring low-angle light source, effectively improving imaging quality and anti-interference capability; and the selection of stop-shoot or fly-shoot modes can adapt to production lines with different cycle requirements, achieving a balance between detection speed and accuracy.

[0023] Step 120: Denoise and illumination equalization are performed on the acquired front and side images, and the foreground and background regions of the PIN pins in the front and side images are separated by an image segmentation algorithm.

[0024] The foreground region refers to the pixel area where the PIN pins are located in the image, while the background region refers to the non-PIN pin areas such as the connector's plastic housing and metal base.

[0025] In step 120, the acquired front and side images are first denoised using a non-local means denoising algorithm with a window size of 7×7 to remove image noise while preserving the edge details of the PIN pins. Then, illumination equalization is performed using an adaptive histogram equalization algorithm. The image is divided into multiple small blocks, and histogram equalization is applied to each block separately to eliminate local overexposure or underexposure caused by uneven illumination.

[0026] After preprocessing, the foreground and background regions of the PIN pin are separated using an image segmentation algorithm. For simple backgrounds (such as a black plastic shell), an algorithm combining threshold-based segmentation and morphological operations is used: first, binary thresholding (threshold set to 128) is used to initially separate the PIN pin from the background; then, morphological closing operations (structural element 3×3) are used to fill the small holes inside the PIN pin to obtain the complete foreground region. For complex backgrounds (such as a metal base or multiple material overlays), a deep learning-based semantic segmentation model (such as U-Net or DeepLab architecture) is used. The model is pre-trained using the labeled PIN pin region as a supervision signal, inputting the preprocessed grayscale image, and outputting a binary mask of the PIN pin foreground.

[0027] In some exemplary embodiments, denoising employs a nonlocal means denoising algorithm, and illumination equalization employs an adaptive histogram equalization algorithm.

[0028] Nonlocal means denoising is an image denoising algorithm that uses weighted average denoising based on redundant information between similar blocks in an image, effectively preserving edge details while removing noise. Adaptive histogram equalization is an illumination correction algorithm that divides the image into multiple small blocks and performs histogram equalization on each block, thereby eliminating local overexposure or underexposure caused by uneven global illumination.

[0029] In some exemplary embodiments, the image segmentation algorithm is an algorithm that combines threshold-based segmentation with morphological operations, or a semantic segmentation model based on deep learning; the deep learning model adopts the U-Net or DeepLab architecture.

[0030] As an example, the original image undergoes non-local mean denoising with a window size of 7×7. Then, adaptive histogram equalization is performed with a patch size of 32×32 pixels and a contrast limit parameter of 2.0. Since the connector in this embodiment has a black plastic shell and a relatively simple background, an algorithm combining binarized thresholding segmentation and morphological operations is used: the threshold is set to 128 to separate the PIN pin region from the background, and then morphological closing operations using 3×3 structuring elements are used to fill the small holes inside the PIN pins, resulting in a complete binary mask for the PIN pin foreground. For products with complex backgrounds such as semiconductor packages, the U-Net semantic segmentation model is used. The model uses 500 labeled PIN pin region images as training data, inputs a preprocessed grayscale image, and outputs a binary mask for the PIN pin foreground.

[0031] Non-local means denoising removes image noise while preserving PIN pin edge details, preventing edge blurring that could decrease measurement accuracy. Adaptive histogram equalization eliminates local overexposure or underexposure caused by uneven lighting, ensuring image segmentation stability. Image segmentation algorithms separate the PIN pin foreground from the background, effectively eliminating interference from complex backgrounds such as connector plastic housings and metal bases, providing a clean target area for subsequent feature extraction. Different segmentation algorithms can be selected for different background complexities, balancing accuracy with processing efficiency.

[0032] Step 130: Perform edge detection and subpixel interpolation on the separated front and side images respectively, and extract the subpixel coordinates of the center line and endpoints of the PIN in the front image, and the subpixel coordinates of the center line and top point in the side image.

[0033] Subpixel subdivision involves further subdividing a single pixel into smaller units, such as dividing one pixel into 10 subpixel units. Each subpixel unit corresponds to a precision of 0.1 pixels, thereby achieving positioning accuracy higher than the pixel level. Subpixel interpolation calculates the precise location of the extreme point of grayscale change by mathematically fitting the grayscale values ​​near the edge points of integer pixels (such as quadratic polynomial fitting). This location can lie between two integer pixels, thus achieving subpixel-level positioning.

[0034] In some exemplary embodiments, edge detection employs the Canny operator; subpixel interpolation is performed as follows: for the detected integer pixel edge points, the gray values ​​of adjacent pixels along the gradient direction are used to perform a quadratic polynomial fitting, and the extreme points of the fitted curve are taken as the subpixel edge positions.

[0035] Edge detection uses the Canny operator and suppresses false edges through double threshold detection. Subpixel interpolation is performed by taking the gray values ​​of adjacent pixels along the gradient direction for the detected integer edge points and fitting them with a quadratic polynomial. The extreme points of the fitted curve are used as the subpixel edge positions, and the extraction accuracy is better than 0.1 pixels.

[0036] In some exemplary embodiments, the method for extracting the center line of the PIN pin is as follows: calculate the midpoint from the sub-pixel edge points on the left and right sides, obtain a series of sub-pixel center points along the length direction of the PIN pin, and then generate a smooth center line by fitting a B-spline curve. The method for extracting the pin endpoints is as follows: extract the grayscale change curve along the center line direction, and use the grayscale moment method or arctangent function fitting to determine the sub-pixel endpoint position.

[0037] B-spline curve fitting is a curve fitting method that generates a smooth curve using control points, and is used to fit a discrete sequence of center points into a continuous and smooth centerline.

[0038] In some exemplary embodiments, for an image containing multiple pins, a connected component labeling algorithm is used to assign a unique identifier to each pin, and sub-pixel-level feature extraction is performed in parallel on each connected component. This supports multi-threaded or GPU-accelerated architectures, and the total processing time for 200 pins does not exceed 60 milliseconds.

[0039] Connected component labeling is an image processing algorithm that labels adjacent white pixels (foreground pixels) in a binary image as the same connected region and assigns a unique number to each connected region, thereby distinguishing different target objects in the image.

[0040] As an example, the Canny operator is used for edge detection, with a low threshold of 50 and a high threshold of 150 to obtain integer-pixel edge points. For each edge point, the gray values ​​of the three pixels before and after it are taken along the gradient direction, and a quadratic polynomial fitting is performed. The extreme points of the fitted curve are taken as sub-pixel edge positions, achieving an extraction accuracy of 0.1 pixels (corresponding to a physical accuracy of 0.8 micrometers). Centerline extraction: The midpoint is calculated from the sub-pixel edge points on both sides, and a center point is calculated every 1 pixel along the pin length direction to obtain a series of sub-pixel center points. A smooth centerline is then generated by fitting a cubic B-spline curve. Endpoint extraction: The gray-level change curves are extracted along the axial direction near both ends of the centerline, and the sub-pixel endpoint positions are determined by fitting an arctangent function. For a connector image with 50 pins, a connected component labeling algorithm is used to assign a unique ID to each pin, and then multi-threaded parallel processing is used, with the total feature extraction time controlled within 30 milliseconds.

[0041] By employing Canny edge detection and subpixel interpolation processing with quadratic polynomial fitting, the edge positioning accuracy is improved from the integer pixel level to the 0.1 pixel level, corresponding to a physical accuracy of 0.8 micrometers, providing a crucial data foundation for achieving micrometer-level positional accuracy detection. Smooth centerlines are generated through B-spline curve fitting, avoiding the impact of discrete point noise on verticality calculations. Endpoints are determined using the gray-scale moment method or arctangent function fitting, achieving high-precision positioning of top points and providing accurate data for coplanarity calculations. Connected component labeling and a parallel processing architecture support simultaneous detection of multiple PINs, significantly improving detection efficiency; the total processing time for 200 PINs does not exceed 60 milliseconds.

[0042] In step 130, edge detection is first performed on the separated frontal and side images, using the Canny operator to obtain integer-pixel edge points. Then, subpixel interpolation is performed: for each integer-pixel edge point, the gray values ​​of 2-3 pixels before and after it are taken along its gradient direction, and a quadratic polynomial fitting is performed. The extreme point of the fitted curve is the subpixel edge position, with an accuracy of up to 0.1 pixels.

[0043] To extract the center line of the PIN: calculate the midpoint from the sub-pixel edge points on both the left and right sides. That is, for the same horizontal position, the midpoint between the left and right sub-pixel edge points is the center point of that position. Repeat the above calculation along the length of the PIN to obtain a series of sub-pixel center points, and then generate a smooth center line by fitting a B-spline curve.

[0044] For the extraction of PIN pin endpoints: Near both ends of the center line, extract the grayscale change curve along the PIN pin axis. The grayscale jumps from the high grayscale value of the metal pin to the low grayscale value of the background. Use the grayscale moment method or arctangent function to fit the grayscale jump position and determine the sub-pixel level endpoint coordinates.

[0045] For images containing multiple pins, a connected component labeling algorithm is first used to assign a unique identifier to each pin. Then, the above sub-pixel-level feature extraction operation is performed in parallel on each connected component, which can support multi-threading or GPU acceleration.

[0046] Step 140: Based on the extracted sub-pixel level features, calculate the following positional parameters of the PIN pins: planar positional, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the range of height deviations of the top points of all PIN pins relative to the reference plane obtained by least squares fitting.

[0047] Among them, planar position accuracy refers to the positional accuracy of the PIN pins within the base plane, including the offset between actual and theoretical coordinates (ΔX, ΔY), and the deviation between the actual and theoretical spacing between adjacent PIN pins. Perpendicularity refers to the angle between the PIN pin centerline and the base plane normal, used to measure whether the PIN pin is installed perpendicular to the base. The smaller the angle, the better the perpendicularity; ideally, it is 0 degrees (i.e., the PIN pin centerline coincides with the normal). Coplanarity refers to the range of height deviations of all PIN pin top points relative to a reference plane, usually taking the maximum height difference (range) as the coplanarity value. The smaller the value, the more coplanar the PIN pin tops are. Least squares fitting plane: a mathematical fitting method that determines an optimal plane by minimizing the sum of the squares of the vertical distances from all points to the fitting plane, such that the sum of the squares of the deviations from the plane to each point is minimized.

[0048] In step 140, based on the sub-pixel level features extracted in step 130, various positional parameters of the PIN pin are calculated.

[0049] Calculation of planar position: The center line coordinates of the PIN pins extracted from the front image are used as the actual coordinates and compared with the preset theoretical reference coordinates to calculate the offset (ΔX, ΔY) in the X and Y directions; at the same time, the actual distance between adjacent PIN pins is calculated and compared with the theoretical distance to obtain the distance deviation.

[0050] Perpendicularity calculation: The center line of the PIN needle extracted from the side image is fitted into a spatial straight line L using the least squares method. pin Extract multiple points (at least 3 non-collinear points) from the base plane and fit them to the base plane P using the least squares method. base Calculate the normal vector n of the plane. The formula for calculating the perpendicularity angle θ is: , where v pin This is the direction vector of the PIN centerline.

[0051] Coplanarity calculation: Obtain the sub-pixel coordinates of the top of each pin from the side image, and convert them into 3D world coordinates (x, y, z) using camera calibration parameters. i , y i , z i A spatial plane P is fitted using the least squares method. fit z = ax + by + c. Calculate the signed distance d from each top point to the fitted plane. i = |ax i + by i - z i + c| / √(a²+b²+1). Coplanarity is the difference between the maximum and minimum distances: Coplanarity = max(d i ) - min(d i ).

[0052] In some exemplary embodiments, the verticality is calculated as follows: the PIN centerline is extracted from the side image and fitted as a straight line; multiple points are extracted from the base plane and fitted as the base plane; and the angle between the PIN centerline direction vector and the base plane normal vector is calculated.

[0053] In some exemplary embodiments, the coplanarity is calculated as follows: the three-dimensional world coordinates of the top of each pin are obtained from the side image, a spatial plane is fitted using the least squares method, the signed distance from each top point to the fitted plane is calculated, and the difference between the maximum distance and the minimum distance is taken as the coplanarity.

[0054] As an example, the theoretical spacing of the connector pins is 1.27 mm (50 mils), the theoretical perpendicularity angle is 90 degrees (i.e., the pins are perpendicular to the base), and the coplanarity tolerance is ≤5 micrometers. Based on the extracted sub-pixel features, the following are calculated: the actual offset of the pins is 3 micrometers (2 micrometers in the X-axis direction and 1 micrometer in the Y-axis direction), and the spacing deviation between adjacent pins is 2 micrometers; perpendicularity: the center line of the pins is extracted from the side image and fitted as a straight line L_pin, and 5 points are extracted from the base plane and fitted as the base plane P_base. The angle between the center line and the base normal is calculated to be 89.8 degrees, i.e., the perpendicularity deviation is 0.2 degrees; coplanarity: the three-dimensional world coordinates of the top of the 50 pins are obtained from the side image, and the least squares method is used to fit the reference plane. The signed distance from each top point to the plane is calculated, with a maximum distance of 3 micrometers, a minimum distance of 1 micrometer, and a coplanarity of 2 micrometers.

[0055] By calculating planar position based on sub-pixel level features, micrometer-level offset and spacing deviation detection (accuracy up to ±1 micrometer) is achieved, meeting the detection requirements of high-precision PIN pins. By defining perpendicularity as the angle between the PIN pin centerline and the base plane normal, and calculating it based on centerline fitting and base plane fitting in the side image, quantitative assessment of PIN pin tilt is achieved. By defining coplanarity as the height deviation range of all PIN pin top points relative to the least-squares fitted plane, and calculating it based on the three-dimensional coordinates of the top points in the side image, quantitative assessment of the coplanarity of all PIN pin tops is achieved. Therefore, this invention achieves full-dimensional parameter measurement from two-dimensional plane to three-dimensional space, solving the problem of incomplete detection dimensions in traditional methods.

[0056] Step 150: Compare the calculated position parameters with their respective preset tolerance thresholds to determine whether the PIN or its corresponding product is qualified, output the test results, and control the production line to perform corresponding actions based on the determination results.

[0057] Among them, the preset tolerance threshold is the allowable deviation range pre-set according to the product design requirements, such as offset tolerance ≤ 5 micrometers, perpendicularity tolerance ≤ 0.5 degrees, coplanarity tolerance ≤ 3 micrometers, etc. Different products can be configured with different tolerance thresholds. Production line actions refer to the production line control commands automatically triggered based on the detection results, including audible and visual alarms, rejection of non-conforming products, and release of good products.

[0058] In step 150, the planar position (offset, spacing deviation), perpendicularity, and coplanarity parameters calculated in step 140 are compared one by one with their respective preset tolerance thresholds. If all parameters are within the tolerance range, the PIN pin or its corresponding product is deemed qualified; if any parameter exceeds the tolerance range, it is deemed unqualified.

[0059] The system outputs the test results and generates a test report, which includes details of the positional parameters of each pin, pass / fail judgment results, and defective item markings. Simultaneously, based on the judgment results, the system controls the production line to perform corresponding actions: if the product is qualified, the system controls the motion platform to send the product to the next process; if the product is unqualified, the system triggers the production line's audible and visual alarm device and controls the motion platform to remove the unqualified product to the defective product area.

[0060] By automatically comparing the calculated position parameters with the preset tolerance threshold, the objective and rapid determination of the test results is achieved, avoiding the subjectivity and uncertainty of manual judgment; by outputting test reports, data support is provided for quality traceability; by automatically controlling the production line actions (alarm, rejection) based on the judgment results, closed-loop linkage between testing and production line is achieved, meeting the production needs of fully automated industrial production lines.

[0061] The PIN pin position detection method provided by this invention acquires complete spatial pose information of the PIN pin from two orthogonal perspectives through the orthogonal layout and joint calibration of a front camera and a side camera; it eliminates perspective distortion and suppresses metallic reflection by using a telecentric lens and a ring-shaped low-angle light source, improving image quality and anti-interference capability; it effectively eliminates complex background interference through noise reduction, illumination equalization, and image segmentation; and it improves positioning accuracy to 0.1 pixels (corresponding to a physical accuracy of 0.8 micrometers) through sub-pixel-level edge detection and feature extraction. Based on this, it integrates the calculation of planar position, perpendicularity, and coplanarity, realizing full-dimensional parameter measurement from two-dimensional plane to three-dimensional space; finally, it automates the entire detection process through automatic comparison and judgment and production line linkage. Therefore, this invention achieves automated detection of PIN pin position with micrometer-level high precision (±1 micrometer), full-dimensional (planar + perpendicular + coplanar), high efficiency (single product ≤100 milliseconds), and strong adaptability (parametric configuration).

[0062] Figure 2 A schematic diagram of the PIN pin position detection system structure 200 provided in this application embodiment includes: The image acquisition module 210 is used to acquire front and side images of the PIN pin using a front camera, a side camera, a telecentric lens and a ring low-angle light source, and to establish a unified mapping relationship between pixel coordinates and physical coordinates through joint calibration of the front camera and the side camera, and to achieve spatial alignment between the front image and the side image. The image processing module 220 is used to perform noise reduction and illumination equalization processing on the acquired front and side images, and to separate the foreground region of the PIN needle from the background region in the front and side images using an image segmentation algorithm. The image preprocessing module 230 is used to perform edge detection and subpixel interpolation processing on the separated front image and side image respectively, and extract the subpixel level coordinates of the center line and endpoint of the PIN in the front image, and the subpixel level coordinates of the center line and top point in the side image. The positional calculation module 240 is used to calculate the following positional parameters of the PIN pins based on the extracted sub-pixel level features: planar positional, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the range of height deviations of the top points of all PIN pins relative to the reference plane obtained by least squares fitting. The result determination and output module 250 is used to compare the calculated position degree parameters with their respective preset tolerance thresholds to determine whether the PIN pin or its corresponding product is qualified, output the test result, and control the production line to perform corresponding actions based on the determination result.

[0063] Figure 2 The PIN pin position detection system structure 200 shown can achieve Figure 1 The PIN pin position detection method shown can be found in the following reference. Figure 1 The PIN pin position detection method of the illustrated embodiment will not be described in detail again.

[0064] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0065] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting the position of a PIN pin, characterized in that, include: A front camera, a side camera, a telecentric lens, and a ring low-angle light source are used to acquire front and side images of the PIN pin, respectively. A unified mapping relationship between pixel coordinates and physical coordinates is established through joint calibration of the front camera and the side camera, and spatial alignment of the front image and the side image is achieved. The acquired front and side images are subjected to denoising and illumination equalization processing, and the foreground region of the PIN needle in the front and side images is separated from the background region by an image segmentation algorithm. Edge detection and subpixel interpolation are performed on the separated front and side images respectively to extract the subpixel level coordinates of the center line and endpoints of the PIN in the front image, and the subpixel level coordinates of the center line and top point in the side image. Based on the extracted sub-pixel level features, the following positional parameters of the PIN pins are calculated: planar positional degree, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the height deviation range of all PIN pin top points relative to the reference plane obtained by least squares fitting. The calculated position parameters are compared with their respective preset tolerance thresholds to determine whether the PIN or its corresponding product is qualified. The test results are output, and the production line is controlled to perform corresponding actions based on the determination results.

2. The method according to claim 1, characterized in that, The optical axis of the front camera is perpendicular to the product base plane, and the optical axis of the side camera is parallel to the product base plane; the joint calibration adopts the Zhang Zhengyou calibration method, combined with the checkerboard calibration board that appears in the field of view of both cameras at the same time, to solve the extrinsic parameter matrix between the two cameras.

3. The method according to claim 1, characterized in that, When acquiring front and side images, either a stop-shoot mode or a fly-shoot mode is used. The stop-shoot mode involves moving the product to the shooting station and then stopping its movement to trigger the camera to acquire an image. The fly-shoot mode involves the product moving at a constant speed, and the encoder provides feedback on the position to trigger the camera to acquire an image at a predetermined position.

4. The method according to claim 1, characterized in that, The denoising uses a nonlocal mean denoising algorithm, and the illumination equalization uses an adaptive histogram equalization algorithm.

5. The method according to claim 1, characterized in that, The edge detection uses the Canny operator; the subpixel interpolation process is as follows: for the detected integer pixel edge points, the gray values ​​of adjacent pixels along the gradient direction are used to perform a quadratic polynomial fitting, and the extreme points of the fitted curve are used as the subpixel edge positions.

6. The method according to claim 1, characterized in that, The method for extracting the PIN needle centerline is as follows: calculate the midpoint from the sub-pixel edge points on the left and right sides, obtain a series of sub-pixel center points along the length of the PIN needle, and then generate a smooth centerline by fitting a B-spline curve. The method for extracting the endpoints of the PIN pins is as follows: extract the grayscale change curve along the centerline direction, and use the grayscale moment method or arctangent function fitting to determine the sub-pixel endpoint position.

7. The method according to claim 1, characterized in that, For images containing multiple pins, a connected component labeling algorithm is used to assign a unique identifier to each pin, and subpixel-level feature extraction is performed in parallel on each connected component.

8. The method according to claim 1, characterized in that, The verticality is calculated as follows: extract the PIN centerline from the side image and fit it as a straight line; extract multiple points from the base plane and fit them as the base plane; and calculate the angle between the PIN centerline direction vector and the base plane normal vector.

9. The method according to claim 1, characterized in that, The coplanarity is calculated as follows: obtain the three-dimensional world coordinates of the top of each pin from the side image, fit a spatial plane using the least squares method, calculate the signed distance from each top point to the fitted plane, and take the difference between the maximum distance and the minimum distance as the coplanarity.

10. A PIN pin position detection system, characterized in that, include: The image acquisition module is used to acquire front and side images of the PIN pin using a front camera, a side camera, a telecentric lens, and a ring low-angle light source, and to establish a unified mapping relationship between pixel coordinates and physical coordinates through joint calibration of the front camera and the side camera, and to achieve spatial alignment between the front image and the side image. The image processing module is used to perform noise reduction and illumination equalization processing on the acquired front and side images, and to separate the foreground region of the PIN needle from the background region in the front and side images using an image segmentation algorithm. The image preprocessing module is used to perform edge detection and subpixel interpolation on the separated front and side images respectively, and extract the subpixel level coordinates of the center line and endpoint of the PIN in the front image, and the subpixel level coordinates of the center line and top point in the side image. The positional calculation module is used to calculate the following positional parameters of the PIN pins based on the extracted sub-pixel level features: planar positional, including the planar offset of the PIN pins and the spacing deviation between adjacent PIN pins; perpendicularity, i.e., the angle between the center line of the PIN pins and the normal of the base plane; and coplanarity, i.e., the height deviation range of all PIN pin top points relative to the reference plane obtained by least squares fitting. The result determination and output module is used to compare the calculated position parameters with their respective preset tolerance thresholds to determine whether the PIN or its corresponding product is qualified, output the test results, and control the production line to perform corresponding actions based on the determination results.

Images