Terminal pin height defect and skew anomaly recognition method based on visual AI

Abstract

The application discloses a terminal PIN height defect and skew abnormality recognition method based on visual AI, relates to the field of terminal defect and abnormality recognition, and comprises the following steps: image acquisition is carried out by using a forward camera and an oblique camera, and a reference horizontal coordinate sequence and a plane coordinate reference library are acquired; a needle tip two-dimensional dot matrix coordinate set is acquired through key point detection, and a plane offset and a missing position point are extracted; a horizontal offset relative to a reference is calculated, and is converted into a height deviation through a calibration coefficient; the absolute height of each PIN needle and the coplanarity of the whole row are calculated, and the needle shrinkage, the height float and the coplanarity defect are recognized; according to the comparison result, the quality result of each PIN needle is determined, the quality determination result is output, and visual marking and report generation are completed. The application has the advantages that the linkage determination of the plane offset and the height deviation can detect various terminal PIN size defects, and effectively distinguish the overall translation and the needle body skew abnormality which are easy to confuse.

Description

Technical Field

[0001] This invention relates to the field of terminal defect and anomaly identification, specifically to a method for identifying terminal PIN height defects and skew anomalies based on visual AI. Background Technology

[0002] In the context of the rapid development of the modern precision electronics manufacturing industry, various connectors, plugs and other electronic components are widely used in many fields such as automotive electronics, intelligent equipment, consumer electronics and industrial control. As the core basic component that enables circuit conduction, signal transmission and power connection within the components, the quality of the terminal's processing and forming directly determines the overall assembly accuracy, power-on stability and long-term reliability of electronic products. The pins arranged on the terminal are precision slender structural parts. In the mass production and processing stages such as stamping, injection molding, cutting and shaping, they are easily affected by various factors such as the precision of processing equipment, mold wear, material placement deviation, conveying vibration and assembly tooling positioning error, which can lead to various forming defects such as inconsistent pin tip height, pin body skew, abnormal single pin length, missing pins and overall planar position deviation.

[0003] Currently, the industry still largely relies on the traditional method of manual visual inspection combined with simple go / no-go gauge sampling for quality inspection of terminal pins. This type of inspection is not only inefficient and difficult to match the quality inspection rhythm of large-scale continuous production lines, but also highly subjective in manual judgment, which is prone to missed or false detections and cannot achieve accurate quantitative judgment of defect size. As electronic components continue to upgrade towards miniaturization, integration, and high-density arrangement, the arrangement of terminal pins is becoming increasingly dense and the pin spacing is constantly shrinking, making the limitations of traditional manual inspection and simple fixture inspection increasingly prominent.

[0004] Machine vision inspection technology, with its advantages of being non-contact, high-precision, high-efficiency, and capable of real-time data quantification, is gradually replacing traditional quality inspection methods in the application of precision parts appearance and size inspection scenarios. Most existing conventional vision inspection solutions adopt a single top-down shooting mode, which can only complete the identification of two-dimensional appearance defects such as pin plane position and missing pins. It is difficult to accurately calculate key three-dimensional dimensional parameters such as pin tip height deviation, pin body skewness, and coplanarity of the entire row in the spatial dimension, and cannot fully cover the quality defect inspection needs of all types of terminal pins. Summary of the Invention

[0005] To address the aforementioned technical issues, this paper provides a visual AI-based method for identifying terminal pin height defects and skew anomalies. This solution overcomes the limitations of the background technology, which uses a single top-down shooting mode that can only identify two-dimensional appearance defects such as pin plane position and missing pins. It fails to accurately calculate key three-dimensional dimensional parameters such as pin tip height deviation, pin body skewness, and coplanarity of the entire row, thus failing to comprehensively cover the quality defect detection needs of all types of terminal pins.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for identifying terminal pin height defects and skew anomalies based on visual AI, comprising: Image acquisition is performed using a frontal camera and an oblique camera. A coplanar reference for the needle tip is established using a planar reference plate, and a reference horizontal coordinate sequence and a planar coordinate reference library are obtained. Based on the top-down view image, the coordinate set of the two-dimensional point matrix of the needle tip is obtained through key point detection, and the plane offset and missing position points are extracted. Based on the oblique tilted image, the horizontal coordinates of the needle tip are extracted, the horizontal offset relative to the reference is calculated, and then converted into height deviation through calibration coefficients; Calculate the absolute height of each pin and the coplanarity of the entire row based on the height deviation to identify pin shrinkage, floating height and poor coplanarity; The planar offset and height deviation are logically compared. Based on the comparison result, the quality result of each pin is determined, the quality judgment result is output, and the visualization mark and report generation are completed.

[0007] Preferably, the step of using a frontal camera and an oblique camera for image acquisition, establishing a coplanar reference for the needle tip using a planar reference plate, and obtaining the reference horizontal coordinate sequence and the planar coordinate reference library includes: A front-facing camera and an oblique-facing camera are used. The angle between the optical axis of the oblique-facing camera and the normal to the plane of the terminal base is A degrees, and the direction of the optical axis is perpendicular to the PIN pin arrangement direction. The intrinsic parameters, extrinsic parameters, and distortion coefficients of the two cameras are calibrated to obtain the camera calibration parameters; Press all the pin tips of the terminal vertically against a flat reference plate, so that all the pin tips are in contact with the surface of the reference plate and remain coplanar; Using the terminal that is in a coplanar state in contact with the surface of the reference plate as the reference object, the terminal image is acquired synchronously by a front camera and an oblique camera; The pixel coordinates of the center of each PIN needle tip are extracted from the image captured by the front camera. The pixel coordinates are then converted into planar coordinates in the world coordinate system by combining the camera calibration parameters, thus forming a planar coordinate reference library. Images acquired by an oblique camera are used to extract the horizontal pixel coordinates of each PIN pin tip in the image using a key point detection algorithm. The reference horizontal coordinate sequence is constructed according to the actual spatial arrangement order of the PIN pins on the terminal base and stored in the plane coordinate reference library.

[0008] Preferably, the step of extracting the horizontal pixel coordinates of each PIN needle tip in the image using a key point detection algorithm includes: Gaussian filtering is applied to the reference image acquired by the oblique camera to remove noise, and the needle tip region is separated from the background by adaptive threshold segmentation to complete the coarse localization of the needle tip position. The Harris corner detection algorithm is used to retrieve the point with the largest gray-level gradient change rate in the needle tip region after coarse localization, and use it as the initial feature point of the needle tip. Select a local neighborhood centered on the initial feature point, extract the gray values ​​of all pixels in the neighborhood, and calculate the gray gradient magnitude of each pixel in the horizontal and vertical directions. Using the grayscale gradient magnitude of pixels in the neighborhood as weights, the position offset of the pixel relative to the initial feature point is weighted and averaged to obtain the coordinate correction amount. Based on the coordinate correction, a unique and stable horizontal pixel coordinate is output for the tip of each PIN pin.

[0009] Preferably, the step of obtaining the two-dimensional point matrix coordinate set of the needle tip through key point detection based on the front top view image, and extracting the plane offset and missing position points includes: Place the terminal to be tested at the testing station and capture a top-view image using a front-facing camera; Based on the collected top-view image, the pixel coordinates of the center of each PIN needle tip are extracted and converted into planar coordinates in the world coordinate system to construct a two-dimensional point matrix coordinate set for the needle tip. Among them, the method of generating the two-dimensional point matrix coordinate set of the needle tip of the top view image is consistent with the planar coordinate reference library; For each PIN, calculate the Euclidean distance between the current pin tip plane coordinates and the corresponding standard coordinates in the plane coordinate reference library, define the Euclidean distance as the plane offset, and record the offset vector from the standard coordinates to the current pin tip plane coordinates; If no two-dimensional dot matrix coordinates of the pin tip are detected in the nearest neighbor region of the standard coordinates of the pin, it is determined that the pin has a missing position point. Set a plane offset threshold. If the plane offset is greater than the plane offset threshold, then the PIN pin index record will be added to the plane offset anomaly set.

[0010] Preferably, the step of extracting the horizontal coordinates of the needle tip based on the oblique tilted image, calculating the horizontal offset relative to the reference, and converting it into a height deviation through a calibration coefficient includes: Place the terminal to be tested at the testing station and acquire tilted images of the terminal to be tested using the same oblique camera parameters as when establishing the planar coordinate reference library, and extract the horizontal pixel coordinates of each PIN tip. For each PIN, calculate the difference between the current horizontal pixel coordinate of the pin tip and the corresponding reference coordinate in the reference horizontal coordinate sequence, and define this difference as the original horizontal offset. Based on the planar offset vector of the PIN obtained by the front camera, the original horizontal offset is compensated to obtain the compensated horizontal offset. The height deviation of the PIN is obtained by multiplying the compensated horizontal offset by the height-horizontal offset mapping coefficient, which is calibrated using a step calibration block.

[0011] Preferably, the step of calculating the absolute height of each pin and the coplanarity of the entire row based on the height deviation, and identifying pin retraction, floating height, and poor coplanarity, includes: With the height of the reference plate plane as the reference zero point, the relative height of each PIN pin in the current terminal to be tested is equal to its height deviation; Calculate the maximum and minimum relative heights of all pins in the current terminal to be tested to obtain the coplanarity, which is equal to the maximum relative height minus the minimum relative height. Set a height deviation threshold. For each PIN, if the absolute value of the relative height is greater than the height deviation threshold, the PIN is determined to have a length defect. The term "length defect" refers to a defect where the relative height is positive (e.g., a floating height defect) or a defect where the relative height is negative (e.g., a shrinkage defect). Set a coplanarity threshold. When the coplanarity of all pins in a row is greater than the coplanarity threshold, it is determined that the terminal has a coplanarity problem.

[0012] Preferably, the step of logically comparing the plane offset with the height deviation, and determining the quality result of each PIN pin based on the comparison result, includes: For each PIN that is not identified as missing, its planar offset and absolute value of height deviation are obtained, and both are compared with their respective preset thresholds: If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, then the PIN pin is determined to be skewed. If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the absolute value of the residual between the original horizontal offset of the oblique camera and the projection component of the plane offset vector on the oblique camera imaging plane is further calculated. If the absolute value of the residual is less than the residual threshold, the pin is determined to be a translational defect; otherwise, the pin is determined to be a skewed defect. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, the PIN is determined to be a length defect, and further classified as a shortened PIN or a floating PIN based on the sign of the height deviation. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the PIN is considered qualified.

[0013] Preferably, the step of outputting the quality assessment result and completing the visualization marking and report generation includes: Output the defect type and quantification parameters for each PIN, including: plane offset vector, height deviation, coplanarity, and defect category; On the top-view image, the location of the defect is marked with different colors, and on the tilted image, the position of the needle tip is marked with crosshairs and the height deviation is represented by color gradations. Generate inspection reports and data logs that include defect location maps, deviation value tables, and statistical reports.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This solution utilizes deep coupling between a forward-facing camera and an oblique camera. It compensates for the original horizontal offset by projecting the plane offset vector measured in the forward direction, thus obtaining height deviations caused solely by height changes and eliminating interference from plane translation on height measurement. Furthermore, for challenging cases with large plane offsets but normal height deviations, this solution introduces residual analysis. By comparing the absolute value of the residual between the original horizontal offset and the plane projection component with a residual threshold, it accurately distinguishes between overall translation and skew, including the rare case of height compensation caused by long needle skew, avoiding the problem of mutual misjudgment between translation and skew in existing technologies. Moreover, this solution calculates height using the needle tip, eliminating the need for needle root image features. This effectively solves the problem of height measurement failure caused by occlusion of the needle roots in multi-row PIN needles. It can also clearly output refined classification results for defects such as overall translation, skew, needle retraction, floating height, missing needles, and poor coplanarity. Moreover, it only requires an industrial camera and a planar reference plate, effectively reducing equipment costs and making it suitable for high-cycle online full inspection. Attached Figure Description

[0015] Figure 1 This is a flowchart of the terminal PIN height defect and skew anomaly identification method based on visual AI of the present invention; Figure 2 This invention employs a frontal camera and an oblique camera for image acquisition, establishes a coplanar reference for the needle tip using a planar reference plate, and obtains a reference horizontal coordinate sequence and a planar coordinate reference library flowchart. Figure 3 This is a flowchart illustrating the process of extracting the horizontal pixel coordinates of each PIN needle tip in an image using a key point detection algorithm, as described in this invention. Figure 4This invention is based on a top-down view image, and obtains a two-dimensional point matrix coordinate set of the needle tip through key point detection, and extracts the plane offset and missing position points. (Flowchart) Figure 5 This invention extracts the horizontal coordinates of the needle tip from an obliquely tilted image, calculates the horizontal offset relative to a reference, and converts it into a height deviation flowchart using calibration coefficients. Figure 6 This invention provides a flowchart for logically comparing planar offset and height deviation, and determining the quality of each PIN based on the comparison results. Figure 7 This is a structural diagram of the electronic device proposed in this invention; Figure 8 This is a schematic diagram of the structure of the computer-readable storage medium proposed in this invention. Detailed Implementation

[0016] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0017] Reference Figure 1 As shown, a method for identifying terminal pin height defects and skew anomalies based on visual AI includes: Image acquisition is performed using a frontal camera and an oblique camera. A coplanar reference for the needle tip is established using a planar reference plate, and a reference horizontal coordinate sequence and a planar coordinate reference library are obtained. Based on the top-down view image, the coordinate set of the two-dimensional point matrix of the needle tip is obtained through key point detection, and the plane offset and missing position points are extracted. Based on the oblique tilted image, the horizontal coordinates of the needle tip are extracted, the horizontal offset relative to the reference is calculated, and then converted into height deviation through calibration coefficients; Calculate the absolute height of each pin and the coplanarity of the entire row based on the height deviation to identify pin shrinkage, floating height and poor coplanarity; The planar offset and height deviation are logically compared. Based on the comparison result, the quality result of each pin is determined, the quality judgment result is output, and the visualization mark and report generation are completed.

[0018] It should be noted that the forward camera provides a planar offset vector of the pinpoint, while the oblique camera provides the original horizontal offset, which includes the combined effects of height changes and planar translation. By projecting the forward offset vector onto the oblique imaging plane and subtracting the projected component from the original offset, the pure horizontal offset caused solely by height changes can be decoupled and converted into a height deviation using calibration coefficients. Furthermore, for cases where the planar offset is large but the height deviation is normal, the residuals of the original offset and the projected component are compared with a threshold to distinguish between overall translation and skew. Through the deep coupling of the forward and oblique cameras, the collaborative identification of planar and three-dimensional defects is achieved.

[0019] Reference Figure 2 As shown, the process of acquiring images using a frontal camera and an oblique camera, establishing a coplanar reference for the needle tip using a planar reference plate, and obtaining the reference horizontal coordinate sequence and planar coordinate reference library includes: A front-facing camera and an oblique-facing camera are used. The angle between the optical axis of the oblique-facing camera and the normal to the plane of the terminal base is A degrees, and the direction of the optical axis is perpendicular to the PIN pin arrangement direction. The intrinsic parameters, extrinsic parameters, and distortion coefficients of the two cameras are calibrated to obtain the camera calibration parameters; Press all the pin tips of the terminal vertically against a flat reference plate, so that all the pin tips are in contact with the surface of the reference plate and remain coplanar; Using the terminal that is in a coplanar state in contact with the surface of the reference plate as the reference object, the terminal image is acquired synchronously by a front camera and an oblique camera; The pixel coordinates of the center of each PIN needle tip are extracted from the image captured by the front camera. The pixel coordinates are then converted into planar coordinates in the world coordinate system by combining the camera calibration parameters, thus forming a planar coordinate reference library. Images acquired by an oblique camera are used to extract the horizontal pixel coordinates of each PIN pin tip in the image using a key point detection algorithm. The reference horizontal coordinate sequence is constructed according to the actual spatial arrangement order of the PIN pins on the terminal base and stored in the plane coordinate reference library.

[0020] It is important to note that establishing a coplanar datum and obtaining a standard coordinate template are core prerequisites for achieving high-precision defect judgment. These directly determine the accuracy of calculations for key parameters such as plane offset, height deviation, and tilt angle. If the product's own plastic shell or uncalibrated state is directly used as the datum, factors such as clamping tilt, camera viewing angle deviation, and terminal placement angle errors can easily lead to pin tip coordinate datum offset, resulting in problems such as height misjudgment, tilt angle calculation distortion, and inaccurate plane offset judgment. Therefore, this solution constructs a physically real coplanar datum using a planar datum plate, simultaneously pressing all PIN tips onto the datum plate surface, forcing all tips to be in the same ideal plane. This serves as a unified datum for inspection. Under this datum state, a forward-facing camera is used to acquire the standard world coordinates of each PIN tip, forming a planar coordinate datum library. This provides a true basis for subsequent defect judgments such as plane offset, spacing anomalies, and missing pins. Simultaneously, an oblique camera is used to acquire the datum horizontal coordinate sequence of the tips from an oblique perspective, ensuring the consistency of the PIN pin arrangement order and index during 3D reconstruction and avoiding problems such as mismatch, sequence number disorder, and invalid correspondence between multiple PIN pins.

[0021] Reference Figure 3 As shown, the step of extracting the horizontal pixel coordinates of each PIN needle tip in the image using a key point detection algorithm includes: Gaussian filtering is applied to the reference image acquired by the oblique camera to remove noise, and the needle tip region is separated from the background by adaptive threshold segmentation to complete the coarse localization of the needle tip position. The Harris corner detection algorithm is used to retrieve the point with the largest gray-level gradient change rate in the needle tip region after coarse localization, and use it as the initial feature point of the needle tip. Select a local neighborhood centered on the initial feature point, extract the gray values ​​of all pixels in the neighborhood, and calculate the gray gradient magnitude of each pixel in the horizontal and vertical directions. Using the grayscale gradient magnitude of pixels in the neighborhood as weights, the position offset of the pixel relative to the initial feature point is weighted and averaged to obtain the coordinate correction amount. Based on the coordinate correction, a unique and stable horizontal pixel coordinate is output for the tip of each PIN pin.

[0022] It should be noted that the adaptive threshold refers to selecting a fixed-size local window centered on each pixel, calculating the grayscale mean of all pixels within the local window, and setting the dynamic segmentation threshold of the local window as the difference between the grayscale mean and a non-negative constant. The non-negative constant is calibrated based on the principle of complete segmentation of the pinhead and no background interference, and is determined through grayscale statistical experiments on multiple benchmark images. The specific method for weighted averaging using grayscale gradient magnitude as weight is as follows: calculate the horizontal weighted average offset and the vertical weighted average offset respectively. The horizontal offset is equal to the sum of the products of the horizontal coordinate difference of each pixel and the gradient magnitude of that pixel, divided by the total gradient magnitude. The vertical offset is calculated similarly. Add the initial feature point coordinates to the two offsets to obtain the needle tip coordinates.

[0023] Reference Figure 4 As shown, the step of obtaining a two-dimensional point matrix coordinate set of the needle tip through key point detection based on a frontal top view image, and extracting the planar offset and missing position points includes: Place the terminal to be tested at the testing station and capture a top-view image using a front-facing camera; Based on the collected top-view image, the pixel coordinates of the center of each PIN needle tip are extracted and converted into planar coordinates in the world coordinate system to construct a two-dimensional point matrix coordinate set for the needle tip. Among them, the method of generating the two-dimensional point matrix coordinate set of the needle tip of the top view image is consistent with the planar coordinate reference library; For each PIN, calculate the Euclidean distance between the current pin tip plane coordinates and the corresponding standard coordinates in the plane coordinate reference library, define the Euclidean distance as the plane offset, and record the offset vector from the standard coordinates to the current pin tip plane coordinates; If no two-dimensional dot matrix coordinates of the pin tip are detected in the nearest neighbor region of the standard coordinates of the pin, it is determined that the pin has a missing position point. Set a plane offset threshold. If the plane offset is greater than the plane offset threshold, then the PIN pin index record will be added to the plane offset anomaly set.

[0024] It should be noted that the top-view image can intuitively reflect the planar arrangement of the terminal pins. The accuracy of the pin tip two-dimensional dot matrix coordinates directly determines the reliability of the judgment of planar offset and missing defects. To ensure the consistency of the benchmark, the generation method of the pin tip two-dimensional dot matrix coordinate set of the top-view image to be inspected is consistent with the planar coordinate benchmark library to eliminate the errors caused by coordinate transformation and positioning algorithms. The planar offset is obtained by calculating the Euclidean distance between the current coordinates and the standard coordinates, realizing the quantitative representation of the planar offset. By searching for valid pin tip coordinates in the nearest neighbor domain of the standard coordinates, it is determined whether there is a missing defect. Based on the planar offset threshold, the pins with offset exceeding the tolerance are screened and the index is recorded to complete the accurate identification of planar offset and missing defects. The plane offset threshold is equal to the allowable tolerance of the plane position in the product design requirements minus the theoretical maximum error of the system, where the theoretical maximum error of the system is the sum of the absolute values ​​of the camera calibration error, coordinate transformation error and reference calibration error.

[0025] Reference Figure 5As shown, the process of extracting the horizontal coordinates of the needle tip based on the obliquely tilted image, calculating the horizontal offset relative to the reference, and converting it into a height deviation through a calibration coefficient includes: Place the terminal to be tested at the testing station and acquire tilted images of the terminal to be tested using the same oblique camera parameters as when establishing the planar coordinate reference library, and extract the horizontal pixel coordinates of each PIN tip. For each PIN, calculate the difference between the current horizontal pixel coordinate of the pin tip and the corresponding reference coordinate in the reference horizontal coordinate sequence, and define this difference as the original horizontal offset. Based on the planar offset vector of the PIN obtained by the front camera, the original horizontal offset is compensated to obtain the compensated horizontal offset. The height deviation of the PIN is obtained by multiplying the compensated horizontal offset by the height-horizontal offset mapping coefficient, which is calibrated using a step calibration block.

[0026] It should be noted that, based on the principle of triangulation, the height deviation of the needle tip is converted into the horizontal pixel offset in the tilted image: when the camera takes a picture at a fixed tilt angle, the change in the height of the needle tip is linearly related to its horizontal coordinate in the image; by using a coplanar reference, that is, all needle tips are flattened to zero height, the reference horizontal coordinates of each needle are obtained, and then the original horizontal offset of the needle tip in the tilted image of the terminal under test is measured. The actual height deviation can be calculated using a pre-calibrated mapping coefficient, so that the needle root does not need to be seen, and the height information can be obtained only from the visible needle tip, and key data is provided for subsequent coplanarity calculation and collaborative classification of translation, skew, and length defects; The compensation for the original horizontal offset refers to: using the planar offset vector of the PIN obtained by the front camera, calculating the projection component of the planar offset vector on the imaging plane of the oblique camera, subtracting the projection component from the original horizontal offset, and obtaining the compensated horizontal offset caused only by the change in height. The height-to-horizontal offset mapping coefficient is calibrated using a step calibration block. Specifically, this involves selecting a step calibration block with two known height differences, denoting the height difference as the calibration height difference, capturing a tilted image of the calibration block under imaging conditions identical to the actual detection, measuring the horizontal pixel offset difference between the two steps in the image, dividing the calibration height difference by the horizontal pixel offset difference, and obtaining the quotient as the mapping coefficient. After calibration, the mapping coefficient is stored in the system parameters.

[0027] The process of calculating the absolute height of each pin and the coplanarity of the entire row based on height deviation, and identifying pin retraction, floating height, and coplanarity defects, includes: With the height of the reference plate plane as the reference zero point, the relative height of each PIN pin in the current terminal to be tested is equal to its height deviation; Calculate the maximum and minimum relative heights of all pins in the current terminal to be tested to obtain the coplanarity, which is equal to the maximum relative height minus the minimum relative height. Set a height deviation threshold. For each PIN, if the absolute value of the relative height is greater than the height deviation threshold, the PIN is determined to have a length defect. The term "length defect" refers to a defect where the relative height is positive (e.g., a floating height defect) or a defect where the relative height is negative (e.g., a shrinkage defect). Set a coplanarity threshold. When the coplanarity of all pins in a row is greater than the coplanarity threshold, it is determined that the terminal has a coplanarity problem.

[0028] It should be noted that, with the planar reference plate as the zero height reference, the height deviation of each pin is directly defined as the relative height of that pin. Since all pin tips are coplanar in the reference state, that is, the height is zero, the sign and magnitude of the relative height directly reflect the expansion and contraction state of a single pin relative to the reference plane: a relative height greater than 0 indicates floating height, which is manifested as a protruding pin tip, and a relative height less than 0 indicates retraction of the pin, which is manifested as a concave pin tip. By calculating the range of the relative heights of all pin tips, the degree of inconsistency in the height of the entire row of pin tips can be quantified, that is, the coplanarity. By transforming the raw height deviation data into intuitive physical defect judgment results, it clearly distinguishes between float height, needle shrinkage and coplanarity defects. Among them, single needle judgment serves subsequent defect classification, while coplanarity judgment provides array-level quality assessment. The height deviation threshold is directly set based on the allowable deviation of single needle height specified in the product technical specifications. For example, when the design requires a single needle height tolerance of ±0.05mm, the height deviation threshold is set to 0.05mm. The coplanarity threshold is directly set based on the maximum allowable coplanarity of the entire row of needle tips specified in the product technical specifications. For example, when the design requires a coplanarity not exceeding 0.08mm, the coplanarity threshold is set to 0.08mm.

[0029] Reference Figure 6 As shown, the logical comparison between the plane offset and the height deviation, and the determination of the quality of each PIN pin based on the comparison result, include: For each PIN that is not identified as missing, its planar offset and absolute value of height deviation are obtained, and both are compared with their respective preset thresholds: If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, then the PIN pin is determined to be skewed. If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the absolute value of the residual between the original horizontal offset of the oblique camera and the projection component of the plane offset vector on the oblique camera imaging plane is further calculated. If the absolute value of the residual is less than the residual threshold, the pin is determined to be a translational defect; otherwise, the pin is determined to be a skewed defect. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, the PIN is determined to be a length defect, and further classified as a shortened PIN or a floating PIN based on the sign of the height deviation. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the PIN is considered qualified.

[0030] It should be noted that the absolute value of the residual is equal to the original horizontal offset minus the absolute value of the projected component, and the residual threshold is set according to three times the standard deviation of the repeated measurement noise. Traditional methods cannot distinguish between overall translation and skew based solely on planar offset, which can easily lead to misjudging translation as skew, resulting in incorrect repairs or scrapping. This solution introduces orthogonal information such as height deviation and combines it with residual analysis of the original oblique horizontal offset and the planar offset projection components to uniquely determine the defect type and achieve accurate classification. Planar offset reflects the positional change of the needle tip in the horizontal plane, while height deviation reflects the vertical displacement of the needle tip relative to the reference plane. Different combinations of these two physical quantities, supplemented by residual judgment, correspond to different defect modes: Overall translation defect: The needle body remains vertical but is translated as a whole. At this time, the plane offset is greater than the plane offset threshold, the absolute value of the height deviation is not greater than the height deviation threshold, and the absolute value of the residual between the original horizontal offset and the plane offset projection component is less than the residual threshold. That is, the horizontal offset observed obliquely is explained by the plane translation. Skewness defect: The needle body tilts, which can be divided into two situations: Typical skewness is accompanied by a change in needle tip height, that is, the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold. Usually, a negative height deviation indicates that the needle tip is lowered; Atypical skewness, such as the skewness of a long needle leading to height compensation, is manifested as the plane offset being greater than the plane offset threshold, the absolute value of the height deviation not being greater than the height deviation threshold, but the absolute value of the residual between the original horizontal offset and the plane offset projection component is greater than or equal to the residual threshold. Both of the above situations are judged as skewness defects. Length and shortness defects: The needle body is vertical and the position is correct, but the height is abnormal. At this time, the plane offset is not greater than the plane offset threshold, and the absolute value of the height deviation is greater than the height deviation threshold. Based on the sign of the height deviation, it can be further distinguished as needle shrinkage or floating. Qualified: The plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold; In addition, this solution also identifies the following two types of defects: Missing defects: Determined by key point detection in the top-view image. If no two-dimensional point matrix coordinates of the needle tip are detected in the nearest neighbor region of the standard coordinates of a certain PIN needle, then the needle is determined to have a missing defect. Coplanarity failure: Determined by calculating the coplanarity of the entire row. Calculate the difference between the maximum and minimum relative heights of all pins of the current terminal, i.e., coplanarity. If the difference is greater than the coplanarity threshold, the terminal is determined to have coplanarity failure, regardless of whether a single pin is qualified. Through this classification logic, this solution can accurately distinguish between products with overall translation, skewing, needle shrinkage, floating, missing parts, poor coplanarity, and qualified products, thus avoiding misjudgment.

[0031] The output quality assessment results and the completion of visualization marking and report generation include: Output the defect type and quantification parameters for each PIN, including: plane offset vector, height deviation, coplanarity, and defect category; On the top-view image, the location of the defect is marked with different colors, and on the tilted image, the position of the needle tip is marked with crosshairs and the height deviation is represented by color gradations. Generate inspection reports and data logs that include defect location maps, deviation value tables, and statistical reports.

[0032] Furthermore, the method according to the embodiments of this application can also be achieved by means of... Figure 7 The architecture of the electronic device shown is used to implement this. For example... Figure 7 As shown, the electronic device 500 may include a bus 501, one or more CPUs 502, a read-only memory (ROM) 503, a random access memory (RAM) 504, a communication port 505 connected to a network, an input / output component 506, a hard disk 507, etc. The storage device in the electronic device 500, such as the ROM 503 or the hard disk 507, may store a visual AI-based terminal pin height defect and skew anomaly identification method provided in this application. The electronic device 500 may also include a user interface 508. Of course, Figure 7 The architecture shown is merely exemplary and can be omitted as needed when implementing different devices. Figure 7 One or more components in the illustrated electronic device.

[0033] Figure 8 This is a schematic diagram of a computer-readable storage medium structure provided in one embodiment of this application. Figure 8The diagram illustrates a computer-readable storage medium 600 according to one embodiment of this application. The computer-readable storage medium 600 stores computer-readable instructions. When executed by a processor, the computer-readable instructions can perform a visual AI-based terminal pin height defect and skew anomaly identification method according to an embodiment of this application, as described with reference to the above figures. The storage medium 600 includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0034] In summary, the advantages of this invention are: by deeply coupling the frontal and oblique cameras and using the linkage of planar offset and height deviation for judgment, it can detect various terminal pin size defects and effectively distinguish between easily confused overall translational and pin body skew abnormalities.

[0035] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for identifying terminal pin height defects and skew anomalies based on visual AI, characterized in that, include: Image acquisition is performed using a frontal camera and an oblique camera. A coplanar reference for the needle tip is established using a planar reference plate, and a reference horizontal coordinate sequence and a planar coordinate reference library are obtained. Based on the top-down view image, the coordinate set of the two-dimensional point matrix of the needle tip is obtained through key point detection, and the plane offset and missing position points are extracted. Based on the oblique tilted image, the horizontal coordinates of the needle tip are extracted, the horizontal offset relative to the reference is calculated, and then converted into height deviation through calibration coefficients; Calculate the absolute height of each pin and the coplanarity of the entire row based on the height deviation to identify pin shrinkage, floating height and poor coplanarity; The planar offset and height deviation are logically compared. Based on the comparison result, the quality result of each pin is determined, the quality judgment result is output, and the visualization mark and report generation are completed.

2. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 1, characterized in that, The process of acquiring images using both a frontal and oblique camera, establishing a coplanar reference for the needle tip using a planar reference plate, and obtaining the reference horizontal coordinate sequence and planar coordinate reference library includes: A front-facing camera and an oblique-facing camera are used. The angle between the optical axis of the oblique-facing camera and the normal to the plane of the terminal base is A degrees, and the direction of the optical axis is perpendicular to the PIN pin arrangement direction. The intrinsic parameters, extrinsic parameters, and distortion coefficients of the two cameras are calibrated to obtain the camera calibration parameters; Press all the pin tips of the terminal vertically against a flat reference plate, so that all the pin tips are in contact with the surface of the reference plate and remain coplanar; Using the terminal that is in a coplanar state in contact with the surface of the reference plate as the reference object, the terminal image is acquired synchronously by a front camera and an oblique camera; The pixel coordinates of the center of each PIN needle tip are extracted from the image captured by the front camera. The pixel coordinates are then converted into planar coordinates in the world coordinate system by combining the camera calibration parameters, thus forming a planar coordinate reference library. Images acquired by an oblique camera are used to extract the horizontal pixel coordinates of each PIN pin tip in the image using a key point detection algorithm. The reference horizontal coordinate sequence is constructed according to the actual spatial arrangement order of the PIN pins on the terminal base and stored in the plane coordinate reference library.

3. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 2, characterized in that, The step of extracting the horizontal pixel coordinates of each PIN needle tip in the image using a key point detection algorithm includes: Gaussian filtering is applied to the reference image acquired by the oblique camera to remove noise, and the needle tip region is separated from the background by adaptive threshold segmentation to complete the coarse localization of the needle tip position. The Harris corner detection algorithm is used to retrieve the point with the largest gray-level gradient change rate in the needle tip region after coarse localization, and use it as the initial feature point of the needle tip. Select a local neighborhood centered on the initial feature point, extract the gray values ​​of all pixels in the neighborhood, and calculate the gray gradient magnitude of each pixel in the horizontal and vertical directions. Using the grayscale gradient magnitude of pixels in the neighborhood as weights, the position offset of the pixel relative to the initial feature point is weighted and averaged to obtain the coordinate correction amount. Based on the coordinate correction, a unique and stable horizontal pixel coordinate is output for the tip of each PIN pin.

4. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 2, characterized in that, The process of obtaining a two-dimensional point matrix coordinate set for the needle tip based on a frontal top-view image through key point detection, and extracting planar offset and missing position points includes: Place the terminal to be tested at the testing station and capture a top-view image using a front-facing camera; Based on the collected top-view image, the pixel coordinates of the center of each PIN needle tip are extracted and converted into planar coordinates in the world coordinate system to construct a two-dimensional point matrix coordinate set for the needle tip. Among them, the method of generating the two-dimensional point matrix coordinate set of the needle tip of the top view image is consistent with the planar coordinate reference library; For each PIN, calculate the Euclidean distance between the current pin tip plane coordinates and the corresponding standard coordinates in the plane coordinate reference library, define the Euclidean distance as the plane offset, and record the offset vector from the standard coordinates to the current pin tip plane coordinates; If no two-dimensional dot matrix coordinates of the pin tip are detected in the nearest neighbor region of the standard coordinates of the pin, it is determined that the pin has a missing position point. Set a plane offset threshold. If the plane offset is greater than the plane offset threshold, then the PIN pin index record will be added to the plane offset anomaly set.

5. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 2, characterized in that, The process of extracting the horizontal coordinates of the needle tip based on the obliquely tilted image, calculating the horizontal offset relative to the reference, and converting it into a height deviation using a calibration coefficient includes: Place the terminal to be tested at the testing station and acquire tilted images of the terminal to be tested using the same oblique camera parameters as when establishing the planar coordinate reference library, and extract the horizontal pixel coordinates of each PIN tip. For each PIN, calculate the difference between the current horizontal pixel coordinate of the pin tip and the corresponding reference coordinate in the reference horizontal coordinate sequence, and define this difference as the original horizontal offset. Based on the planar offset vector of the PIN obtained by the front camera, the original horizontal offset is compensated to obtain the compensated horizontal offset. The height deviation of the PIN is obtained by multiplying the compensated horizontal offset by the height-horizontal offset mapping coefficient, which is calibrated using a step calibration block.

6. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 5, characterized in that, The process of calculating the absolute height of each pin and the coplanarity of the entire row based on height deviation, and identifying pin retraction, floating height, and coplanarity defects, includes: With the height of the reference plate plane as the reference zero point, the relative height of each PIN pin in the current terminal to be tested is equal to its height deviation; Calculate the maximum and minimum relative heights of all pins in the current terminal to be tested to obtain the coplanarity, which is equal to the maximum relative height minus the minimum relative height. Set a height deviation threshold. For each PIN, if the absolute value of the relative height is greater than the height deviation threshold, the PIN is determined to have a length defect. The term "length defect" refers to a defect where the relative height is positive and a defect where the needle is retracted. Set a coplanarity threshold. When the coplanarity of all pins in a row is greater than the coplanarity threshold, it is determined that the terminal has a coplanarity problem.

7. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 6, characterized in that, The step of logically comparing the plane offset with the height deviation, and determining the quality of each PIN pin based on the comparison result, includes: For each PIN that is not identified as missing, its planar offset and absolute value of height deviation are obtained, and both are compared with their respective preset thresholds: If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, then the PIN pin is determined to be skewed. If the plane offset is greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the absolute value of the residual between the original horizontal offset of the oblique camera and the projection component of the plane offset vector on the oblique camera imaging plane is further calculated. If the absolute value of the residual is less than the residual threshold, the PIN is determined to be a global translation defect; otherwise, the PIN is determined to be a skew defect. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is greater than the height deviation threshold, the PIN is determined to be a length defect, and further classified as a shortened PIN or a floating PIN based on the sign of the height deviation. If the plane offset is not greater than the plane offset threshold and the absolute value of the height deviation is not greater than the height deviation threshold, then the PIN is considered qualified.

8. The method for identifying terminal PIN height defects and skew anomalies based on visual AI according to claim 7, characterized in that, The output quality assessment results and the completion of visualization marking and report generation include: Output the defect type and quantification parameters for each PIN, including: plane offset vector, height deviation, coplanarity, and defect category; On the top-view image, the location of the defect is marked with different colors, and on the tilted image, the position of the needle tip is marked with crosshairs and the height deviation is represented by color gradations. Generate inspection reports and data logs that include defect location maps, deviation value tables, and statistical reports.

9. An electronic device, characterized in that, include: At least one processor; And, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a visual AI-based terminal PIN height defect and skew anomaly identification method as described in any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a visual AI-based method for identifying terminal PIN height defects and skew anomalies, as described in any one of claims 1-8.

Images