Electronic component pin defect detection method and system based on machine vision

Abstract

The invention discloses an electronic component pin defect detection method and system based on machine vision, and aims to solve the problems of poor robustness and insufficient adaptive capacity of the existing automatic detection technology. According to the method, after a pin region is extracted through threshold segmentation and morphological optimization, screening is carried out by adopting composite features of area and slenderness, so that the sensitivity of a traditional method to the pose change of a component is avoided, and the robustness of pin target identification is remarkably improved. Furthermore, a dynamic reference is established according to the median of the sizes of the tested pins in the current batch, adaptive defect judgment is performed, and the core defect of poor adaptability to working condition change caused by dependence on a fixed threshold in the prior art is overcome. According to the invention, accurate and stable automatic detection of pin inclination and incomplete defects without frequent manual intervention is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automation production and quality control, in particular to an online visual inspection method and system applied in the production process of electronic components for automatically detecting the tilt and defect of metal pins. BACKGROUND

[0002] The pin of electronic component is the key structure to realize electrical connection, and the integrity of its physical form is crucial to product reliability. In the production and assembly process, the pin is prone to tilt, breakage or defect due to mechanical stress and other factors, which is one of the main reasons for circuit connection failure and economic loss.

[0003] Currently, the industry has gradually shifted from traditional manual visual inspection to automated detection technology based on machine vision. However, the existing technology still has the following problems: 1. Insufficient robustness of image preprocessing: uneven illumination, noise interference and other factors in industrial field lead to unstable image quality, affecting the subsequent processing effect; 2. Target segmentation is easily disturbed: factors such as reflection and shadow lead to incomplete binarization segmentation results, and there are breaks or holes in the pin area; 3. Target recognition is sensitive to pose changes: traditional feature selection methods are difficult to effectively distinguish pins from interference, and have poor adaptability to rotation, translation and other changes; 4. Defect judgment lacks self-adaptive ability: relying on fixed threshold, unable to adapt to product batch differences and working condition changes, frequent manual intervention is required for parameter adjustment.

[0004] Therefore, there is an urgent need in the field for a new automated detection method that can overcome the above-mentioned defects to achieve accurate and stable identification of electronic component pin defects. SUMMARY

[0005] To solve the above technical problems, the present application provides an electronic component pin defect detection method based on machine vision, which comprises the following steps: S1: image acquisition and preprocessing; S2: target segmentation and morphological optimization; S3: accurate selection based on composite features; S4: feature extraction and defect judgment.

[0006] Further, step S1 comprises: S1-1: build a visual inspection hardware platform; S1-2: image acquisition and grayscale; S1-3: image smoothing and denoising.

[0007] Further, step S2 comprises: S2-1: image binarization segmentation based on fixed grayscale threshold range; S2-2: Optimize the binary image by using morphological open and close operations to repair holes, bridge breaks and remove noise.

[0008] Further, step S3 comprises: S3-1: Perform connected component analysis on the foreground region after morphological optimization; S3-2: Filter out the real pin region based on the composite features of area and anisometry.

[0009] Further, step S4 comprises: S4-1: Extract the geometric features (area, principal axis angle) of the pin region after filtering; S4-2: Sort the pin region and calculate the area median of the current batch as a dynamic reference; S4-3: Traverse each pin based on the dynamic reference and absolute angle deviation to adaptively judge the defects of breakage and tilt.

[0010] Further, the application also provides an electronic component pin defect detection system, comprising: An image acquisition unit for acquiring images of the components to be tested; A processing unit connected to the image acquisition unit for executing image processing and analysis algorithms; An illumination unit for providing uniform illumination for the pins to be tested; Wherein, the processing unit is configured to execute the above method.

[0011] Compared with the prior art, the application has the following beneficial effects: 1. The image preprocessing stability is significantly improved, laying a high-quality foundation for subsequent processing By providing uniform and shadow-free illumination with a ring-shaped LED light source, the interference of uneven ambient light, reflection and shadow is greatly weakened from the source, ensuring the consistency of image acquisition quality, which is a prerequisite for high robustness detection. On the basis of gray-scale uniform data dimension, a circular structural element with a radius of 3 is used for median filtering. This parameter setting can effectively suppress salt and pepper noise while optimally preserving the sharpness of the pin edge, avoiding the edge blur problem that may be caused by mean filtering or Gaussian filtering, and reserving key details for subsequent accurate segmentation.

[0012] 2. The target segmentation anti-interference and repair capability is enhanced, and the adaptability is strong The fixed gray threshold range of [26, 255] is used for binarization, which is calibrated by experiments and can robustly cover the gray distribution of normal pins under uniform illumination, ensuring complete extraction of the foreground target. Innovatively, a morphological optimization strategy of "opening operation first and then closing operation" with differentiated radius parameters is adopted: using a circular structural element with a radius of 10 for opening operation can effectively remove isolated noise points with an area smaller than the structural element; then using a circular structural element with a larger radius (32) for closing operation can not only fill small holes inside the pin, but also effectively "bridge" the pin area broken due to local reflection, restoring the morphological integrity of the pin. This "small opening and large closing" parameter combination is optimized for common morphological defects of pins and has strong generalization ability.

[0013] 3. High target recognition robustness, insensitive to pose changes The introduction of the composite feature screening condition of "area [1000, 5000] and thin length > 2.0" constitutes an efficient pin recognition filter: the area threshold effectively filters out small (such as noise) or large (such as background foreign matter) interference areas. The thin length feature (defined as the ratio of the equivalent ellipse length to the short semi-axis) can accurately capture the essential morphological feature of the pin, which is "thin and long". By setting the lower limit to 2.0, it can accurately distinguish between thin and long pins and interference objects (such as pads, stains) that are close in shape to circles or squares. This feature is naturally invariant to the rotation and translation of the target, fundamentally overcoming the sensitivity of traditional methods (such as simple width-height ratio) to the pose changes of components.

[0014] 4. Strong defect judgment adaptability, no need for manual parameter adjustment, and accurate result quantization Defect Judgment: The median area of ​​all pins in the current batch is used as a dynamic benchmark, rather than a fixed threshold. This method automatically adapts to overall size variations caused by different product batches and camera magnifications. Combined with a scaling factor γ=0.7, a reasonable quantitative standard for "significantly smaller" is defined, effectively detecting genuine defective pins while avoiding misjudgments due to normal size fluctuations, offering better fault tolerance. Tilt Defect Judgment: The principal axis angle is extracted by calculating the second-order central moment of the pin area. This method has clear physical meaning and strong noise resistance. By taking the absolute value and comparing it with 90°, and setting an angle tolerance threshold δ=25°, the judgment standard for left and right tilt is unified, ensuring the consistency of the detection logic. This threshold effectively distinguishes between slight skew allowed by the process and tilt defects that require judgment. Dynamic Benchmark Extensibility: Furthermore, this invention proposes an alternative judgment criterion based on statistical interquartile range, using the data's own distribution (e.g., Q1 - 1.5IQR) to define the lower limit of outliers, further improving the method's statistical adaptability to non-uniform size fluctuations within batches. In summary, the defect judgment module of this invention achieves fully automated parameter adaptation and decision-making, and outputs objective and quantitative results, providing reliable data support for closed-loop management of production quality control (such as automatic sorting and process capability analysis). (See attached figures.) Figure 1 This is a schematic diagram of the system hardware configuration; Figure 2 This is a schematic diagram of the defect types of the pin under test (Example 1). Figure 3 This is a schematic diagram of the defect type of the pin under test (Example 2); Figure 4 This is a flowchart of the defect detection algorithm; Figure 5 This is a schematic diagram illustrating the effects of each stage of image processing; Figure 6 This is a schematic diagram of the overall pin defect detection system. Detailed Implementation

[0015] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0016] Example 1 A machine vision-based method for detecting pin defects in electronic components, referring to... Figure 3 The algorithm flow includes the following steps: S1: Image Acquisition and Preprocessing S1-1: Building a visual inspection hardware platform Reference Figure 1 An image acquisition unit, processing unit, illumination unit, and test material position are configured on a stable experimental stand. The image acquisition unit is a Hikrobot MV-CU060-10GC GigE interface industrial camera, the processing unit is an industrial computer, and the illumination unit is a ring LED light source to provide a uniform, shadowless illumination environment for the test pin. All hardware units are connected to an industrial computer that serves as the processing and control unit.

[0017] S1-2: Image Acquisition and Grayscale Conversion The image acquisition unit acquires a raw RGB color image of the pin under test. To eliminate color information interference and unify the image data dimensions, the processing unit calls the rgb_to_gray operator to convert the image into an 8-bit single-channel grayscale image.

[0018] S1-3: Image smoothing and noise reduction The processing unit calls the `median_image` operator to perform median filtering on the grayscale image using a circular structuring element with a radius of 3, effectively suppressing any random salt-and-pepper noise that may exist in the image, resulting in a clear and smooth preprocessed image. Median filtering, in its core idea, replaces the original gray value of a pixel with the median gray value within its neighborhood. For a given pixel... and its neighboring windows (For example, a 3×3 matrix), its output grayscale value The calculation process can be defined by the following formula:

[0019] in, These are the original gray values ​​of each point within the neighborhood window W. This method effectively filters out salt-and-pepper noise and, compared to mean filtering, better preserves the edge details of the image.

[0020] S2: Target Segmentation and Morphological Optimization S2-1: Segmentation threshold The processing unit calls the threshold operator to set a fixed grayscale threshold range of [26, 255], and the process is precisely defined using the following formula.

[0021]

[0022] in: This represents the input image (preprocessed grayscale image) at pixel coordinates. The grayscale value at that location.

[0023] The binarized image output represents the corresponding coordinate points. The pixel value at the location. In this invention, pixel value 1 represents the foreground target (i.e., the pin area), and pixel value 0 represents the background.

[0024] It is a preset minimum grayscale threshold used to define the darkest part of the target.

[0025] It is the preset maximum grayscale threshold, used to define the brightest part of the target.

[0026] This function evaluates each pixel of the input image: if its grayscale value... In If a pixel is within a closed interval, it is classified as a foreground object; otherwise, it is classified as background.

[0027] In a specific embodiment of the present invention, according to experimental calibration, the optimal values ​​of the above parameters are: =26 =255 In specific embodiments of the present invention, the above-described fixed threshold segmentation method is preferably used. As an alternative optimization scheme, to cope with complex lighting changes in industrial environments, an adaptive threshold segmentation method based on local statistical features can also be used. The calculation formula is as follows:

[0028] in: It is a pixel. The dynamic threshold.

[0029] and These are two groups of pixels, each with a size of [size missing]. The mean and standard deviation of gray levels within the neighborhood window.

[0030] It is the maximum possible standard deviation of the grayscale dynamic range (typically 128 for 8-bit images).

[0031] It is an adjustable correction factor, typically ranging from -0.2 to 0.2, used to fine-tune the sensitivity of the threshold.

[0032] The final binarization result of this pixel Determined by the following formula:

[0033] in It is a pixel. The original grayscale value. This adaptive method can effectively overcome the problem of local over-brightness or under-brightness caused by uneven illumination, and significantly improve the accuracy of segmentation. S2-2: Optimized Morphology To repair internal holes and external noise that may be caused by reflection in the binarized region, and to integrate the broken areas, opening and closing operations are performed on the foreground region sequentially.

[0034] The processing unit calls the `opening_circle` operator to perform an opening operation using a circular structuring element with a radius of 10, removing isolated small bright spots and contour burrs. Subsequently, the processing unit calls the `closing_circle` operator to perform a closing operation using a circular structuring element with a radius of 32, filling the holes inside the pin region and bridging broken parts, ultimately obtaining a pin region with a complete shape and clear edges. The opening and closing operations are standard morphological operations performed on the binary image A based on a circular structuring element B, such as... Figure 4 The mathematical definitions of the effects at each stage are as follows:

[0035]

[0036] 1. Opening Operation: The opening operation is a process that first performs an erosion operation on an image, and then performs a dilation operation on the result. This operation can effectively remove isolated small targets (noise) that cannot accommodate the structuring element B and smooth the contours of objects.

[0037] 2. Closing Operation: The closing operation is a process that first dilates the image and then erodes the result. This operation can effectively fill small holes inside the target area and bridge adjacent broken sections.

[0038] Formula explanation: : Represents the input binarized image region.

[0039] : Represents the structure element used for calculation (in this embodiment, it is a circle).

[0040] In a specific embodiment of the present invention, an opening operation (structuring element) is first performed on the foreground region. (with a radius of 10), then perform the closing operation (structuring element). The radius is 32).

[0041] S3: Precise Filtering Based on Composite Features S3-1: Analyzing Connected Components The processing unit calls the connection operator to perform connected component analysis on the morphologically optimized foreground region, identifying each independent and unconnected region as a separate object.

[0042] S3-2: Screening for composite features The `select_shape` operator is invoked by the processing unit, setting the composite filtering conditions as follows: the 'area' feature value of the region must be within the range of [1000, 5000] pixels, and the 'anisometry' feature value must be greater than 2.0. The anisometry feature is a geometric parameter used to measure the elongation of the region, and its mathematical model can be expressed as:

[0043] in: This represents the length of the semi-major axis of the equivalent ellipse.

[0044] This represents the length of the minor semi-axis of the equivalent ellipse.

[0045] The “slenderness” feature is designed to describe the elongation characteristics of the region, distinguishing it from the traditional aspect ratio.

[0046] The elongation value of a region is greater than or equal to 1. When the value is close to 1, it indicates that the shape of the region is close to a circle or square; the larger the value, the more elongated the shape of the region.

[0047] In this invention, by setting the minimum threshold for slenderness to 2.0, pin targets with slender morphological characteristics can be very effectively distinguished from interference noise or reflective areas that may exist in the production image but have similar area but are more rounded and plump, thereby greatly improving the accuracy and robustness of target screening.

[0048] S4: Feature Extraction and Defect Assessment S4-1: Extracting Geometric Features For the four selected pin regions, the processing unit calls the `area_center` operator to extract their respective areas and stores them in an area tuple `Areas`; it then calls the `smallest_rectangle2` operator to extract their respective principal axis angles and stores them in an angle tuple `Phis`. The principal axis angles extracted in these steps... This is obtained by calculating the second-order central moment of the pin region. For a segmented pin region R, the steps for calculating its principal axis angle are as follows: 1. Calculate the zeroth moment (i.e., area A) and first moment of the region to determine its centroid coordinates. :

[0049] 2. Second central moments of the computational domain and :

[0050]

[0051]

[0052] 3. Finally, the principal axis angle Φ (i.e., the angle between the major axis of the equivalent ellipse of the region and the x-axis) is calculated by the following formula:

[0053] The angle value Φ obtained by the above method has the advantages of clear physical meaning and high repeatability, providing accurate and reliable data input for subsequent tilt determination. S4-2: Data Sorting and Benchmark Calculation The processing unit calls the `tuple_sort_index` operator to sort the pins from left to right according to their column coordinates (Cols), obtaining the index tuples `Indices`. The processing unit then calls the `tuple_sort` operator to sort the area tuples `Areas`, selecting the median as the size reference for determining defects (AreaMedian).

[0054] S4-3: Traversal and Defect Judgment A loop is established for traversal and defect judgment, iterating through each sorted pin. This invention uses an adaptive relative comparison method to judge defects, not relying on fixed standard values, but using the size distribution of all tested pins in the current batch as a dynamic benchmark.

[0055] The mathematical model for this judgment criterion is expressed as follows:

[0056] in: : This represents the logical judgment result of the defect. Its value is a binary flag, where 1 indicates that a defect exists according to this criterion, and 0 indicates that the defect does not exist.

[0057] : Represents the actual pixel area of ​​the i-th pin currently being detected. This value is obtained by geometrically measuring the selected pin area.

[0058] This represents the median area calculated from all successfully identified pins in the current image. This value serves as a dynamic dimensional benchmark, representing the normal dimensional level for the current batch of products.

[0059] γ: This is a preset area ratio threshold, which physically defines the tolerance for the concept of "significantly smaller than". In a specific embodiment of the present invention, experimental verification shows that the optimal value of this threshold is 0.7.

[0060] Secondly, tilt defects are identified. To achieve accurate assessment of the pin's spatial orientation, this invention employs a quantitative judgment method based on the principal axis angle. This method first uses the smallest_rectangle2 operator to calculate the oriented bounding rectangle that can completely enclose each pin region with the smallest area, and then extracts the angle between the major axis of this rectangle and the horizontal axis as the principal axis angle characterizing the pin's orientation.

[0061] To ensure that the judgment is unaffected by the pin tilt direction (i.e., positive or negative angle), this invention first takes the absolute value of the measured spindle angle when calculating the deviation, and then calculates its absolute deviation from the standard vertical angle (90°). This processing method unifies the judgment benchmark and ensures that the algorithm has equal detection sensitivity for different tilt directions.

[0062] The mathematical model for this judgment criterion is expressed as follows:

[0063] : This represents the logical judgment result of the tilt defect. Its value is a binary flag, where 1 indicates that a tilt defect exists according to this criterion, and 0 indicates that the defect does not exist.

[0064] : Represents the major axis angle of the i-th pin currently being detected. This value is obtained by calculating the angle between the major axis and the horizontal axis of the oriented bounding rectangle of the region.

[0065] : is a preset angular tolerance threshold, in degrees (°), used to define the allowable range of angular deviation.

[0066] In a specific embodiment of the present invention, based on product process requirements and experimental verification, the optimal value for the angle tolerance threshold delta is 25 degrees. That is, when the angle deviation of a pin is greater than 25 degrees, the system determines that it has a tilt defect.

[0067] Furthermore, to further improve adaptability to dimensional fluctuations within a batch, the dynamic benchmark can also adopt a judgment criterion based on statistical interquartile range. The steps of the interquartile range method are as follows: 1. For all selected pin area tuples Sort in ascending order.

[0068] 2. Calculate its first quartile. (Area value ranked in the 25th percentile) and the third quartile (75% of the area values).

[0069] 3. Calculate the interquartile range

[0070] A pin is determined to be the lower limit threshold of normal size. Determined by the following formula:

[0071] Here, α is an adjustable scaling factor, which is usually taken as 1.5 based on experience. Finally, for the... Defects in individual pins as follows:

[0072] This method uses the distribution characteristics of the data to define the normal range, compared to a fixed proportionality coefficient. It has stronger adaptability and statistical significance.

[0073] Example 2 An electronic component pin defect detection system for implementing the method described in Embodiment 1 includes: The image acquisition unit is used to acquire images of the component under test; A processing unit, connected to the image acquisition unit, is used to execute image processing and analysis algorithms; An illumination unit is used to provide uniform illumination to the pin under test. The processing unit is configured to execute the method described in Embodiment 1.

[0074] The above description is merely a preferred embodiment of the present invention. It should be noted that the present invention is not limited to the specific embodiments described above. Any simple modifications, equivalent changes, and alterations made by those skilled in the art to the above embodiments based on the technical essence of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting pin defects in electronic components based on machine vision, characterized in that, Includes the following steps: S1: Image acquisition and preprocessing, acquiring images of the component under test and performing grayscale conversion and noise reduction; S2: Target segmentation and morphological optimization, which performs binarization segmentation on the preprocessed image and optimizes the segmentation results through morphological operations; S3: Precise screening based on composite features, performing connected component analysis on the optimized region, and screening out the pin region based on composite features of area and thin length; S4: Feature extraction and defect judgment. Extract the geometric features of the selected pin areas and use the median of the features of the current batch of pins as a dynamic benchmark to adaptively judge pin defects and tilting defects.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: S1-1: Construct a visual inspection hardware platform that includes an image acquisition unit, a processing unit, and a ring-shaped LED light source; S1-2: Acquire the RGB image of the pin under test through the image acquisition unit and convert it into a grayscale image; S1-3: Median filtering is used to smooth and denoise the grayscale image.

3. The method according to claim 1, characterized in that, Step S2 specifically includes: S2-1: The preprocessed image is binarized using a fixed grayscale threshold range to distinguish pixels into foreground and background; S2-2: Morphological optimization of the binary image is performed by opening and closing operations in sequence. The opening operation is used to remove noise, and the closing operation is used to fill holes and bridge breaks.

4. The method according to claim 3, characterized in that, The fixed grayscale threshold range is [26, 255]; the opening operation uses a circular structuring element with a radius of 10; the closing operation uses a circular structuring element with a radius of 32.

5. The method according to claim 1, characterized in that, Step S3 specifically includes: S3-1: Perform connected component analysis on the morphologically optimized binary image to identify all independent regions; S3-2: Set the threshold range for area [1000, 5000] and the lower threshold for thinness 2.0, and filter out regions that meet both conditions as candidate pins.

6. The method according to claim 5, characterized in that, The slenderness is defined as the ratio of the length of the major semi-axis to the length of the minor semi-axis of the equivalent ellipse of the region.

7. The method according to claim 1, characterized in that, Step S4 specifically includes: S4-1: Extract the area and spindle angle of each selected pin region; S4-2: Calculate the median area of ​​all pins and use it as a dynamic benchmark for judging defects; S4-3: Traverse each pin. If its area is less than the product of the dynamic reference and a scaling factor γ, it is determined to be a defect. If the absolute value of its spindle angle deviates from 90°, it is determined to be a tilt defect.

8. The method according to claim 7, characterized in that, The proportionality coefficient γ is set to 0.7, and the angular tolerance threshold δ is set to 25°.

9. A system for detecting pin defects in electronic components for implementing the method according to any one of claims 1-8, characterized in that, include: The image acquisition unit is used to acquire images of the component under test; A processing unit, connected to the image acquisition unit, is used to execute image processing and analysis algorithms; An illumination unit is used to provide uniform illumination to the pin under test. The processing unit is configured to execute steps S1 to S4 of the method.

10. The system according to claim 9, characterized in that, The image acquisition unit is a GigE interface industrial camera, the illumination unit is a ring LED light source, and the processing unit is an industrial computer.

Images