Solder ball vision inspection device and method

Abstract

Disclosed are a solder ball vision inspection device and method for detecting the shapes of solder balls irregularly disposed adjacent to each other. The solder ball vision inspection device according to an embodiment may comprise: an image-capturing unit that captures an image of one or more solder balls; and a determination unit that determines defects in the solder balls by using center coordinates which are extracted on the basis of the outlines of the one or more image-captured solder balls based on the image.

Description

Solder ball vision inspection device and method

[0001] The present invention relates to a solder ball vision inspection device and method for detecting the shape of solder balls arranged irregularly adjacent to each other.

[0002] Solder balls, an essential material for semiconductor packaging, are extremely small, and image analysis using electron microscopes is typically performed to inspect them. Solder ball quality inspection involves verifying that each individual solder ball maintains a circular shape and possesses area and length that meet specifications. However, when multiple solder balls exist within a single image during the inspection process—especially if they are adjacent to each other or arranged irregularly—existing image analysis methods may face limitations in accurately distinguishing individual solder balls.

[0003] One of the major problems with existing image inspection technologies is the difficulty in accurately detecting boundaries between adjacent solder balls. When solder balls are placed close together or overlap, errors may occur where multiple solder balls are recognized as a single object, making it difficult to accurately measure their shape and size. Furthermore, since conventional contour detection methods operate based on fixed brightness thresholds, fluctuations in image brightness can cause the outlines of solder balls to become distorted or parts to be incorrectly detected. These limitations lead to low reliability of inspection results and difficulties in solder ball quality control.

[0004] The purpose is to provide a solder ball vision inspection device and method for detecting the shape of solder balls arranged irregularly adjacent to each other.

[0005] According to one aspect, a solder ball vision inspection device may include an image capturing unit for capturing images of one or more solder balls; and a determination unit for determining defects in solder balls using center coordinates extracted based on the outlines of one or more solder balls captured based on the images.

[0006] The judgment unit can detect the outlines of one or more solder balls using pixel brightness based on an image.

[0007] The judgment unit divides the image into two or more local regions according to a predetermined criterion, finds the darkest pixel for each divided local region, and expands the region in a direction where brightness increases centered on the darkest pixel for each local region, performing expansion up to the point where the expanded regions between the two or more local regions meet, thereby detecting the outline of the solder ball.

[0008] The judgment unit can extract center coordinates for one or more solder balls based on the outlines of one or more detected solder balls.

[0009] The judgment unit can detect solder ball objects by receiving center coordinates for one or more solder balls and using a segment model trained to extract images of solder balls from an image.

[0010] The judgment unit detects the circumscribed circle and inscribed circle of the solder ball object and can determine a primary defect based on the area ratio of the circumscribed circle and the inscribed circle.

[0011] The judgment unit converts objects other than the solder ball object determined to be defective into black based on the result of the first defect judgment, and can determine a second defect based on the converted image.

[0012] The judgment unit can determine secondary defects by inputting the converted image into a deep learning model trained to determine defects based on the shape of the object.

[0013] The judgment unit can detect a solder ball in which part of one or more solder balls included in the image have been cut.

[0014] If the number of solder balls with parts cut off exceeds a predetermined standard, the judgment unit may decide to change the shooting area and perform re-shooting.

[0015] According to one aspect, a solder ball vision inspection method performed on a computing device having one or more processors and a memory for storing one or more programs executed by one or more processors may include: an image capturing step of capturing an image of one or more solder balls; and a determination step of determining a defect in a solder ball using center coordinates extracted based on the outline of one or more solder balls captured based on the image.

[0016] The determination step can detect the outlines of one or more solder balls using pixel brightness based on the image.

[0017] The determination step divides the image into two or more local regions according to a predetermined criterion, finds the darkest pixel for each divided local region, and expands the region in a direction of increasing brightness centered on the darkest pixel for each local region, performing expansion until the point where the expanded regions of the two or more local regions meet, thereby detecting the outline of the solder ball.

[0018] The determination step can extract center coordinates for one or more solder balls based on the outlines of one or more detected solder balls.

[0019] The determination step can detect solder ball objects by using a segment model trained to extract images of solder balls from an image by receiving center coordinates for one or more solder balls as input.

[0020] The judgment step detects the circumscribed circle and inscribed circle of the solder ball object and can determine a primary defect based on the area ratio of the circumscribed circle and the inscribed circle.

[0021] The judgment step converts objects other than the solder ball objects judged as defective to black based on the result of the first defect judgment, and can determine a second defect based on the converted image.

[0022] The judgment step can determine secondary defects by inputting the converted image into a deep learning model trained to determine defects based on the shape of the object.

[0023] The determination step can detect solder balls in which some of the one or more solder balls included in the image have been cut.

[0024] The judgment step may decide to change the shooting area and perform re-shooting if the number of partially cut solder balls exceeds a predetermined standard.

[0025] The present invention can improve inspection accuracy by accurately detecting the contours of each solder ball by segmenting them into individual objects even when multiple solder balls are adjacent within an image.

[0026] In addition, the deep learning model can refer to the center point extracted from the outline of the solder ball to separate the circular border more precisely, thereby compensating for boundary recognition errors that may occur in the existing method.

[0027] Furthermore, by automating the external inspection of solder balls, abnormal solder balls that are not circular can be effectively detected, thereby improving the yield of the manufacturing process and enhancing the reliability of quality control.

[0028] FIG. 1 is a configuration diagram of a solder ball vision inspection device according to one embodiment.

[0029] FIGS. 2 to 4 are illustrative diagrams for explaining a solder ball vision inspection method according to one embodiment.

[0030] FIG. 5 is a flowchart illustrating a solder ball vision inspection method according to one embodiment.

[0031] Hereinafter, an embodiment of the present invention will be described in detail with reference to the attached drawings. In describing the present invention, specific descriptions of related known functions or configurations will be omitted if it is determined that such detailed descriptions may unnecessarily obscure the essence of the present invention. Furthermore, the terms described below are defined considering their functions in the present invention, and these may vary depending on the intentions or conventions of the user or operator. Therefore, their definitions should be based on the content throughout this specification.

[0032] Hereinafter, embodiments of a solder ball vision inspection device and method will be described in detail with reference to the drawings.

[0033] FIG. 1 is a configuration diagram of a solder ball vision inspection device according to one embodiment.

[0034] Referring to FIG. 1, a solder ball vision inspection device (100) may include an image capturing unit (110) that captures an image of one or more solder balls and a judgment unit (120) that determines defects in solder balls using center coordinates extracted based on the outline of one or more solder balls captured based on the image.

[0035] For example, the image capturing unit (110) can capture images of one or more solder balls to provide basic data for inspection and analysis. Since solder balls, which are essential materials for semiconductor packaging, are very fine in size, the image capturing unit (110) can perform image capturing using a scanning electron microscope (SEM) for precise quality inspection.

[0036] The image capturing unit (110) utilizes an electron microscope to capture the surface condition, shape, size, etc. of a solder ball at high resolution, and can generate an image in the form of an overlaid image after capturing a single solder ball multiple times from various angles. In addition, the image capturing unit (110) can obtain high-quality images by performing focus adjustment and lighting optimization according to the size and surface characteristics of the solder ball. The image capturing unit (110) can perform data storage and transmission for the captured images so that they can be utilized in subsequent image processing and judgment steps.

[0037] According to one embodiment, the determination unit (120) can detect the outlines of one or more solder balls using the brightness of pixels based on an image. For example, the determination unit (120) can detect the outlines of one or more solder balls by analyzing an image acquired from the image capturing unit (110). To this end, the determination unit (120) can recognize the boundaries of the solder balls based on the brightness information of the pixels and distinguish individual solder balls.

[0038] According to one embodiment, the determination unit (120) divides an image into two or more local regions according to a predetermined standard, finds the darkest pixel for each divided local region, and expands the region in a direction where the brightness increases centered on the darkest pixel for each local region, and performs expansion until the point where the expanded region between the two or more local regions meets, thereby detecting the outline of the solder ball.

[0039] According to one example, the judgment unit (120) may divide the image into local regions according to specific criteria during the detection process and then perform the process of finding the darkest pixel in each local region. At this time, the darkest pixel acts as a basis in the image. Here, since the background has dark pixels and the solder ball consists of bright pixels, the darkest pixel corresponds to the background and serves as the starting point for forming the outline. Subsequently, the judgment unit (120) may proceed with the analysis by expanding in a direction in which brightness gradually increases based on the dark pixel.

[0040] The determination unit (120) continues to perform expansion until the expanded areas originating from each local area meet each other during the expansion process, and can determine the outline of the solder ball by forming a boundary at the point where the expanded areas meet each other. At this time, the individual areas start at a low brightness and gradually spread out in a direction of higher brightness, and at the moment when the boundaries spreading out from different local areas overlap, the point can be recognized and detected as the outline of the solder ball.

[0041] Through this, the judgment unit (120) can accurately distinguish individual solder balls even when they are adjacent to each other. While the existing thresholding-based outline detection method is susceptible to fluctuations and may incorrectly recognize adjacent solder balls as a single object, the judgment unit (120) can distinguish the boundaries of individual solder balls that are close to each other more precisely.

[0042] For example, the judgment unit (120) can recognize the outline of a solder ball using a watershed algorithm. For example, the watershed algorithm assumes the image as topography and converts pixel brightness values ​​into elevations so that water fills from the lowest point (local minimum, Basin) to distinguish areas. The watershed algorithm starts from the darkest pixel (local minimum) and expands in a brighter direction, and can distinguish objects by setting boundaries based on the point where the expanded boundaries of different areas meet.

[0043] According to one embodiment, the determination unit (120) can extract center coordinates for one or more solder balls based on the outline of one or more detected solder balls.

[0044] For example, the determination unit (120) can calculate the center coordinates of the solder ball using a geometric center (centroid). For example, the determination unit (120) can find a pixel area (an area inside the contour) constituting the solder ball based on the detected outline and calculate the centroid of this area. In this case, the centroid can be defined as the average coordinate of all pixels inside the object.

[0045] For example, the judgment unit (120) can calculate the center coordinates of the solder ball using ellipse fitting. For example, the judgment unit (120) can approximate the outline of the solder ball into an ellipse shape and set the center of the ellipse as the center coordinates. This is an example, and the judgment unit (120) can obtain the center coordinates using a method other than the method described in the example.

[0046] According to one embodiment, the determination unit (120) can detect solder ball objects by using a segment model trained to receive center coordinates for one or more solder balls and extract an image of a solder ball from an image. The determination unit (120) can perform solder ball segmentation by extracting the center coordinates of each solder ball through boundary detection and then inputting the center coordinates into a segment model that utilizes the center coordinates as a prompt.

[0047] According to one example, a prompt-based segment model can locate an object within an image based on input center coordinates and segment the precise boundaries of a solder ball based on learned patterns. In particular, since the deep learning-based segment model is robust against various lighting conditions and noise, it can achieve higher detection reliability than conventional image processing methods.

[0048] According to one embodiment, the judgment unit (120) detects the circumscribed circle and the inscribed circle of a solder ball object and can determine a primary defect based on the area ratio of the circumscribed circle and the inscribed circle. For example, the judgment unit (120) can analyze the shape of the detected solder ball object to detect the circumscribed circle and the inscribed circle, and based on this, evaluate the shape precision of the solder ball to determine whether there is a primary defect. At this time, the circumscribed circle and the inscribed circle refer to the largest circle containing the solder ball and the smallest circle that can be contained inside the solder ball, respectively, and the judgment unit (120) can determine the shape alignment of the solder ball using the area ratio between the two circles, as shown in FIG. 2.

[0049] According to one example, the judgment unit (120) may determine that the product is good if this ratio falls within a preset normal range (e.g., 1.5 to 2), and determine that it is defective (Recheck) if it falls outside the preset range or if the difference in area exceeds a specific threshold. The judgment unit (120) can quantitatively evaluate the shape of the solder ball using this rule-based inspection technique and determine defects based on a specific threshold.

[0050] According to one embodiment, the judgment unit (120) converts objects other than the solder ball object determined to be defective into black based on the result of the first defect determination, and can determine a second defect based on the converted image. For example, the judgment unit (120) can determine a second defect by inputting the converted image into a deep learning model trained to determine defects based on the shape of the object.

[0051] According to one example, the judgment unit (120) removes (converts to black) solder balls determined to be good from the image and generates a new image containing only objects classified as defective, thereby enabling further analysis. As described in FIG. 2, the image converted to black is used as input data for a second defect determination and is input into a deep learning model for defect determination, allowing for a more precise re-inspection. The deep learning model performs defect determination while having learned various factors such as shape distortion, minute surface defects, and size anomalies, and can evaluate the quality of solder balls with higher precision than the existing rule-based inspection method. Through this, the judgment unit (120) can compensate for problems of false positives or false negatives that may occur in the first inspection and increase the accuracy of the determination.

[0052] FIG. 3 illustrates a comparison between an input image and a judgment result during the quality inspection process of a solder ball. In the left image, an original solder ball image before inspection is displayed, and solder balls of various sizes and shapes are arranged. Some of the solder balls appear to have deformed shapes or are damaged, and the judgment unit (120) determines whether they are normal through quality inspection.

[0053] The image on the right visually displays the inspection results of the solder balls, where solid lines indicate objects judged to be good, and dotted lines indicate objects judged to be defective (Recheck). The judgment unit (120) analyzes the outline of the solder balls and distinguishes between normal and defective by applying shape and size criteria.

[0054] According to one embodiment, the judgment unit (120) can detect a solder ball with a portion cut off among one or more solder balls included in an image, and if the number of solder balls with a portion cut off is greater than or equal to a predetermined standard, the image capturing unit can decide to change the capturing area and perform re-capturing.

[0055] For example, the judgment unit (120) can detect one or more solder balls within a captured image and analyze whether some of the solder balls are cut to determine whether it affects quality inspection. FIG. 4 shows the process in which, after the image capturing unit (110) captures the solder balls, the judgment unit (120) detects the outline of the solder balls within the image and determines whether the image contains solder balls that are partially cut. If there are cut solder balls, the judgment unit (120) checks whether the number exceeds a specific standard, and if it exceeds a standard, it can decide to change the shooting area and perform re-shooting.

[0056] Referring to FIG. 4, the captured image may contain solder balls that are partially included within the shooting area, and the judgment unit (120) can detect these partial solder balls and calculate the number. In FIG. 4, the shooting area is divided into a primary shooting area and a re-shooting area, and a solder ball that is partially cut off is identified in specific parts (A, B, C). The judgment unit (120) determines whether these cut solder balls may affect the inspection result, and if it exceeds a preset standard, it determines that additional image shooting is required.

[0057] Based on this, the judgment unit (120) determines whether the captured image is suitable for inspection, and if it contains a large number of cut solder balls, it may request the image capturing unit (110) to adjust the shooting area and perform re-shooting. Through this, the overall shape of the solder balls can be recognized more accurately, the reliability of the inspection can be increased, and inspection errors that may occur due to cut solder balls can be minimized.

[0058] FIG. 5 is a flowchart illustrating a solder ball vision inspection method according to one embodiment.

[0059] According to one embodiment, the solder ball vision inspection device may be a computing device having one or more processors and a memory that stores one or more programs executed by one or more processors.

[0060] According to one embodiment, the solder ball vision inspection device captures an image of one or more solder balls (510), and can determine defects in the solder balls using center coordinates extracted based on the outline of one or more solder balls captured based on the image (520).

[0061] One aspect of the present invention may be implemented as computer-readable code on a computer-readable recording medium. Codes and code segments implementing the above program can be easily inferred by a computer programmer in the art. A computer-readable recording medium may include any type of recording device in which data that can be read by a computer system is stored. Examples of computer-readable recording media may include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, etc. Additionally, a computer-readable recording medium may be distributed across networked computer systems and written and executed as computer-readable code in a distributed manner.

[0062] The present invention has been described above focusing on its preferred embodiments. Those skilled in the art will understand that the present invention may be implemented in modified forms without departing from the essential characteristics of the invention. Accordingly, the scope of the present invention should not be limited to the aforementioned embodiments but should be interpreted to include various embodiments within the scope equivalent to those described in the claims.

Claims

1. An image capturing unit for capturing images of one or more solder balls; and It includes a determination unit that determines defects in solder balls using center coordinates extracted based on the outlines of one or more solder balls captured based on the above image, The above judgment unit A solder ball vision inspection device that detects a solder ball in which some of the one or more solder balls included in the above image have been cut.

2. In Paragraph 1, The above judgment unit A solder ball vision inspection device that detects the outlines of one or more solder balls using pixel brightness based on the above image.

3. In Paragraph 2, The above judgment unit The above image is divided into two or more local regions according to a predetermined standard, and Finding the darkest pixel for each segmented local area, A solder ball vision inspection device that detects the outline of a solder ball by expanding an area in a direction of increasing brightness centered on the darkest pixel in each local area, and performing expansion up to the point where the expanded areas between two or more local areas meet.

4. In Paragraph 2, The above judgment unit A solder ball vision inspection device that extracts center coordinates for one or more solder balls based on the outlines of one or more detected solder balls.

5. In Paragraph 4, The above judgment unit A solder ball vision inspection device that detects solder ball objects using a segment model trained to receive center coordinates for one or more solder balls and extract an image of a solder ball from an image.

6. In Paragraph 5, The above judgment unit Detecting the circumscribed circle and inscribed circle of the above solder ball object, A solder ball vision inspection device that determines primary defects based on the area ratio of the circumscribed circle and the inscribed circle.

7. In Paragraph 6, The above judgment unit A solder ball vision inspection device that, based on the result of a first defect judgment, converts objects other than the solder ball object judged to be defective into black, and judges a second defect based on the converted image.

8. In Paragraph 7, The above judgment unit A solder ball vision inspection device that determines secondary defects by inputting the converted image into a deep learning model trained to determine defects based on the shape of an object.

9. In Paragraph 1, The above judgment unit A solder ball vision inspection device that determines to perform re-shooting by changing the shooting area when the number of partially cut solder balls exceeds a predetermined standard.

10. One or more processors, and A method performed in a computing device having a memory for storing one or more programs executed by one or more processors, wherein A video capturing step for capturing images of one or more solder balls; and The method includes a determination step for determining defects in solder balls using center coordinates extracted based on the outlines of one or more solder balls captured based on the above image, wherein The above determination step A solder ball vision inspection method for detecting a solder ball in which some of the one or more solder balls included in the above image are cut.

11. In Paragraph 10, The above determination step A solder ball vision inspection method that detects the outlines of one or more solder balls using pixel brightness based on the above image.

12. In Paragraph 11, The above determination step The above image is divided into two or more local regions according to a predetermined standard, and Finding the darkest pixel for each segmented local area, A solder ball vision inspection method for detecting the outline of a solder ball by expanding an area in a direction of increasing brightness centered on the darkest pixel of each local area, and performing expansion up to the point where the expanded areas of two or more local areas meet.

13. In Paragraph 11, The above determination step A solder ball vision inspection method for extracting center coordinates for one or more solder balls based on the outlines of one or more detected solder balls.

14. In Paragraph 13, The above determination step A solder ball vision inspection method that detects solder ball objects using a segment model trained to receive center coordinates for one or more solder balls and extract an image of a solder ball from an image.

15. In Paragraph 14, The above determination step Detecting the circumscribed circle and inscribed circle of the above solder ball object, A solder ball vision inspection method that determines primary defects based on the area ratio of the circumscribed circle and the inscribed circle.

16. In Paragraph 15, The above determination step A solder ball vision inspection method that, based on the result of a first defect judgment, converts objects other than the solder ball object judged to be defective into black, and judges a second defect based on the converted image.

17. In Paragraph 16, The above determination step A solder ball vision inspection method for determining secondary defects by inputting the converted image into a deep learning model trained to determine defects based on the shape of an object.

18. In Paragraph 10, The above determination step A solder ball vision inspection method that determines to perform re-shooting by changing the shooting area when the number of partially cut solder balls exceeds a predetermined standard.

Images