Method and system for identifying wire bond defect of a semiconductor device

Abstract

The present invention provides a method and system (1500) for identifying at least one wire bond defect of a semiconductor device (1800). The invention comprises an image acquisition device (1510) configured to perform the step of capturing the semiconductor device (1800), a project surface representation module (1521) configured to perform the step of generating a project surface representation of the captured data, a computing module (1522) configured to perform the step of processing the projected surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom. The invention further involves determination of at least one tilt angle of the extracted point cloud, rotation of the extracted point cloud based on the tile angle, computation of a corrected point cloud. The invention also involves detection of wire bond defects, and classification of wire bond defects utilising the corrected point cloud.

Description

[0001]

[0002] METHOD AND SYSTEM FOR IDENTIFYING

[0003] WIRE BOND DEFECT OF A SEMICONDUCTOR DEVICE

[0004] FIELD OF INVENTION

[0005] The present invention relates to the technical field of defect identification technologies. More specifically, a method and system for identifying at least one wire bond defect of a semiconductor device.

[0006] BACKGROUND OF THE INVENTION

[0007] Wire bonding is a commonly used technique in the field of electronics assembly and packaging.

[0008] Wire bonding is a method of making electrical connections between integrated circuits (ICs) or other semiconductor devices and the external leads (typically made of metal) of a package or substrate. Wire bonding is used in a wide range of electronic devices, including microchips, integrated circuits, and various types of electronic packages.

[0009] There are a few disclosed technologies over the prior art that relate to systems and methods for identifying wire bond defects of a semiconductor device. Among them include the United States Patent Application US20150110384A1 which provides an image inspection method of die to database. In the method, a plurality of inspection areas in a plurality of positions in the to-be-inspected chips within a wafer are selected, a plurality of raw images of the inspection areas are obtained, and a plurality of locations of the raw images are then decoded. After that, an image extraction is performed on the raw images to obtain a plurality of image contours. Thereafter, the image contours are compared with a design database of the chip to obtain a result of a defect inspection and execute the same thing in the whole wafer.

[0010] However, there are multiple challenges when it comes to wire bond manufacturing and reliability. These challenges tend to be a function of several parameters such as the material systems, bonding parameters, and use environment.

[0011] Different wire bond-bond pad metal systems such as Aluminium- Aluminium (Al-Al), Gold-Aluminium (Au-Al), and Copper-Aluminium (Cu-Al) require different manufacturing parameters and behave differently under the same use environments.

[0012] During the wire bonding process, defects such as sagging wire, excessive wire height and die edge clearance may disrupt signal transmission.

[0013] These defects may vary in type and location, which makes it hard for rule-based machine vision solutions on two-dimensional images to accurately determine a defective wire bond.

[0014] To detect such defects, three-dimensional enabled vision inspection came into existence.

[0015] However, using these techniques may have certain limitations such as the generation of millions of vertices (three-dimensional points) in each snapshot of a three-dimensional camera, which may lead to an increase in the cost of these techniques.

[0016] Therefore, there is a need for a method and a system that may identify different hidden structure defects such as “Die edge clearance”, “Wire loop height” and “Sagging wire” without increasing the cost of the overall process. A method and a system that may overcome above above-discussed problems and provide cost-efficient and effective identification of at least one wire bond defect of the semiconductor device while improving defect identification capability is further desirable.

[0017] SUMMARY OF INVENTION

[0018] The present invention intends to provide a method for identifying at least one wire bond defect of a semiconductor device. The method comprises the steps of capturing, by an image acquisition device, the semiconductor device for obtaining captured data related to the semiconductor device, generating, by a project surface representation module, a project surface representation of the captured data, processing, by a computing module, the projected surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom, and detecting, by a wire bond defect detection module that operates a wire bond defect detection model, wire bond defects within the projected surface representation of the captured data. The method further involves determining, by the computing module, at least one tilt angle of an extracted point cloud of the one or more regions of interest, performing, by the computing module, a rotation of the extracted point cloud by the determined tilt angle, computing, by the computing module, a corrected point cloud, and classifying, by a wire bond defect classification module that operates a wire bond defect classification model, the wire bond defect by utilising the computation of the corrected point cloud. Preferably, the method further comprises the steps of performing, by a wire bond detection and annotation module, detection and annotation of one or more wire bonds within the extracted point cloud of the one or more regions of interest, which further comprises the steps of loading generated projected surface representation for the point cloud, duplicating wire detection map as a colour mask image, adding annotation information for the wire bonding region on mask image by setting the wire bond region as a different colour for a different class, detecting a contour on a non-zero blob region, measuring a height, width and axes information of the contour, mapping a contour colour to label based on a user-defined rule, storing information of the contour into bounding box object format, and storing the colour mask image and bounding box information into a database. The method further comprises training, by a wire bond defect detection model training module, the wire bond defect detection model based on an annotated point cloud data, wherein the steps of training the wire bond defect detection model based on the annotated point cloud data comprises loading the generated projected surface representation for the point cloud with the bounding box information, dividing the loaded data into train and test based on a user-defined ratio, creating an object detection algorithm using one of machine learning or deep learning techniques, modifying, by a modifying module, the wire bond defect detection model based on the training, and storing a modified wire bond defect detection model in the database. The point cloud is preferably three-dimensional.

[0019] Preferably, the step of generating the projected surface representation of the captured data comprises the steps of normalising coordinates of the captured data to a range of about 0 and 1 in the axes, converting normalised coordinates to a three-column map table, with a first column being column value, a second column being row information, and a third column being an intensity value, and creating the projected surface representation for the captured data by utilising the map table information.

[0020] Preferably, the step of determining, by the computing module, the tilt angle of the extracted point cloud of the one or more regions of interest comprises the steps of obtaining a centroid and coordinates of the point cloud, computing distance between the centroid and the coordinates of the extracted point cloud, and detecting the tilt angle of the extracted point cloud. Preferably, the point cloud is three-dimensional.

[0021] Preferably, the step of classifying the wire bond defect by utilising the computation of the corrected point cloud comprises the steps of creating two binary maps by utilising data related to the corrected point cloud, wherein the two binary maps comprise a Horizontal Gap Map (HGM) and a Vertical Gap Map (VGM), determining depth data of the corrected point cloud, creating a sliding window with a user-defined length to find a gap by checking maximum and minimum values in the sliding window for each row of the depth data of the point cloud, creating a sliding window with the user-defined length to find the gap by checking maximum and minimum values in the sliding window for each column of the depth data of the point cloud, creating a Horizontal- Vertical (HV) Intersect Map by utilising an intersecting process between the Horizontal Gap Map (HGM) and Vertical Gap Map (VGM) map, performing masking on the projected surface representation for the point cloud by utilising the Horizontal -Vertical (HV) Intersect Map and storing as a Projected Surface Representation (PSR), computing a mean value of non-zero Projected Surface Representation (PSR), performing thresholding on the projected surface representation for the point cloud using mean value and user-defined ratio and storing it as a Raw Wire Bonding Map, and performing a morphological closing operation to filter any one or both noise and blobs in a Raw Wire Map and storing it as Wire Bonding Map.

[0022] Preferably, the step of classifying, by a wire bond defect classification module that operates a wire bond defect classification model, the wire bond defect by utilising the computation of the corrected point cloud, further comprises the steps of loading the sampled point cloud, performing classification process on the loaded point cloud, determining one or more classified defects, and displaying the classified defect on a display unit.

[0023] Preferably, the method further comprises the steps of training, by a wire bond defect classification model training module, the wire bond defect classification model, which comprises the steps of, loading the sampled point cloud dataset, dividing loaded data into train and test based on user-defined ratio, creating a defect classification algorithm using one of a machine learning or a deep learning techniques, and modifying, by the modifying module, the wire bond defect classification model based on the training. Preferably, the method further comprises the step of storing a modified wire bond defect classification model in the database.

[0024] Preferably, the method further comprises the steps of integrating the detected wire bond defect and the classified wire bond defect, and displaying the integrated result on the display unit.

[0025] The present invention further intends to provide a system for identifying at least one wire bond defect of a semiconductor device, characterised in that, the system comprises an image acquisition device for capturing the semiconductor device for obtaining data related to the semiconductor device, a project surface representation module for generating a project surface representation of the captured data, a computing module operatively configured for processing the project surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom, determining at least one tilt angle of the extracted point cloud of one or more regions of interest, performing a rotation of the extracted point cloud by the determined tilt angle for correcting the extracted point cloud, and computing at least one corrected point cloud. The system further comprises a wire bond defect detection module that operates a wire bond defect detection model, for detecting wire bond defects within the projected surface representation of the captured data, and a wire bond defect classification module that operates a wire bond defect classification model, for classifying the wire bond defect by utilising the computation of the corrected point cloud.

[0026] Preferably, the system further comprises an annotation module for annotating the point cloud of the identified wire bond defect, a point cloud sampling module for sampling the annotated point cloud based on user-defined setting, a wire bond defect detection model training module for training the defect detection model based on the annotated point cloud data, a modifying module for modifying the wire bond defect detection model based on the training of the wire bond defect detection model, and a database for storing the modified defect detection model.

[0027] This together with the other aspects of the present invention along with the various features of novelty that characterize the present disclosure is pointed out with particularity in claims annexed hereto and forms a part of the present invention.

[0028] For a better understanding of the present disclosure, its operating advantages, and the specified objective attained by its uses, reference should be made to the accompanying descriptive matter in which there are illustrated exemplary embodiments of the present invention.

[0029] One skilled in the art will readily appreciate that the invention is well adapted to carry out the objects and obtain the ends and advantages mentioned, as well as those inherent therein. The embodiments described herein are not intended as limitations on the scope of the invention.

[0030] BRIEF DESCRIPTION OF DRAWINGS

[0031] FIG. 1 illustrates a flowchart describing a method for identifying at least one wire bond defect of a semiconductor device, which involves a first stage where one or more models that include a wire bond defect detection model and a wire bond defect classification model are trained, according to various embodiments of the present invention.

[0032] FIG. 2 illustrates a flowchart describing the method for identifying at least one wire bond defect of a semiconductor device, which involves a second stage where one or more models that include the wire bond defect detection model and the wire bond defect classification model being deployed, according to various embodiments of the present invention.

[0033] FIG. 3 illustrates a flowchart representing example steps of generating, by a project surface representation module, a project surface representation of captured data for identifying one or more regions of interest, according to various embodiments of the present invention.

[0034] FIG. 4 illustrates a flowchart representing example steps of extracting, by a computing module, a point cloud of the identified one or more regions of interest, according to various embodiments of the present invention.

[0035] FIGS 5 and 6, in combination, illustrate a flowchart representing example steps of determining a tilt angle of the extracted point cloud along the x-axis and performing a rotation of the extracted point cloud by the determined tilt angle, according to various embodiments of the present invention.

[0036] FIG. 7 illustrates a schematic representation of a region of interest of the semiconductor device, according to various embodiments of the present invention.

[0037] FIG. 8 illustrates a schematic representation of an extracted point cloud without tilting along the x-axis, according to various embodiments of the present invention.

[0038] FIG. 9 illustrates a schematic representation of the extracted point cloud with tilting along the x-axis, according to various embodiments of the present invention.

[0039] FIG. 10 illustrates a schematic representation of the extracted point cloud with a determination of a tilt angle along the x-axis, according to various embodiments of the present invention.

[0040] FIG. 11 illustrates a schematic representation of the extracted point cloud before rotation with the determined tilt angle along the x-axis, according to various embodiments of the present invention.

[0041] FIG. 12 illustrates a schematic representation of the extracted point cloud after rotation with the determined tilt angle along the x-axis, according to various embodiments of the present invention.

[0042] FIGS 13 and 14, in combination, illustrate a flowchart representing example steps of determining a tilt angle of the extracted point cloud along the y-axis and performing the rotation of the extracted point cloud by the determined tilt angle, according to various embodiments of the present invention.

[0043] FIG. 15 illustrates a schematic representation of a region of interest of the semiconductor device, according to various embodiments of the present invention.

[0044] FIG. 16 illustrates a schematic representation of an extracted point cloud without tilting along the y-axis, according to various embodiments of the present invention.

[0045] FIG. 17 illustrates a schematic representation of the extracted point cloud with tilting along the y-axis, according to various embodiments of the present invention.

[0046] FIG. 18 illustrates a schematic representation of the extracted point cloud with a determination of a tilt angle along the y-axis, according to various embodiments of the present invention.

[0047] FIG. 19 illustrates a schematic representation of the extracted point cloud before rotation with the determined tilt angle along the y-axis, according to various embodiments of the present invention.

[0048] FIG. 20 illustrates a schematic representation of the extracted point cloud after rotation with the determined tilt angle along the y-axis, according to various embodiments of the present invention.

[0049] FIGS 21 and 22, in combination, illustrate a flowchart representing example steps of detecting and annotating, by a wire bond detection and annotation module, according to various embodiments of the present invention.

[0050] FIG. 23 illustrates a flowchart representing example steps of performing wire bond defect detection model training, by the wire bond defect detection training module, according to various embodiments of the present invention.

[0051] FIG. 24 illustrates a flowchart representing example steps of detecting wire bond defects, by a wire bond defect detection module, according to various embodiments of the present invention.

[0052] FIG. 25 illustrates a flowchart representing example steps of training a wire bond defect detection model, according to various embodiments of the present invention.

[0053] FIG. 26 illustrates a flowchart representing example steps of performing wire bond defect classification model training, by the wire bond defect classification training module, according to various embodiments of the present invention

[0054] FIG. 27 illustrates a flowchart representing example steps of performing point cloud sampling, by a point cloud sampling module, according to various embodiments of the present invention.

[0055] FIG. 28 illustrates a flowchart representing example steps of classifying the wire bonding defect, by a wire bond defect classification module, according to various embodiments of the present invention.

[0056] FIG. 29 illustrates a flowchart representing example steps of training a wire bond defect detection model, according to various embodiments of the present invention.

[0057] FIG. 30 illustrates a block diagram of a system for identifying the wire bond defect of the semiconductor device, according to various embodiments of the present invention.

[0058] FIG. 31 illustrates a schematic diagram representing the hardware components of the system, according to various embodiments of the present invention.

[0059] DETAILED DESCRIPTION OF THE INVENTION

[0060] The present invention relates to the technical field of defect detection technologies. More specifically, a method and a system for identifying at least one wire bond defect of a semiconductor device.

[0061] The exemplary embodiments described herein detail for illustrative purposes are subjected to many variations. It should be emphasised, however, that the present invention is not limited to as disclosed.

[0062] It is understood that various omissions and substitutions of equivalents are contemplated as circumstances may suggest or render expedient, but these are intended to cover the application or implementation without departing from the scope of the present invention.

[0063] From here on, the following terms have the meanings indicated below.

[0064] The terms “a” and “an” herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items.

[0065] The terms “having”, “comprising”, “including”, and “variations” thereof signify the presence of a component.

[0066] The term “semiconductor device” preferably refers to a semiconductor chip comprising a plurality of wire bonds that enable connection with interconnects or metal contacts.

[0067] The inventive aspects of the invention along with various components and engineering involved will now be explained with reference to FIGS 1 to 31 herein.

[0068] From hereon, one or more flowcharts pertaining to the method and sub-methods of the present invention for identifying at least one wire bond defect of a semiconductor device are to be described. It is noted that the steps described in these flowcharts are not to be interpreted as non-limiting, and minor modifications to the steps (e.g. additions, omissions, or swaps) are permissible by a skilled person without substantial deviation from as described.

[0069] FIG. 1 illustrates a flowchart describing a method for identifying at least one wire bond defect of a semiconductor device, which involves a first stage where one or more models that include a wire bond defect detection model and a wire bond defect classification model are trained, according to various embodiments of the present invention.

[0070] FIG. 2 illustrates a flowchart describing the method for identifying at least one wire bond defect of a semiconductor device, which further involves a second stage where one or more models that include the wire bond defect detection model and the wire bond defect classification model being deployed, according to various embodiments of the present invention.

[0071] It is to be noted that the wire bond defects that the method may identify, through detection and / or classification, may include but shall not be limited to hidden structure defects such as “Die edge clearance”, “Wire loop height” and “Sagging wire”.

[0072] FIG. 1 shall now be described. The steps of FIG. 1 may begin with Step 101. Step 101 involves capturing, by an image acquisition device 1510, a semiconductor device 1800 for obtaining data related to the semiconductor device 1800. Preferably, the captured data is in point cloud format, or it may be converted into point cloud format from depth map format by the image acquisition device 1510.

[0073] Following Step 101 is Step 102. Step 102 involves generating, by a project surface representation module 1521, a project surface representation of the captured data, preferably three-dimensional data for identifying one or more regions of interest. Preferably, the captured data refers to data related to the semiconductor device 1800 captured by the image acquisition device 1510 in Step 101.

[0074] Following Step 102 is Step 103. Step 103 involves processing, by a computing module 1522, the projected surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom. Preferably, the point cloud is three-dimensional.

[0075] Following Step 103 is Step 104 and Step 106, which are steps that may occur series or concurrent manner.

[0076] In particular, Step 104 involves performing, by a wire bond detection and annotation module 1526, detection and annotation of one or more wire bonds within the extracted point cloud of the one or more regions of interest.

[0077] Following Step 104 is Step 105. Step 105 involves performing, by a wire bond defect detection model training module 1524, wire bond defect detection model training. In particular, a wire bond defect detection model may be trained based on the point cloud of the identified regions of interest in which wire bonds had been detected and annotated as per the previous step, Step 104.

[0078] In particular, Step 106 may involve determining, by the computing module 1522, a tilt angle of an extracted point cloud of the one or more regions of interest.

[0079] Following Step 106 is Step 107. Step 107 involves performing, by the computing module 1522, a rotation of the extracted point cloud by the determined tilt angle for correcting the point cloud.

[0080] Following Step 107 is Step 108. Step 108 involves computing, by the computing module 1522, the corrected point cloud.

[0081] Following Step 108 is Step 109. Step 109 involves performing, by a point cloud sampling module 1527, sampling upon the corrected point cloud.

[0082] Following Step 109 is Step 110. Step 110 involves performing, by a wire bond defect classification module 1528, wire bond defect classification model training. In particular, a wire bond defect classification model may be trained based on the corrected point cloud information from the previous step, Step 109.

[0083] Finally, following Step 105 and Step 110 is Step 111. Step 111 involves storing the trained wire bond defect detection model obtained from Step 105 and the trained wire bond defect classification model obtained from Step 110 in the database 1530.

[0084] FIG. 2 shall now be described. The steps of FIG. 2 may begin with Step 201. Step 201 involves loading one or more models from the database 1530, which may include a trained wire bond defect detection model and a trained wire bond defect classification model.

[0085] Following Step 201 is Step 202. Step 202 involves capturing, by an image acquisition device 1510, a semiconductor device 1800 for obtaining data related to the semiconductor device 1800. Preferably, the captured data is in point cloud format, or it may be converted into point cloud format from depth map format by the image acquisition device 1510.

[0086] Following Step 202 is Step 203. Step 203 involves generating, by a project surface representation module 1521, a project surface representation of the captured data, preferably three-dimensional data for identifying one or more regions of interest. Preferably, the captured data refers to data related to the semiconductor device 1800 captured by the image acquisition device 1510 in Step 201.

[0087] Following Step 203 is Step 204. Step 204 involves processing, by a computing module 1522, the projected surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom. Preferably, the point cloud is three-dimensional.

[0088] Following Step 204 is Step 205. Step 205 involves detecting, by a defect detection module 1523, which operates a trained wire bond defect detection model loaded from the database 1530, wire bond defects that may be present within the extracted point cloud regions of interest.

[0089] Following Step 205 is Step 206. Step 206 is a decision step whereby it is determined whether or not wire bond defects were found by the defect detection module 140 Should this be the case, then Step 206 proceeds to Step 213. Else, Step 206 proceeds to Step 207.

[0090] In particular, Step 207 involves processing, by a computing module 1522, the projected surface representation further extracts one or more regions of interest within the captured data (i.e. the three-dimensional point cloud). More specifically, the x-axis, the y-axis, the width value, the height value, and the lowest plane region value obtained from step 204 will be used to further extract these regions of interest.

[0091] Following Step 207 is Step 208. Step 208 involves determining, by the computing module 1522, a tilt angle of the extracted point cloud of the one or more regions of interest.

[0092] Following Step 208 is Step 209. Step 209 involves determining, by the computing module 1522, a rotation of the extracted point cloud by the determined tilt angle for correcting the extracted point cloud of the regions of interest.

[0093] Following Step 209 is Step 210. Step 210 involves performing, by a point cloud sampling module 1527, sampling upon the corrected point cloud.

[0094] Following Step 204 is Step 205. Step 205 involves classifying, by a wire bond defect classification module 1528, which operates a trained wire bond defect classification model loaded from the database 1530, wire bond defects that may be present within the extracted point cloud regions of interest.

[0095] Following Step 212 is Step 213. Step 213 involves displaying results on a display. In particular, the results may include results as provided by the defect detection module 1523 and the defect classification module 1528. With this, Step 213 may loop back to Step 202 and repeat therefrom for another semiconductor device.

[0096] It is to be noted that certain steps performed in the first stage and the second stage may substantially similar to each other, i.e. certain steps performed in in the first stage and the second stage may substantially similar sub-steps. These instances shall be described accordingly.

[0097] FIG. 3 illustrates a flowchart representing example steps of generating, by the project surface representation module 1521, the project surface representation of the captured data. The steps as described in FIG. 3 may be regarded as sub-steps of Step 102 previously described in FIG. 1, or sub-steps of Step 203 previously described in FIG. 2.

[0098] The steps of FIG. 3 may begin with Step 301. which involves normalising the coordinates of the point cloud to be at a range of about 0 and 1 along the x-axis, y-axis, and z-axis.

[0099] Following Step 301 is Step 302. Step 302 involves converting normalised coordinates to a three-column map table, with a first column being column value, a second column being row information, and a third column being an intensity value.

[0100] Preferably, by way of example, the normalised coordinates are converted to the three- column map table by multiplying one or more parameters that include (a) a normalised x-axis with a predefined width, (b) a normalised y-axis with a negative predefined height, and (c) a z-axis with a pixel value of 255.

[0101] Following Step 302 is Step 303. Step 303 involves creating the project surface representation for the captured data by utilising the map table information.

[0102] Following Step 303 is Step 304. Step 304 involves storing the projected surface representation and the map table information in a database 1530.

[0103] FIG. 4 illustrates a flowchart representing example steps of processing, by a computing module 1522, the captured data to identify and extract at least one three-dimensional point cloud of one or more regions of interest within the captured data. The steps described in FIG. 4 may be regarded as sub-steps of step 103 previously described in

[0104] FIG. 1, or sub-steps of Step 204 previously described in FIG. 2.

[0105] The steps of FIG. 4 may begin with Step 401. Step 401 involves defining a template for the projected surface representation of the captured data.

[0106] Following Step 401 is Step 402. Step 402 involves defining the one or region of interest comprising a lowest plane region and a die region of the semiconductor device / chip 1800

[0107] Preferably, the lowest plane region is defined as a height of the device under inspection. Moreover, preferably, a value is calculated from the average height in the region defined. Moreover, preferably, any data point having a height lower than the obtained will be ignored. Moreover, it is to be noted that the die region is defined as a flat region of the die which may be used to detect the tilting of the die of the semiconductor device 1800 which will be explained in the below paragraphs.

[0108] Following Step 402 is Step 403. Step 403 involves loading the projected surface representation for the captured data obtained from the image acquisition device 1510.

[0109] Following Step 403 is Step 404. Step 404 involves performing an image processingbased template matching technique on the projected surface representation for the captured data, for identifying at least one matching region.

[0110] Following Step 404 is Step 405, which involves obtaining one or more parameters of the at least one matching region. Preferably, the one or more parameters comprise the coordinates value of the width and height along the x-axis and the y-axis.

[0111] Following Step 405 is Step 406, which involves extracting at least one region of interest from the projected surface representation for the captured data by utilising the obtained coordinates values of the width and height along the x-axis and the y-axis of the at least one matching region.

[0112] Following Step 406 is Step 407, which involves obtaining a coordinate value of the z- axis of the lowest plane region from the captured data.

[0113] Following Step 407 is step 408, which involves extracting the three-dimensional point cloud from the captured data based on the values of x, y, and z axes of the lowest plane region and width and height of the at least one matching region.

[0114] FIGS 5 and 6, in combination, illustrate a flowchart representing example steps of determining the tilt angle of the extracted point cloud along the x-axis and performing the rotation of the extracted point cloud by the determined tilt angle. The steps described in FIGS 5 and 6, may be regarded as sub-steps of Step 106 and Step 107 previously described in FIG. 1, or sub-steps of Step 208 and 209 previously described in FIG. 2.

[0115] The steps of FIGS 5 and 6, may begin with Step 501. Step 501 involves loading a processed point cloud.

[0116] Following Step 501 is Step 502. Step 502 involves obtaining a centre point and at least two locations along a left side and a right side from the centre point of the x-axis of the extracted three-dimensional point cloud. Preferably, by way of example, the at least two locations along the x-axis comprise XA and XB, wherein XA and XB may be defined to be at any edge of the die region. The centre point may be a centre, or a midpoint, between the location XA and location XB, and it may be along the die region.

[0117] Following Step 502 is Step 503. Step 503 involves moving a view of the point cloud to a position defined as X having distance v from the centre point.

[0118] Following Step 503 is Step 504. Step 504 involves determining a distance between the position X and the at least two locations XA and XB , respectively. Preferably, a first distance di between position X and location xais determined, and a second distance d2 between position X and location / > is determined.

[0119] Following Step 504 is Step 505. Step 505 is a decision step whereby it is determined whether or not the first distance di between position X and location xaand the second distance dz between position X and location Xb are equal.. Should this be the case, this flowchart ends. Else, Step 505 proceeds to Step 506.

[0120] Following Step 505 is Step 506. Step 506 involves detecting and determining the tilt angle ei of the three-dimensional point cloud.

[0121] Following Step 506 is Step 507. Step 507 involves setting the centroid of the point cloud to be as a new centre point.

[0122] Following Step 507 is Step 508. Step 508 involves rotating the extracted three- dimensional point cloud by the determined tilt angle ei about the new centre point.

[0123] Finally, following Step 508 is Step 509. Step 509 involves storing the corrected three- dimensional point cloud in the database 1530.

[0124] FIG. 7 illustrates a schematic representation of a region of interest of the semiconductor device 1800 showing the die region, the two locations XA and XB, and the centre point that is between these two locations XA and XB. FIG. 7 may also be regarded as an illustrative representation of Step 502.

[0125] FIG. 8 illustrates a schematic representation of an extracted point cloud without tilting along the x-axis. FIG. 8 may also be regarded as an illustrative representation of Steps

[0126] 504 and 505. As shown, there is a first distance di that is a distance between position A and location XA, and a second distance d2 that is a distance between position X and location XB, wherein di is equal to di.

[0127] FIG. 9 illustrates a schematic representation of the extracted point cloud with tilting along the x-axis. FIG. 9 may also be regarded as an illustrative representation of Steps 504 and 505. As shown, there is a first distance di that is a distance between position A and location XA, and a second distance d that is a distance between position X and location XB, wherein di is not equal to d2.

[0128] FIG. 10 illustrates a schematic representation of the extracted point cloud with the determination of the tilt angle ei along the x-axis which shows that di is not equal to d2. FIG. 9 may also be regarded as an illustrative representation of Step 506.

[0129] FIG. 11 illustrates a schematic representation of the extracted point cloud before rotation with the determined tilt angle ei along the x-axis.

[0130] FIG. 12 illustrates a schematic representation of the extracted point cloud after rotation with the determined tilt angle ei along the x-axis.

[0131] FIGS 13 and 14, in combination, illustrate a flowchart representing example steps of determining the tilt angle of the extracted point cloud along the y-axis and performing the rotation of the extracted point cloud by the determined tilt angle. The steps described in FIGS 13 and 14, may be regarded as sub-steps of Step 106 and Step 107 previously described in FIG. 1, or sub-steps of Step 208 and 209 previously described in FIG. 2.

[0132] The steps of FIGS 13 and 14 may begin with Step 601. Step 601 involves loading the processed point cloud.

[0133] Following Step 601 is Step 602. Step 602 involves obtaining the centre point and at least two locations along a left side and a right side from the centre point of the y-axis of the extracted three-dimensional point cloud. Preferably, by way of example, the at least two locations along the y-axis comprise and yB, wherein and ya may be defined to be at any edge of the die region. The centre point may be a centre, or a midpoint, between the location and location and it may be along the die region.

[0134] Following Step 602 is Step 603. Step 603 involves moving a view of the point cloud to a position defined as Y having distance v from the centre point.

[0135] Following Step 603 is Step 604. Step 604 involves determining a distance between the position Y and the at least two locations yA and yB, respectively. Preferably, a first distance di between position Y and location is determined, and a second distance d2 between position Y and location yB is determined.

[0136] Following Step 604 is Step 605. Step 605 is a decision step whereby it is determined whether or not the first distance di between position Y and location yA and the second distance d2 between position Y and location yB are equal. . Should this be the case, this flowchart ends. Else, Step 605 proceeds to Step 606.

[0137] Following Step 605 is Step 606. Step 606 involves detecting and determining the tilt angle ei of the three-dimensional point cloud.

[0138] Following Step 606 is Step 607. Step 607 involves setting the centroid of the point cloud to be as a new centre point.

[0139] Following Step 607 is Step 608. Step 608 involves rotating the extracted three- dimensional point cloud by the determined tilt angle ei about the new centre point.

[0140] Finally, following Step 608 is Step 609. Step 609 involves storing the corrected three- dimensional point cloud in the database 1530.

[0141] FIG. 15 illustrates a schematic representation of a region of interest of the semiconductor device 1800 showing the die region, the two locations y and ;. and the centre point that is between these two locations y and i7;. FIG. 15 may also be regarded as an illustrative representation of Step 602.

[0142] FIG. 16 illustrates a schematic representation of the extracted point cloud without tilting along the y-axis. FIG. 16 may also be regarded as an illustrative representation of Steps 604 and 605. As shown, there is a first distance di that is a distance between position Y and location y and a second distance d2 that is a distance between position Y and location y#, wherein di is equal to d2.

[0143] FIG. 17 illustrates a schematic representation of the extracted point cloud with tilting along the y-axis. FIG. 17 may also be regarded as an illustrative representation of Steps 604 and 605. As shown, there is a first distance di that is a distance between position Y and location yy and a second distance d2 that is a distance between position Y and location ye, wherein di is not equal to d2.

[0144] FIG. 18 illustrates a schematic representation of the extracted point cloud with the determination of the tilt angle ei along the y-axis which shows di is not equal to d2. FIG. 18 may also be regarded as an illustrative representation of Step 606.

[0145] FIG. 19 illustrates a schematic representation of the extracted point cloud before rotation with the determined tilt angle ei along the y-axis.

[0146] FIG. 20 illustrates a schematic representation of the extracted point cloud after rotation with the determined tilt angle ei along the y-axis.

[0147] It is to be noted that the set of steps illustrated in the flowchart of FIGS 5 and 6, and the set of steps illustrated in the flowchart of FIGS 13 and 14 may be executed in a sequential manner. By way of example, in the case for an execution in a sequential manner, the steps of the flowchart of FIGS 4 and 5 may be executed, and afterwards, the steps of the flowchart of FIGS 13 and 14 may be executed, or vice versa. It is preferred that the set of steps as illustrated in the flowchart of FIGS 5 and 6 and the set of steps illustrated in the flowchart of FIGS 13 and 14 are both executed so that the three-dimensional point cloud is corrected along both the x-axis and the y-axis.

[0148] FIGS 21 and 22, in combination, illustrate a flowchart representing example steps of detecting and annotating the wire bond defect by utilising the computation of the corrected point cloud. The steps described in FIGS 21 and 22 may be regarded as substeps of Step 104 previously described in FIG. 1.

[0149] The steps of FIGS 21 and 22 may begin with Step 701. Step 701 involves loading the processed point cloud.

[0150] Following Step 701 is Step 702. Step 702 which involves creating two binary maps by utilising data related to the corrected point cloud, wherein the two binary maps comprise a Horizontal Gap Map (HGM) and a Vertical Gap Map (VGM).

[0151] Following Step 702 is Step 703. Step 703 involves determining a depth data of the corrected point cloud.

[0152] Following Step 703 is Step 704. Step 704 involves creating a sliding window with a user-defined length to find a gap by checking maximum and minimum values in the sliding window for each row of the depth data of the point cloud.

[0153] Following Step 704 is Step 705. Step 705 involves creating a Horizontal-Vertical (HV) Intersect Map by utilising an intersecting process between the Horizontal Gap Map (HGM) and the Vertical Gap Map (VGM). The intersecting process may involve the use of the AND operator.

[0154] Following Step 705 is Step 706. Step 706 involves performing masking on the projected surface representation for the point cloud by utilising the HV Intersect Map and storing it as a Projected Surface Representation (PSR).

[0155] Following Step 706 is Step 707. Step 707 involves computing a mean value of nonzero Projected Surface Representation (PSR).

[0156] Following Step 707 is Step 708. Step 708 involves performing thresholding on the projected surface representation for the point cloud using the mean value and a user- defined ratio and storing it as a Raw Wire Bonding Map.

[0157] Following Step 708 is Step 709. Step 709 involves performing a morphological closing operation to filter any one or both noise and blobs in the Raw Wire Map and storing it as a Wire Bonding Map.

[0158] Following Step 709 is Step 710. Step 710 involves loading generated projected surface representation for the point cloud.

[0159] Following Step 710 is Step 711. Step 711 involves duplicating the wire detection map as a colour mask image.

[0160] Following Step 711 is Step 712. Step 712 involves adding annotation information for the wire bonding region on the mask image by setting the wire bond region as a different colour for a different class.

[0161] Following Step 712 is Step 713. Step 713 involves detecting a contour on a non-zero blob region.

[0162] Following Step 713 is Step 714. Step 714 involves measuring a height, width and axis information of the contour.

[0163] Following Step 714 is Step 715. Step 715 involves mapping a contour colour to a label based on a user-defined rule.

[0164] Following Step 715 is Step 716. Step 716 involves storing information of the contour into bounding box object format.

[0165] Following Step 716 is Step 717. Step 717 involves storing the colour mask image and bounding box information in the database 1530.

[0166] FIG. 23 illustrates a flowchart representing steps for performing wire bond defect detection model training by the wire bond defect detection training module 1524. The steps described in FIG. 23 may be regarded as sub-steps of Step 105 previously described in FIG. 1.

[0167] The steps of FIG. 23 may begin with Step 801. Step 801 involves training, by a defect detection model training module 1524, a defect detection model based on the annotated three-dimensional point cloud information.

[0168] Following Step 801 is Step 802. Step 802 involves modifying, by a modifying module 1525, a defect detection model based on the training of the defect detection model. With this, the modified wire bond defect detection model may be subsequently stored in the database 1530 as a trained wire bond defect detection model, per step 111 of FIG. 1.

[0169] FIG. 24 illustrates a flowchart representing example steps of detecting the wire bonding defect, by a wire bond defect detection module 1523 that operates a trained wire bond defect detection model. The steps described in FIG. 24, may be regarded as sub-steps of Step 205 previously described in FIG. 2.

[0170] The steps of FIG. 24 begin with Step 901. Step 901 involves loading a generated projected surface representation of the point cloud that may be previously generated in step 203 of FIG. 2.

[0171] Following Step 901 is Step 902. Step 902 involves performing a wire bond defect detection process on the loaded projected surface representation.

[0172] Finally, following Step 902 is Step 903. Step 903 involves determining a detected wire bond defect of a semiconductor device 1800.

[0173] FIG. 25 illustrates a flowchart representing example steps of training the wire bond defect detection model based on the annotated three-dimensional point cloud information. The steps described in FIG. 25 may be regarded as sub-steps of step 801 previously described in FIG. 23.

[0174] The steps of FIG. 25 may begin with Step 1001. Step 1001 involves loading the generated projected surface representation for the point cloud with the bounding box information.

[0175] Following Step 1001 is Step 1002. Step 1002 involves dividing the loaded data into train and test based on a user-defined ratio.

[0176] Following Step 1002 is Step 1003. Step 1003 involves creating an object detection algorithm using one of machine learning or deep learning techniques.

[0177] FIG. 26 illustrates a flowchart representing steps executed for performing wire bond defect classification model training by the wire bond defect classification training module 1528. The steps described in FIG. 26 may be regarded as sub-steps of Step 110 previously described in FIG. 1.

[0178] The steps of FIG. 26 begin with Step 1101. Step 1101 involves training, by the wire bond defect classification model training module 1529, a three-dimensional wire bond defect classification model.

[0179] Finally, following Step 1101 is Step 1102. Step 1102 involves modifying, by the modifying module 1525, the three-dimensional wire bond defect classification model based on the training executed in step 1102. With this, the modified wire bond defect classification model may be subsequently stored in the database 1530 as a trained wire bond defect classification model, per step 111 of FIG. 1.

[0180] FIG. 27 illustrates a flowchart representing example steps of performing point cloud sampling by the point cloud sampling module 1527. The steps described in FIG. 27, may be regarded as sub-steps of Step 109 previously described in FIG. 1, or sub-steps of Step 211 previously described in FIG. 2.

[0181] The steps of FIG. 27 begin with Step 1201. Step 1201 involves loading the corrected point cloud information.

[0182] Following Step 1201 is Step 1202. Step 1202 involves performing sampling on the corrected point cloud based on a user-defined setting.

[0183] Following Step 1202 is Step 1203. Step 1203 involves storing the sampled point cloud in the database 1530.

[0184] FIG. 28 illustrates a flowchart representing example steps of classifying the wire bonding defect, by a wire bond defect classification module 1528 that operates a trained wire bond defect classification model. The steps described in FIG. 28, may be regarded as sub-steps of Step 212 previously described in FIG. 2.

[0185] The steps of FIG. 28 begin with Step 1301. Step 1301 involves loading the sampled point cloud dataset.

[0186] Following Step 1301 is Step 1302. Step 1302 involves performing the classification process on a loaded sampled point cloud.

[0187] Finally, following Step 1302 is Step 1303. Step 1303 involves determining a classified defect of the semiconductor device 1800.

[0188] FIG. 29 illustrates a flowchart representing example steps of training the wire bond defect classification model. The steps described in FIG. 29 may be regarded as substeps of step 1101 previously described in FIG. 26.

[0189] The steps of FIG. 29 begin with Step 1401. Step 1401 involves loading the sampled point cloud dataset.

[0190] Following Step 1401 is Step 1402. Step 1402 involves dividing a loaded dataset into train and test based on a user-defined ratio.

[0191] Following Step 1402 is Step 1403. Step 1403 involves creating a defect classification algorithm using one of a machine learning or a deep learning technique.

[0192] Whilst not shown, the method of the present invention may further involve a step that involves integrating the detected wire bond defect and the classified wire bond defect, which is followed by a step that involves displaying the integrated result on the display unit 1701.

[0193] Furthermore, it is to be noted that the first stage and the second stage may be implemented in a sequential manner. By way of example, the steps of the first stage and each of their related sub-steps may be performed, and afterwards, the steps of the second stage and each of their related sub-steps may be performed.

[0194] FIG. 30 illustrates a block diagram of a system 1500 for identifying the wire bond defect of the semiconductor device 1800.

[0195] Referring to FIG. 30, the system 1500 comprises the image acquisition device 1510 for capturing the semiconductor device 1800 for obtaining data related to the semiconductor device 1800.

[0196] Referring to FIG. 30, the system 1500 further includes the project surface representation module 1521 for generating the project surface representation of the captured data.

[0197] Referring to FIG. 30, the system 1500 further includes the computing module 1522 operatively configured for extracting the point cloud based on the project surface representation of the captured data.

[0198] Further, the computing module 1522 determines the tilt angle of the extracted point cloud of the one or more regions of interest.

[0199] Preferably, the computing module 1522 performs the rotation of the extracted point cloud by the determined tilt angle for correcting the extracted point cloud, and computing the corrected point cloud.

[0200] Referring to FIG. 30, the system 1500 further comprises the defect classification module 1526 that operates a wire bond classification model for classifying the wire bond defect by utilising the computation of the corrected point cloud.

[0201] Referring to FIG. 30, the system 1500 further includes the wire bond detection and annotation module 1526 for detecting and annotating one or more wire bonds within the extracted point cloud of the one or more regions of interest.

[0202] Referring to FIG. 30, the system 1500 further includes the point cloud sampling module 1527, the wire bond defect detection model training module 1524, the modifying module 1525, the defect classification module 1528, and the wire bond defect classification model training module 1529 which have already been explained in the above paragraphs.

[0203] FIG. 31 illustrates a schematic diagram representing hardware components of the system 1500, according to various embodiments of the present invention.

[0204] Referring to FIG. 31, the hardware components of the system 1500 include the semiconductor device 1800, the image acquisition device 1510, a computer 1700 having a processor or a processing unit 1520, and the display unit 1701.

[0205] It should be noted that the aforementioned modules may be operated by the processor or processing unit 1520. Furthermore, while the aforementioned modules may be in a software embodiment, they may also be a hardware embodiment where they are directly connected to the processor or processing unit 1520. Alternatively, these modules may each be an independent computer system.

[0206] Finally, the processor or processing unit 1520 that may be, but shall not be limited to, a conventional processor, an application-specific integrated circuit (ASIC), a field- programmable gate array (FPGA), a graphics processing unit (GPU), or a combination thereof.

[0207] Advantageously, the method of the present invention improves the defect identification capability by increasing the number of detectable defect categories and reducing computational resources.

[0208] More specifically, at first, the detection method performs defect detection on the projected surface representation of the inspected device which consumes lesser processing power. Thereafter, additional defects missed in the first task will then be subjected to a defect classification step which performs the detection on the point cloud.

[0209] The present disclosure includes as contained in the appended claims, as well as that of the foregoing description. Although this invention has been described in its preferred form with a degree of particularity, it is understood that the present disclosure of the preferred form has been made only by way of example and that numerous changes, modifications, or variations in the details of construction and the combination and arrangements of parts may be resorted to without departing from the scope of the invention.

Claims

CLAIMS1. A method for identifying at least one wire bond defect of a semiconductor device (1800), characterised in that, the method comprising the steps of capturing, by an image acquisition device (1510), the semiconductor device (1800) for obtaining captured data related to the semiconductor device (1800); generating, by a project surface representation module (1521), a project surface representation of the captured data; processing, by a computing module (1522), the projected surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom; and detecting, by a wire bond defect detection module (1523) that operates a wire bond defect detection model, wire bond defects within the projected surface representation of the captured data, wherein the method further involves determining, by the computing module (1522), at least one tilt angle of an extracted point cloud of the one or more regions of interest; performing, by the computing module (1522), a rotation of the extracted point cloud by the determined tilt angle; computing, by the computing module (1522), a corrected point cloud; and classifying, by a wire bond defect classification module (1523) that operates a wire bond defect classification model, the wire bond defect by utilising the computation of the corrected point cloud.

2. The method according to claim 1, further comprising the steps of performing, by a wire bond detection and annotation module (1526), detection and annotation of one or more wire bonds within the extracted point cloud of the one or more regions of interest, which further comprises the steps of loading generated projected surface representation for the point cloud;duplicating wire detection map as a colour mask image; adding annotation information for the wire bonding region on mask image by setting the wire bond region as a different colour for a different class; detecting a contour on a non-zero blob region; measuring a height, width and axes information of the contour; mapping a contour colour to label based on a user-defined rule; storing information of the contour into bounding box object format, and storing the colour mask image and bounding box information into a database (1530); and, training, by a wire bond defect detection model training module (1524), the wire bond defect detection model based on an annotated point cloud data, wherein the steps of training the wire bond defect detection model based on the annotated point cloud data comprises loading the generated projected surface representation for the point cloud with the bounding box information; dividing the loaded data into train and test based on a user- defined ratio; creating an object detection algorithm using one of machine learning or deep learning techniques; modifying, by a modifying module (1525), the wire bond defect detection model based on the training; and storing a modified defect detection model in the database (1530); wherein the point cloud is three-dimensional.

3. The method according to claim 1, wherein the step of generating the projected surface representation of the captured data comprises the steps of normalising coordinates of the captured data to a range of about 0 and 1 in the axes;converting normalised coordinates to a three-column map table, with a first column being column value, a second column being row information, and a third column being an intensity value; and creating the projected surface representation for the captured data by utilising the map table information.

4. The method according to claim 1, wherein the step of determining, by the computing module (1522), the tilt angle of the extracted point cloud of the one or more regions of interest comprises the steps of obtaining a centroid and coordinates of the point cloud; computing distance between the centroid and the coordinates of the extracted point cloud; and detecting the tilt angle of the extracted point cloud, wherein the point cloud is three-dimensional.

5. The method according to claim 1, wherein the step of classifying the wire bond defect by utilising the computation of the corrected point cloud comprises the steps of creating two binary maps by utilising data related to the corrected point cloud, wherein the two binary maps comprise a Horizontal Gap Map,HGM, and a Vertical Gap Map,VGM; determining depth data of the corrected point cloud; creating a sliding window with a user-defined length to find a gap by checking maximum and minimum values in the sliding window for each row of the depth data of the point cloud; creating a sliding window with the user-defined length to find the gap by checking maximum and minimum values in the sliding window for each column of the depth data of the point cloud; creating a Horizontal-Vertical, HV Intersect Map by utilising an intersecting process between the Horizontal Gap Map,HGM, and Vertical Gap Map,VGM, map;performing masking on the projected surface representation for the point cloud by utilising the Horizontal -Vertical, HV Intersect Map and storing as a Projected Surface Representation, PSR; computing a mean value of non-zero Projected Surface Representation, PSR; performing thresholding on the projected surface representation for the point cloud using mean value and user-defined ratio and storing it as a Raw Wire Bonding Map; and performing a morphological closing operation to filter any one or both noise and blobs in a Raw Wire Map and storing it as Wire Bonding Map.

6. The method according to claim 1, wherein the step of classifying, by a wire bond defect classification module (1523) that operates a wire bond defect classification model, the wire bond defect by utilising the computation of the corrected point cloud, further comprises the steps of loading the sampled point cloud; performing classification process on the loaded point cloud; determining one or more classified defects; and displaying the classified defect on a display unit (1601).

7. The method according to claim 6, further comprising the steps of training, by a wire bond defect classification model training module (1429), the wire bond defect classification model, which comprises the steps of, loading the sampled point cloud dataset; dividing loaded data into train and test based on user-defined ratio; creating a defect classification algorithm using one of a machine learning or a deep learning techniques; and modifying, by the modifying module (1525), the wire bond defect classification model based on the training; and storing a modified wire bond defect classification model in the database(1530).

8. The method according to claim 1, further comprising the steps of integrating the detected wire bond defect and the classified wire bond defect; and displaying the integrated result on the display unit (1601).

9. A system (1500) for identifying at least one wire bond defect of a semiconductor device (1800), characterised in that, the system (1500) comprising an image acquisition device (1510) for capturing the semiconductor device for obtaining data related to the semiconductor device (1800); a project surface representation module (1521) for generating a project surface representation of the captured data; a computing module (1522) operatively configured for processing the project surface representation of the captured data to identify and extract at least one point cloud of one or more regions of interest therefrom; determining at least one tilt angle of the extracted point cloud of one or more regions of interest; performing a rotation of the extracted point cloud by the determined tilt angle for correcting the extracted point cloud; and computing at least one corrected point cloud; a wire bond defect detection module (1523) that operates a wire bond defect detection model, for detecting wire bond defects within the projected surface representation of the captured data; and a wire bond defect classification module (1523) that operates a wire bond defect classification model, for classifying the wire bond defect by utilising the computation of the corrected point cloud.

10. The system according to claim 9, further comprising an annotation module (1426) for annotating the point cloud of the identified wire bond defect; a point cloud sampling module (1527) for sampling the annotated point cloud based on user-defined setting; a wire bond defect detection model training module (1524) for training the wire bond defect detection model based on the annotated point cloud data; a modifying module (1525) for modifying the wire bond defect detection model based on the training of the wire bond defect detection model; and a database (1530) for storing the modified defect detection model.

Images