Detecting defects in semiconductor devices based on images of said semiconductor devices in an inspection process

Abstract

An inspection process for detecting one or more defects in a source image of a semiconductor device. A presence of a first defect in the semiconductor device is determined by processing the source image through a first object detection model to obtain a first indication whether or not the first defect has been detected. A location of a marking on the semiconductor device is determined by processing the source image through a second object detection model, the marking comprising one or more letters, numbers and / or symbols. The marking is extracted from the source image in a marking extraction process based on the determined location of the marking and OCRed. A defect class is output through a defect classification model based on any the first indication, an indication of a problem in the extracting of the marking, and an indication of a problem in the OCR.

Description

[0001] Detecting defects in semiconductor devices based on images of said semiconductor devices in an inspection process

[0002] Technical field

[0003] The present disclosure relates to optical inspection of semiconductor devices. More specifically, the present disclosure relates to detecting defects in semiconductor devices based on images of said semiconductor devices in an inspection process.

[0004] Background

[0005] During and / or after manufacturing, semiconductor packages may be inspected for defects. For example, automated optical inspection (AOI) systems may be used in manufacturing and quality control processes of electronic components, including semiconductor packages, printed circuit boards (PCBs), and other electronic assemblies, utilizing cameras and image processing algorithms to inspect components for defects.

[0006] An AOI process typically begins with the acquisition of high-resolution images of the component under inspection. AOI systems may use one camera positioned above the component or multiple cameras positioned at different angles to capture images of the component from various perspectives. These cameras may utilize different lighting techniques, such as bright-field, dark-field, or coaxial lighting, to enhance the visibility of features and defects.

[0007] Once the images are acquired, image processing algorithms are applied to analyze the images. These algorithms are designed to detect various types of defects, including surface abnormalities, solder joint defects, component misalignments, and other manufacturing flaws. Image processing techniques such as edge detection, pattern recognition, and deep learning may be employed to identify and classify defects accurately.

[0008] The AOI system may compare the acquired images against predefined criteria or reference images to detect anomalies and defects. Defects are typically classified based on their type, severity, and impact on product quality. Common types of defects detected by AOI systems include solder joint voids, bridges, opens, misalignments, component polarity errors, missing components, and solder paste deposition issues.

[0009] Summary

[0010] A summary of aspects of certain examples disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these certain embodiments and that these aspects are not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects and / or a combination of aspects that may not be set forth.

[0011] According to an aspect of the present disclosure, an inspection process for detecting one or more defects in a semiconductor device based on a source image of the semiconductor device is presented. The inspection process may include receiving the source image. The inspection process may further include determining a presence of a first defect in the semiconductor device by processing the source image through a first object detection model to obtain a first indication whether or not the first defect has been detected. The inspection process may further include determining a location of a marking on the semiconductor device by processing the source image through a second object detection model. The marking may comprise one or more letters, numbers and / or symbols. The inspection process may further include extracting the marking from the source image in a marking extraction process based on the determined location of the marking The inspection process may further include applying an optical character recognition (OCR) to the extracted marking. The inspection process may further include outputting a defect class through a defect classification model based on any one of the first indication, an indication of a problem in the extracting of the marking, and an indication of a problem in the OCR.

[0012] Preferably, the marking includes one or more of a data code, a batch code and a package code.

[0013] In an embodiment, the inspection process may further include extracting a specific feature of the semiconductor device from the source image through a segmentation model. The inspection process may further include classifying the specific feature through a classification model to obtain a second indication whether or not the specific feature could be extracted and is accepted by the classification model. The defect classification model may determine the defect class further based on the second indication.

[0014] In an embodiment, The inspection process may further include extracting one or more leads of the semiconductor device from the source image through a leads segmentation model. The inspection process may further include classifying the leads through a leads classification model to obtain the second indication whether or not the leads could be extracted and are accepted by the leads classification model. The inspection process may further include extracting a body part of the semiconductor device from the source image through a body segmentation model. The inspection process may further include classifying the body part through a body classification model to obtain a third indication whether or not the body part could be extracted and is accepted by the body classification model. The defect classification model may determine the defect class further based on the third indication.

[0015] In an embodiment, the segmentation model and the classification model may be combined in a single model.

[0016] In an embodiment, the models are deep learning models.

[0017] In an embodiment, the inspection process may further include post-processing the source image to determine further aspects of the semiconductor device and obtain further details of defects in the source image. The defect classification model may determine the defect class further based on the further details.

[0018] In an embodiment, the inspection process may further include outputting an indication that no defects have been detected by any one of the models and no processing errors have occurred.

[0019] In an embodiment, the inspection process may further include producing a performance report based on the defect class and indicative of which of the models produced an indication that a defect has been detected.

[0020] In an embodiment, the inspection process may further include updating or replacing one or more of the models that produced an indication that a defect has been detected.

[0021] In an embodiment, the updating may comprise training the one or more models.

[0022] In an embodiment, the training may use one or more source images for which no defects have been detected by any one of the models of the inspection process and no processing errors have occurred in the inspection process. In an embodiment, the replacing may comprise replacing a model of the inspection process with a model from another inspection process.

[0023] According to an aspect of the present disclosure, a computer system is proposed. The computer system may comprise one or more deep learning models and an OCR module. The computer system may be configured to implement the inspection process having one or more of the above described features.

[0024] In an embodiment, the computer system may comprise one or more processors and a memory storing the one or more deep learning models. The one or more processors may be configured to execute the one or more deep learning models to produce an output based on an input source image in accordance with the inspection process having one or more of the above described features.

[0025] According to an aspect of the present disclosure, an automated optical inspection (AOI) system is presented. The AOI system may comprise a computer system as described above.

[0026] Brief description of the Drawings

[0027] Embodiments of the present disclosure will now be described, by way of example only, with reference to the accompanying schematic drawings in which corresponding reference symbol indicate corresponding parts, in which:

[0028] Figs. 1A-1C show examples of source images of semiconductor packages including leads and a body part including a marking;

[0029] Fig. 2 shows an inspection process of an example embodiment of the present disclosure; and

[0030] FIG. 3 shows an example embodiment of a computing system for implementing certain aspects of the present technology.

[0031] The figures are intended for illustrative purposes only, and do not serve as restriction of the scope of the protection as laid down by the claims.

[0032] Detailed description

[0033] It will be readily understood that the components of the embodiments as generally described herein and illustrated in the appended figures could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of various embodiments, as represented in the figures, is not intended to limit the scope of the present disclosure but is merely representative of various embodiments. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0034] The described embodiments are to be considered in all respects only as illustrative and not restrictive. The scope of the present disclosure is, therefore, indicated by the appended claims rather than by this detailed description. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.

[0035] Reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the present disclosure should be or are in any single example of the present disclosure. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present disclosure. Thus, discussions of the features and advantages, and similar language, throughout this specification may, but do not necessarily, refer to the same example.

[0036] Furthermore, the described features, advantages, and characteristics of the present disclosure may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize, in light of the description herein, that the present disclosure may be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the present disclosure. Reference throughout this specification to "one embodiment," "an embodiment," or similar language means that a particular feature, structure, or characteristic described in connection with the indicated embodiment is included in at least one embodiment of the present disclosure. Thus, the phrases "in one embodiment," "in an embodiment," and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.

[0037] One of the fault detection steps in a manufacturing process of a semiconductor device may include an AOL In the AOI process, an image may be taken of the semiconductor device, such as source image 100 shown in Figs. 1A-1C. In Figs. 1A- 1C a top view of a semiconductor package is shown in the source image 100, including leads 102 and a body part 104 including a marking. In Fig. 1A, the semiconductor package includes three leads 102 and a body part 104 including a marking comprising a batch code “A6W” and a date code “35”. In Fig. 1 B, the semiconductor package includes three leads 102 and a body part 104 including a marking comprising a batch code “3BW” and a date code “38”. In Fig. 1 C, the semiconductor package includes two leads 102 and a body part 104 including a marking comprising a package code “HO”.

[0038] A date code may be a code including letters, numbers and / or other symbols indicative of a production date that, consequently, changes periodically. E.g., the semiconductor device in the source image 100 of Fig. 1A may be produced in month 35 from the start of production of this type of semiconductor device and the semiconductor device in the source image 100 of Fig. 1 B may be produced in month 38 from the start of production. The AOI system may be configured to recognize the date code in the source image 100, but due to the dynamic nature of the date code, in known AOI systems the image recognition modules may not be able to do so all the time.

[0039] A batch code may be a code include letters, numbers and / or other symbols and is indicative of a batch production of semiconductor packages.

[0040] A package code may be a code including letters, numbers and / or other symbols and is indicative of the type of package of the semiconductor package.

[0041] A semiconductor package may include one or more of a date code, a batch code, a package name and any other code that may be recognized by the AOI system.

[0042] When introducing a new type of semiconductor device to the AOI system, e.g., the two-pins semiconductor package in the source image 100 of Fig. 1C, to an AOI system trained for inspecting another type of semiconductor device, e.g., the three- pins semiconductor package in the source image 100 of Fig. 1 B, although the two packages have similar features (i.e. , leads 102 and a body part 104 with markings), known AOI systems may not be able to successfully recognize the new type of semiconductor device and incorrectly conclude that the two-pins semiconductor device is defective.

[0043] Changes in the source image 100 as a result of, e.g., changes to the structure of the semiconductor device or changes in the markings on the semiconductor device, may thus lead, in known inspection systems, to a, for the inspection system, defect image and an incorrect inspection result. In known systems it can be difficult to pinpoint the cause of the incorrect inspection result, typically requiring time consuming debugging and possibly resulting in unnecessary updating of AOI detection modules. The present disclosure presents a solution to these problems.

[0044] Fig. 2 shows an inspection process 200 of an example embodiment of the present disclosure. The inspection process 200 may detect the presence of a defect in a sample image 100 of a semiconductor device, such as a semiconductor package including leads 102 and a body part 104. Input to the inspection process 200 is the sample image 100 of the manufactured semiconductor device. Output of the inspection process 200 is a defect class 230 identifying a type of defect or an indication 240 that no defect is detected, which may result in reclaiming the sample of the semiconductor device of which the sample image 100 was taken.

[0045] Detection of a defect in a sample image 100 either means that there is a problem in identifying or extracting features of the semiconductor device from the sample image 100 or that there is a defect in the semiconductor device. The present disclosure aims to resolve the problem of identifying or extracting features from the sample image 100 by identifying where the problem exists and (re)training the involved module(s) of the inspection process 200.

[0046] The inspection process 200 may implement deep learning models 202-210 for detecting and / or classifying characteristics of the semiconductor device based on the sample image 100, an optical character recognition (OCR) process 220-222 and a post-processing process 224.

[0047] The deep learning models 202-210 may be based on deep learning models. Deep learning models are a class of deep learning models capable of learning hierarchical representations of data through layers of interconnected nodes. There are several popular architectures that may be used for the deep learning models, including, but limited to, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), gated recurrent units (GRUs), deep belief networks (DBNs) and restricted Boltzmann machines (RBMs). CNNs are particularly suitable for processing images.

[0048] Defects in the semiconductor device may be detect based on the image 100 by an object detection model 202. The object detection model 202 may detect various types of defects. Examples of such defects are solder joint voids; bridges; opens; misalignments; missing components; solder paste deposition issues; and missing, illegible or unrecognizable batch codes and / or date codes. Output of the object detection model 202 may be an indication whether or not a defect has been detected. The object detection model 202 may, e.g., output results indicative of chipping, tin particle, metal burr, empty pocket, foreign material, and etcetera. The output of the object detection model 202 may be in any usable form, such as “pass” or “reject”, “pass” or “fail”, “true” or “false”, “1” or “0”, and etcetera, for not having detected a defect versus having detected a defect, respectively.

[0049] Optionally, to improve detectability of where a defect is present in the source image 100, specific features of the semiconductor device, in this example the leads 102 and the body 104 of a semiconductor package, may be extracted from the image 100 using segmentation models 204a, 206a. Leads segmentation model 204a may extract the leads 102 from the sample image 100, illustrated in Fig. 2 as the output of leads segmentation model 204a only including the leads 102 from the source image 100. Body segmentation model 206a may extract the body part from the sample image 100, illustrated in Fig. 2 as the output of body segmentation model 206a only including the body part 104 including the batch code and date code from the source image 100.

[0050] Feature classification models 204b, 206b may be applied to the extracted features 102, 104 after the extraction of these features using the segmentation models 204a, 206a. Leads classification model 204b may, e.g., determine whether or not leads 102 can be identified in the image 100, if the leads 102 resemble any known leads trained by the model, and etcetera. Body classification model 206b may, e.g., determine whether or not a body part 104 can be identified in the image 100, if the body part 104 resembles any known body parts trained by the model, and etcetera. For each of the extracted features 102, 104 it may thus be determined whether or not the feature is accepted or rejected, based on the respective deep learning model 204b, 206b. Outputs of the feature classification models 204b, 206b may be an indication whether or not the features could be extracted and / or have been accepted by the model. The outputs of the feature classification models may be in any usable form, similar to the object detection model 202.

[0051] Segmentation and classification processes may be combined in a single model. For example, leads segmentation model 204a and leads classification model 204b may be implemented as a single deep learning model taking original image 100 as input and producing a leads classification output (e.g., pass or reject). For example, body segmentation model 206a and body classification model 206b may be implemented in a single deep learning model taking original image 100 as input and producing a body classification output (e.g., pass or reject).

[0052] An object detection model 208 may be used to detect the locations of markings, typically in the form of letters, numbers and / or symbols, on the semiconductor device from the source image 100. For example, the batch code and / or the date code may be printed on the outside of the semiconductor package. The date code may be in any form, e.g., including or consisting of an indication of a day, week, month and / or year and possibly including an indication of a production time. Preferably, the date code includes an indication of a month and / or a year. In the example of Fig. 2, the batch code equals “5CW” and the date code equals “99”. The object detection model 208 need not output the exact markings, but preferably outputs the location or locations in the source image 100 where the markings are found.

[0053] A marking extraction process 222 may extract the part(s) of the source image 100 including the markings from the source image 100 at the location or locations detected by the object detection model 208. The marking extraction process 222 may be programmed to rotate part(s) of the image including the markings, if needed, and, e.g., split the batch code (e.g., “5CW”) from the date code (e.g., “99”) in the image. Next, an OCR process 224 may be used to obtain the characters and numbers forming the batch code and data code from the extracted part(s) of the source image 100.

[0054] The object detection model 202 provides an indication whether or not a defect is detected (e.g., pass or reject) in the sample image 100. A post-processing 226 may be used to further check aspects of the semiconductor device in the source image 100 when no defects were detected by the object detection model 202, such as the angle of the body part 104 and the sizes and shape of leads 102, to further process any possible defects.

[0055] If any of the deep learning models 202, 204b, 206b classifies the source image 100 as including a defect (e.g., outputs a reject) or if any processing step, such as the marking extraction process 222, the OCR process 224 or the post-processing 226, fails, the source image 100 may be considered as a defect image and passed to a defect classification model 210 to perform a defect classification. The defect classification model 210 may output a defect class 230, indicative of the type of defect detected. The type of defect is, e.g., an indication of a defect detected in the object detection model 202 indicative of a defect in the semiconductor device in the source image 100, an indication of a problem in the detectability of the leads 102 in the leads classification model 204b, an indication of a problem in the detectability of the body part 104 in the body classification model 206b, an indication of a markings extraction problem in the marking extraction process 222, an indication of an OCR problem in the OCR process 224 and / or an indication of a problem in the post-processing process 226.

[0056] The defect class 230 may be used as a debugging aid to identify where a particular problem occurs. If, e.g., the body classification model 206b outputs a reject, there may be a problem in the recognition of that specific part of the semiconductor device and deep learning models operating on the body part in source images may need to be trained using images or part of images including this specific part of other, known to be good images, e.g., images passing the inspection process 200. Alternatively, the involved deep learning model may be replaced with another deep learning model from another inspection process. Advantageously, the defect class 230 enables the cause of the defect to be pinpointed and only the affected deep learning model(s) need to be updated or replaced when an image recognition problem has been detected.

[0057] If no defects are detected and all processes finish without problems, the inspection process 200 may output an indication 240 that processing of the source image 100 is performed without any failures or errors. The sampled semiconductor device of the sample image 100 may then be reclaimed as an approved product, if needed, and the source image 100 may be used to positively train any of the models 202-210.

[0058] The multistage modular inspection process, such the inspection process 200 of Fig. 2, incorporates an integrated reporting system that may generate comprehensive performance reports for one or more, preferably all models 202-208. The performance reports may be based on or in the form of the defect class 230 and serve as a valuable tool in identifying the root causes of subpar performance. This is particularly advantageous in scenarios where a factory changes defect codes regularly, e.g., on a monthly basis, resulting in the date code on the semiconductor package to be changed.

[0059] By leveraging the reporting system, users can swiftly pinpoint the specific model 202-208 responsible for the under-performance. This expedites the troubleshooting process, enabling timely identification of the factors contributing to the reduce in performance. With this knowledge at hand, users can focus their attention on retraining the problematic model, utilizing a small subset of relevant images.

[0060] This approach offers significant efficiency gains, as users no longer need to retrain an entire inspection system or spend excessive time investigating the causes of performance degradation. Instead, they can target the specific model 202-210 in question, providing it with the necessary training data to restore its performance to the desired level.

[0061] A factory typically processes numerous semiconductor packages that exhibit similarities in their characteristics. For example, a factory may process packages of Fig. 1 B and Fig. 1C, both including leads 102 and a body part 104 with markings, although the packages are different (i.e., a two-pins package in Fig. 1 B and a 2-pins package in Fig. 1C). In order to expedite the development cycle, the inspection process 200 offers a high degree of flexibility. This flexibility allows users to efficiently utilize existing models by reusing, removing, or reordering them to swiftly construct a solution tailored to the requirements of specific semiconductor packages.

[0062] Furthermore, users can take advantage of the modular nature of the models 202- 210, enabling selective reuse of one or more of the models 202-210 that have proven effective in another inspection process for similar semiconductor packages, eliminating the need to develop new models from scratch. This approach significantly reduces development time and effort, while still ensuring accurate and reliable results.

[0063] Moreover, the solution of the present disclosure allows for the removal of unnecessary models from an inspection process that may not be relevant to a particular semiconductor package. This streamlines application development processes by eliminating redundant components, resulting in a more concise and efficient solution.

[0064] The flexibility offered by the solution of the present disclosure empowers users to leverage existing models effectively and efficiently. This approach enables rapid development of tailored applications for specific semiconductor packages within the factory, reducing development cycles and enhancing productivity. The reporting feature within the multistage modular approach facilitates rapid identification of performance issues, precise troubleshooting, and streamlining of the retraining process. This ensures that even with periodic, e.g., monthly, changes to the defect code, users can swiftly regain optimal performance by selectively addressing the responsible model. By breaking down the inspection process into modular components, it becomes easier to accommodate and update specific features without the need to retrain the entire system. This approach optimizes the training process and avoids unnecessary time wastage associated with retraining all features from scratch.

[0065] Furthermore, the modular nature of the solution enables efficient management of similar packages within the factory. Instead of treating each package independently, the modular model allows for shared components and selective customization, reducing redundancy and streamlining the training process.

[0066] In terms of debugging, the multistage modular model provides enhanced ease and efficiency. With the system divided into modular components, developers can isolate and focus on specific modules during the debugging process. This targeted approach simplifies the identification of issues and accelerates the resolution of system malfunctions or errors, ultimately reducing the overall debugging time.

[0067] The present disclosure is not limited to the specific implementation of the inspection process 200 as shown in Fig. 2.

[0068] For example, a semiconductor device may include different or more features that may be extracted from the source image 100 by other or further segmentation models, such as the segmentation models 204a, 206a, and classified by other or further classification models, such as the classification models 204b, 206b.

[0069] For example, in Fig. 2 the classification results from the models 202, 204b and 206b are shown to be provided to the defect classification model 210 via the marking extraction process 222. Alternatively, one or more of these classification results may be provided to the defect classification model 210 directly or via any other process or module of the inspection process 200.

[0070] The inspection process shown in Fig. 2 may be implemented in one or more computer systems. For example, the marking extraction process 222, the OCR process 224 and / or the post processing 226 may be implemented in a computer device 220. One or more of the other deep learning models 202-210 may also be implemented in the computer device 220. Alternatively, one or more of the deep learning models 202-210 and processes 222-226 may be implemented in different communicatively connected computer systems.

[0071] In an embodiment, the inspection process 200 may be implemented in a standalone server located on-site, functioning as a centralized hub for analyzing defect images from multiple AOI machines. Each of the defect images from the AOI machines may then be processed as a sample image 100 as shown in Fig. 2.

[0072] FIG. 3 shows an example embodiment of a computing system 300 for implementing certain aspects of the present technology. In various examples, the computing system 300 may be any computing device incorporating any of the deep learning models 202-210, incorporating any of the processes 222-226, making up the computer device 220, or any other computing system described herein. In some implementations, a computing system 300 may implement the methods described herein, such as the inspection process 200 of the present disclosure.

[0073] The computing system 300 may include any component of a computing system described herein, which components may be in communication with each other using connection 305. The connection 305 may be a physical connection via a bus, or a direct connection into processor 310, such as in a chipset architecture. The connection 305 may also be a virtual connection, networked connection, or logical connection.

[0074] In some implementations, the computing system 300 may be a distributed system in which the functions described in this disclosure may be distributed within a datacenter, multiple datacenters, a peer network, and etcetera. In some embodiments, one or more of the described system components represents many such components each performing some or all of the functions for which the component is described. In some embodiments, the components may be physical or virtual devices.

[0075] The example system 300 includes at least one processing unit (CPU or processor) 310 and a connection 305 that couples various system components including system memory 315, such as read-only memory (ROM) 320 and randomaccess memory (RAM) 325 to processor 310. The computing system 300 may include a cache of high-speed memory 312 connected directly with, in close proximity to, or integrated as part of the processor 310.

[0076] The processor 310 may include any general-purpose processor and a hardware service or software service, such as services 332, 334, and 336 stored in storage device 330, configured to control the processor 310 as well as a special-purpose processor where software instructions are incorporated into the actual processor design. The processor 310 may essentially be a completely self-contained computing system, containing multiple cores or processors, a bus, memory controller, cache, and etcetera. A multi-core processor may be symmetric or asymmetric. To enable user interaction, the computing system 300 may include an input device 345, which may represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, keyboard, mouse, motion input, speech, and etcetera. The computing system 300 may also include an output device 335, which may be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems may enable a user to provide multiple types of input / output to communicate with the computing system 300. The computing system 300 may include a communications interface 340, which may generally govern and manage the user input and system output. There is no restriction on operating on any particular hardware arrangement, and therefore the basic features here may easily be substituted for improved hardware or firmware arrangements as they are developed.

[0077] A storage device 330 may be a non-volatile memory device and may be a hard disk or other types of computer readable media which may store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, solid state memory devices, digital versatile disks, cartridges, random access memories (RAMs), read-only memory (ROM), and / or some combination of these devices.

[0078] The storage device 330 may include software services, servers, services, and etcetera, that, when the code that defines such software is executed by the processor 310, causes the system to perform a function. In some embodiments, a hardware service that performs a particular function may include a software component stored in a computer-readable medium in connection with the necessary hardware components, such as processor 310, connection 305, output device 335, and etcetera, to carry out the function.

[0079] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims. In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope thereof.

Claims

CLAIMS1. An inspection process (200) for detecting one or more defects in a semiconductor device based on a source image (100) of the semiconductor device, comprising: receiving the source image; determining a presence of a first defect in the semiconductor device by processing the source image through a first object detection model (202) to obtain a first indication whether or not the first defect has been detected; determining a location of a marking on the semiconductor device by processing the source image through a second object detection model (208), the marking comprising one or more letters, numbers and / or symbols, the marking preferably including one or more of a data code, a batch code and a package code; extracting the marking from the source image in a marking extraction process (222) based on the determined location of the marking; applying an optical character recognition, OCR (224), to the extracted marking; and outputting a defect class (230) through a defect classification model (210) based on any one of the first indication, an indication of a problem in the extracting of the marking, and an indication of a problem in the OCR.

2. The inspection process according to claim 1 , further comprising: extracting a specific feature (102, 104) of the semiconductor device from the source image through a segmentation model (204a, 206a); and classifying the specific feature through a classification model (204b, 206b) to obtain a second indication whether or not the specific feature could be extracted and is accepted by the classification model, and wherein the defect classification model determines the defect class further based on the second indication.

3. The inspection process according to claim 2, comprising: extracting one or more leads (102) of the semiconductor device from the source image through a leads segmentation model (204a);classifying the leads through a leads classification model (204b) to obtain the second indication whether or not the leads could be extracted and are accepted by the leads classification model; extracting a body part (104) of the semiconductor device from the source image through a body segmentation model (206a); and classifying the body part through a body classification model (206b) to obtain a third indication whether or not the body part could be extracted and is accepted by the body classification model, and wherein the defect classification model determines the defect class further based on the third indication.

4. The inspection process according to any one of the claims 3-4, wherein the segmentation model (204a, 206a) and the classification model (204b, 206b) are combined in a single model.

5. The inspection process according to any one of the preceding claims, wherein the models (202-210) are deep learning models.

6. The inspection process according to any one of the preceding claims, further comprising: post-processing (226) the source image to determine further aspects of the semiconductor device and obtain further details of defects in the source image, and wherein the defect classification model determines the defect class further based on the further details.

7. The inspection process according to any one of the preceding claims, further comprising outputting an indication (240) that no defects have been detected by any one of the models and no processing errors have occurred.

8. The inspection process according to any one of the preceding claims, further comprising producing a performance report based on the defect class (230) and indicative of which of the models (202-208) produced an indication that a defect has been detected.

9. The inspection process according to any one of the preceding claims, further comprising updating or replacing one or more of the models (202-208) that produced an indication that a defect has been detected.

10. The inspection process according to claim 9, wherein the updating comprises training the one or more models.

11. The method according to claim 10, wherein the training uses one or more source images for which no defects have been detected by any one of the models of the inspection process and no processing errors have occurred in the inspection process.

12. The inspection process according to claim 9, wherein the replacing comprises replacing a model of the inspection process with a model from another inspection process.

13. A computer system (300) comprising one or more deep learning models and an optical character recognition, OCR, module, wherein the computer system is configured to implement the inspection process according to any one of the claims 1- 12.

14. The computer system according to claim 13, comprising one or more processors and a memory storing the one or more deep learning models, wherein the one or more processors are configured to execute the one or more deep learning models to produce an output based on an input source image (100) in accordance with the inspection process of any one of the claims 1-12.

15. An automated optical inspection system comprising a computer system according to any one of the claims 13-14.

Images