Automated Optical Inspection Using a Hybrid Imaging System

Abstract

A method, product, and system for automated optical inspection (AOI) using a hybrid imaging system. The method includes obtaining a predictive model configured to predict an improved quality image of a product based on a low quality image of the product, the predictive model being generated based on images obtained by a dual scanning system including a low quality scanning system and a high quality scanning system. Based on the low quality image of the product captured using the low quality scanning system, the predictive model is used to predict an improved quality image of the product and utilized for defect detection.

Description

[Technical Field]

[0001] FIELD OF THE DISCLOSURE This disclosure relates generally to automated optical inspection, and more particularly to automated optical inspection performed using a hybrid imaging system. [Background technology]

[0002] Automated optical inspection (AOI) is the automated visual inspection of the output of a manufacturing process. Automated optical inspection may be performed, for example, in flat panel display (FPD) manufacturing, printed circuit board (PCB) manufacturing, etc.

[0003] Automated optical inspection may utilize cameras that autonomously scan devices under test for critical failures (e.g., missing components) and quality defects (e.g., fillet size or shape, component tilt). Automated optical inspection is a non-contact testing method, which may reduce the risk of damaging the product itself. Automated optical inspection can be performed at many points throughout the manufacturing process, including bare board inspection, solder paste inspection (SPI), pre- and post-reflow, etc. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Application Publication No. 2020 / 211178 [Patent Document 2] U.S. Patent Application Publication No. 2020 / 018944 Summary of the Invention [Problem to be solved by the invention]

[0005] The technical problem addressed by the subject matter of this disclosure is to provide an AOI system with high quality results while using hardware of reduced cost and quality. [Means for solving the problem]

[0006] One exemplary embodiment of the subject matter of the present disclosure is a method that includes: obtaining a predictive model, the predictive model configured to predict an improved quality image of a product based on a low quality image of the product, the predictive model being generated based on a pair of images obtained by a dual scanning system including a low quality scanning system and a high quality scanning system; utilizing the low quality scanning system to capture a low quality image of the product; predicting an improved quality image of the product using the predictive model based on the low quality image of the product, the improved quality image having a higher quality than a quality of the low quality image; and performing defect detection on the improved quality image, thereby detecting defects without utilizing the high quality scanning system.

[0007] Optionally, the low-quality scanning system is faster than the high-quality scanning system, thereby detecting defects in a shorter time compared to defect detection based on high-quality images obtained using the high-quality scanning system.

[0008] Optionally, said utilizing, said predicting and said performing defect detection are performed by a student module, the student module comprising a low quality scanning system and lacking a high quality scanning system.

[0009] Optionally, said utilizing, said predicting and said performing defect detection are performed by a teacher module, the teacher module comprising a dual scanning system including a low quality scanning system and a high quality scanning system.

[0010] Optionally, the method further includes the teacher module performing a result evaluation of the defect detection, wherein said performing the result evaluation includes utilizing a high quality scanning system to capture high quality images of the product, performing defect detection on the high quality images, and comparing results of said performing defect detection on the high quality images and said performing defect detection on the improved quality images.

[0011] Optionally, said comparing the results includes identifying substantial differences between defects detected using the high quality image and defects detected using the improved quality image.

[0012] Optionally, identifying a substantial difference includes determining that there is no substantial difference in response to detecting two different non-empty sets of defects.

[0013] Optionally, the method further comprises, in response to determining the difference between the results, adding the low quality images and the high quality images to a training dataset used to retrain the predictive model.

[0014] Optionally, obtaining the predictive model includes obtaining a set of pairs of low-quality images and high-quality images of the product obtained using a dual scanning system, wherein the obtaining of the set of pairs is performed at a customer site; obtaining the set of pairs; and training the predictive model using the set of pairs of low-quality images and high-quality images of the product, thereby generating the predictive model; and training the predictive model, wherein the utilizing the low-quality scanning system to capture the low-quality images of the product is performed at the customer site.

[0015] Optionally, the improved quality image has a lower quality than the quality of an image obtained by a high quality scanning system.

[0016] Another exemplary embodiment of the presently disclosed subject matter is a computer program product including a non-transitory computer-readable storage medium holding program instructions that, when read by a processor, cause the processor to: obtain a predictive model, the predictive model configured to predict an improved quality image of a product based on a low-quality image of the product, the predictive model being generated based on a pair of images acquired by a dual scanning system including a low-quality scanning system and a high-quality scanning system; utilize the low-quality scanning system to capture the low-quality image of the product; predict an improved quality image of the product using the predictive model based on the low-quality image of the product, the improved-quality image having a higher quality than a quality of the low-quality image; and perform defect detection on the improved-quality image, thereby detecting defects without utilizing the high-quality scanning system.

[0017] Yet another exemplary embodiment of the subject matter of the present disclosure is a system comprising: one or more teacher modules, each teacher module including a dual scanning system including a low-quality scanning system and a high-quality scanning system configured to obtain low-quality images and high-quality images, respectively, of a scanned product; a plurality of student modules, each student module including a low-quality scanning system; a model generator configured to generate a predictive model, the predictive model configured to predict an improved-quality image of the product based on the low-quality image of the product, the improved-quality image having a quality higher than a quality of the low-quality image; and a defect detector configured to detect defects using automated optical inspection of images of the product, the defect detector configured to detect defects in the improved-quality image predicted by the predictive model.

[0018] Optionally, the number of said one or more teacher modules is less than the number of the plurality of student modules.

[0019] Optionally, said one or more teacher modules and said plurality of student modules are deployed at a customer site.

[0020] Optionally, the low-quality scanning system is faster than the high-quality scanning system.

[0021] Optionally, the one or more teacher modules are configured to be utilized to collect a training data set for use by the model generator, and the plurality of student modules are configured to be utilized to perform automated optical inspection using images obtained by a low-quality scanning system.

[0022] Optionally, the one or more teacher modules are configured to be utilized to perform automated optical inspection using images obtained by a low-quality scanning system without utilizing a high-quality scanning system.

[0023] The subject matter of the present disclosure will be more fully understood and appreciated from the following detailed description taken in conjunction with the drawings, in which corresponding or identical numbers or symbols indicate corresponding or identical components. Unless otherwise indicated, the drawings provide illustrative embodiments or aspects of the present disclosure and are not intended to limit the scope of the disclosure. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a flowchart diagram of a method according to some example embodiments of the disclosed subject matter. [Figure 2A] FIG. 1 is a flowchart diagram of a method according to some example embodiments of the disclosed subject matter. [Figure 2B] FIG. 1 is a flowchart diagram of a method according to some example embodiments of the disclosed subject matter. [Figure 3A] 1 is a block diagram of an apparatus according to some example embodiments of the subject matter of this disclosure. [Figure 3B] 1 is a block diagram of an apparatus according to some example embodiments of the subject matter of this disclosure. [Figure 4] FIG. 1 is a diagram of a computerized environment in accordance with some example embodiments of the subject matter of this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0025] The technical problem addressed by the subject matter of this disclosure is to provide an AOI system that uses reduced cost and quality hardware while providing high quality results. Additionally or alternatively, it may be desirable to provide an AOI system that can speed up the inspection process, increasing the overall number of products that can be inspected within a given time window.

[0026] One of the technical solutions provided by the subject matter of the present disclosure can include a hybrid AOI system. The hybrid AOI system can include a teacher model and a student model. The teacher module can include a high-quality scanning system and a low-quality scanning system, while the student module can include a low-quality scanning system but not a high-quality scanning system.

[0027] In some exemplary embodiments, the high-resolution scanning system and the low-resolution scanning system may be cameras with high-end and low-end imaging hardware, respectively, capable of capturing high-resolution and low-resolution images, etc. In some exemplary embodiments, the high-quality scanning system may be, for example, a video sensor, while the low-quality scanning system may be an optical scanner. In some exemplary embodiments, the high-quality scanning system may capture pixel values ​​for small pixel sizes, while the low-quality scanning system may capture values ​​for larger pixel sizes. By way of example, the difference in pixel size may be magnitude, e.g., the larger pixel size may be about 5 times larger, about 10 times larger, about 20 times larger, etc. than the smaller pixel size. It should be noted that the terms low-quality and high-quality are relative to each other, and a low-quality scanning system may be a system of lower quality than a high-quality scanning system, but may produce images that are considered high quality in absolute terms, such as 300 dots per inch (DPI), 2540 DPI, 4000 DPI, 8000 DPI, etc. As another example, a low-quality scanning system may utilize scanning equipment configured to acquire high-definition (HD) video, such as 720p, 1080p, 4K, 8K, Ultra HD, etc.

[0028] In some exemplary embodiments, a high-quality scanning system may have a slower scan speed than a low-quality scanning system. In some exemplary embodiments, the scan speed may include the physical scan time, the time to acquire a digital image of the scanned product, etc. A low-resolution image may contain a reduced amount of information and therefore may be obtained in memory correspondingly faster than a high-resolution image. Additionally or alternatively, a reduced image resolution may require less digital storage because less information is represented in the digital storage.

[0029] In some demonstrative embodiments, the teacher module can be deployed to a manufacturing process, such as in a factory, fabrication plant, etc. The teacher module can acquire images associated with a customer site, both high-quality and low-quality scanned images. Machine learning techniques can be used to perform image-to-image mapping between the low-quality and high-quality images. For example, and without loss of generality, deep learning using artificial neural networks (ANNs) can be utilized to enable prediction of high-resolution images based on low-resolution images, such as Pix2Pix, generative adversarial networks (GANs), conditional GANs, CycleGAN™, etc.

[0030] In some demonstrative embodiments, the teacher module may obtain a relatively large dataset of pairs of low-resolution and high-resolution images of the same product. The relatively large dataset may include, for example, a set of more than 10,000 pairs, a set of more than 50,000 pairs, a set of more than 100,000 pairs, a set of more than 500,000 pairs, a set of more than 1,000,000 pairs, etc. The image pairs may be taken at the same location on the product being imaged (e.g., PCB, FPB, etc.) such that the objects are aligned.

[0031] After completing the training process, a predictive model is generated and ready for use. Note that the predictive model can be trained to predict images of intermediate quality, higher than that provided by a low-quality scanning module and lower than that provided by a high-quality scanning system. For example, given that a low-quality scanning system may have a pixel size of 5 microns and a high-quality scanning system may have a pixel size of 0.5 microns, the predictive model can be configured to predict images with pixel sizes between 1 micron and 3 microns, between 1.25 microns and 2.5 microns, etc.

[0032] In some demonstrative embodiments, the training dataset may be obtained during production, such as at a customer site using actual examples. Because each customer site may tend to have similar products, e.g., built with identical hardware and similar characteristics, the site-specific training dataset may provide a reference for training the predictive model to achieve relatively high prediction accuracy.

[0033] In some exemplary embodiments, a student module may be employed in production to acquire only low-resolution images. Such images are fed to a prediction module to predict an improved-quality image, such as one having intermediate quality. The improved-quality image may be fed to an optical scanning defect detection algorithm to identify defects. Defect detection based on the predicted improved-quality image may be referred to as an artificial intelligence (AI)-based detection process.

[0034] In some example embodiments, a teacher module can be employed in production in a similar manner to the student module to perform an AI-based detection process, but utilizing only low-quality scanning systems and not high-quality scanning systems.

[0035] Additionally or alternatively, the teacher module can be utilized for evaluation purposes. It should be noted that the predictive quality of such models can be affected by several factors, such as differences in imaging techniques, sensor degradation, lighting conditions, customer processes, the use of different materials, data generation processes, and the resulting products. As one example, in FPD or PCB manufacturing, image pixels vary due to differences in the color of the FPD or PCB being manufactured, the use of different materials in fabrication, and the like. Such differences can affect the reflectance, transmittance, and the like of the image being classified. As another example, in FPD or PCB manufacturing, small but constant, rapid changes may be made to the resulting products to adapt them to customer needs, to accommodate new design features, and the like. As a result, image predictions may require refinement and updating over time. Evaluations can be performed to identify when improved-quality images should not be relied upon. Evaluations can include scanning samples using both a low-quality scanning system and a high-quality scanning system and determining whether defects identified using the improved-quality images based on the low-quality scanning system are identical to defects identified based on the product from the high-quality scanning system. In instances where the evaluation process identifies defects as going undetected, use of the predictive model may be suspended until the predictive model is improved and can be used to achieve a sufficiently high accuracy threshold (e.g., for 1 million samples, less than 1 error per 1 million samples, each defect item being detected with at least one defect (even if different from the true defect)). As an example, a customer may define that errors of up to 0.05%, 0.1%, 0.15%, etc. of true defects may be considered undetected. If the evaluation underperforms in terms of such a user-defined threshold, the predictive model may be retrained before being reused. In some exemplary embodiments, in instances where the evaluation identifies a problem, data obtained during the evaluation, and specifically data associated with different defect detections, may be utilized to retrain the predictive model.

[0036] It should be noted that during evaluation, different defect detections may be deemed acceptable in some cases. As an example, if defects are identified, the product may be discarded. In such cases, it may be sufficient for the AI-based detection to correctly classify the product as defective / non-defective, even if the AI-based detection identifies false defects, a false number of defects, etc. However, for retraining purposes, even low-resolution and high-resolution image pairs that were correctly classified but in which the AI-based detection did not correctly detect all defects (detecting only true defects) can be used for retraining to improve the accuracy of the AI-based detection process. In some exemplary embodiments, retraining can be performed using adaptive training.

[0037] A technical effect of utilizing the subject matter of this disclosure may be to use AI analytics to enhance system capabilities of AOI machines with lower quality scanning systems. In some cases, teacher modules may be more expensive and less available than cheaper, more widely available student modules. Using the subject matter of this disclosure, a single teacher module may be utilized with multiple student modules that rely on predictive models generated using the single teacher module.

[0038] Another technical effect of utilizing the subject matter of the present disclosure may be improving the scanning speed by an AOI system. In some cases, the speed may be improved by increasing the number of devices, such as by introducing additional teacher modules. Additionally or alternatively, the speed may be improved by improving the speed of a teacher module with dual scanning systems (a high-quality scanning system and a low-quality scanning system) by utilizing a low-quality scanning system to quickly scan products and utilize AI-based detection without activating a slower but higher-quality high-quality scanning system.

[0039] Yet another technical effect of utilizing the subject matter of this disclosure is to provide a training process that is decoupled from external computing and databases and adapted to customer site conditions. The subject matter of this disclosure allows for increased accuracy of improved quality images without revealing data about the factory or fabrication plant utilizing AI-based detection to AOI system developers or other external parties.

[0040] Yet another technical effect of utilizing the subject matter of the present disclosure is shortening the time to market (TTM) required for a product from conception to sale. TTM can be important in industries where products quickly become obsolete, particularly in the world of microelectronics, such as FPDs and PCBs. TTM can be shortened given that a larger number of modules can be employed per budget. Additionally or alternatively, TTM can be shortened by increasing the scanning speed of a dual system, such as a teacher module.

[0041] The subject matter of the present disclosure may provide one or more technical improvements over any existing techniques and techniques previously routine or customary in the art. Additional technical problems, solutions, and advantages will be apparent to those skilled in the art in view of the present disclosure.

[0042] Referring now to FIG. 1, a flowchart diagram of a method is shown, according to some example embodiments of the disclosed subject matter.

[0043] In step 100, a training dataset is obtained. In some exemplary embodiments, the training dataset may include pairs of images (low-quality and high-quality images of the same product), also referred to as (low, high). In some exemplary embodiments, the low-quality image may be obtained using a low-quality scanning system to scan the product, and the high-quality image may be obtained using a high-quality scanning system to scan the same product. In some exemplary embodiments, the two images may be aligned to match one another. Note that the sensors used to capture the images may be located on the same device, also referred to as the teacher module, but may be located in different locations, positioned at different angles, etc. In some exemplary embodiments, one image may be pre-processed to be aligned with the other image. For example, the low-quality image may be transformed using one or more linear transformations to the position depicted in the high-quality image. As another example, the high-quality image may be transformed to match the position in the low-quality image. Note that the transformations applied may be determined automatically or manually. The transformations may be consistent for each teacher module (or type thereof). In some exemplary embodiments, the transformation may be predetermined and may be based on installation parameters of the sensors, such as location within the module, distance between sensors, field of view angle, etc.

[0044] In some exemplary embodiments, the (low, high) pairs can be obtained by a teacher module at a customer site. Additionally or alternatively, the training dataset can be aggregated from multiple deployed teacher modules at the same site. Note that in some cases, different sites may have different characteristics of their electronic products. Accordingly, the training dataset for each site may be different and may include samples from the same site or from sites whose products share the same characteristics.

[0045] In some example embodiments, the training dataset may include a dataset obtained at a customer site that represents an electronic product that will be produced at the customer site and analyzed by the AOI system. Additionally, or alternatively, the training dataset may consist of an initial base dataset that may be provided by the manufacturer of the teacher module and represents a common use case for the AOI system.

[0046] In step 110, the training dataset of step 100 may be utilized to train a predictive model. In some exemplary embodiments, the training may be performed on-premise by a teacher module. Additionally or alternatively, the training may be performed on-premise by a different device, such as a server, computer, etc. Additionally or alternatively, the training may be performed at a remote location, such as the cloud, by a remote server using a cloud computing platform, etc. The training may be training of a model, such as a decision tree-based model, an ANN-based model, a deep convolutional neural network, etc. In some exemplary embodiments, the training may be based on techniques such as, but not limited to, Pix2Pix™, generative adversarial networks (GANs), conditional GANs, CycleGAN™, etc.

[0047] In step 120, the predictive model generated in step 110 is transferred to a module for use in AOI based on a low-quality scanning system. In some exemplary embodiments, the predictive model may be transferred to a student module operating at the same site where the training dataset was obtained. Additionally or alternatively, the predictive module may be transferred to one or more teacher modules, such as the teacher module used to collect the training dataset. The teacher module may utilize the predictive model to perform AOI based on a low-quality scanning system without utilizing its own high-quality scanning system. Additionally or alternatively, the one or more teacher modules may evaluate the performance of the predictive model. In some exemplary embodiments, the evaluation may be performed over time to ensure that the AI-based detection process provides sufficiently accurate results.

[0048] Referring now to FIG. 2A, a flowchart diagram of a method according to some exemplary embodiments of the presently disclosed subject matter is shown. The method of FIG. 2A can be implemented by a computerized system that includes a low-quality scanning system, such as a student module. Additionally or alternatively, the method of FIG. 2A can be implemented by a computerized system having dual-quality scanning systems (e.g., a high-quality scanning system and a low-quality scanning system), such as a teacher module. In such cases, the computerized system can avoid utilizing both scanning systems and rely only on the low-quality scanning system.

[0049] In step 200, a predictive model is obtained. After the predictive model is generated in FIG. 1 , the predictive model can be obtained. In some exemplary embodiments, the predictive model may be obtained by receiving the model over a computerized communication medium, such as a wired connection, a wireless connection, a computerized network, etc. Additionally or alternatively, the predictive model may be obtained from local storage. The predictive model may be generated on the same device (e.g., in the case of a teacher module) or on another device (e.g., a different teacher module, a server, a cloud computing platform, etc.).

[0050] In step 210, an image is obtained using a low-quality scanning system. In some exemplary embodiments, the low-quality scanning system can be invoked to obtain an image of the product being reviewed by the AOI system. For example, the low-quality image can be captured by a camera sensor imaging a PCB or FPD manufactured at the site.

[0051] In step 220, an improved quality image is generated. The improved quality image may be generated using a predictive model. The predictive model may be applied to the images obtained in step 210 to predict the improved quality image. In some exemplary embodiments, pre-processing of the low-quality images may be performed prior to application of the predictive model to align the images according to the expectations of the predictive model (e.g., if the training dataset was aligned in a similar manner). Additionally or alternatively, the model's predicted images may be processed and transformed to provide the improved quality image, such as by performing a transformation that is the inverse of the transformation performed on the high-quality images of the training dataset.

[0052] In step 230, defect detection can be applied to the improved quality image. Defect detection can be performed by any means, including, but not limited to, utilizing a dedicated, user-specified algorithm, applying a classification engine, utilizing AI-based classification techniques, applying a machine learning model, etc. Based on the results of the defect detection, the AOI system can provide relevant output to the user, such as indicating defective products, providing a log of defects in the products, indicating defective and non-defective products, etc. In some exemplary embodiments, products identified as defective can be discarded automatically, manually, semi-automatically, etc.

[0053] Referring now to Figure 2B, a flowchart diagram of a method according to some exemplary embodiments of the presently disclosed subject matter is shown. The method of Figure 2B can be implemented by a computerized system having dual quality scanning systems (e.g., a high-quality scanning system and a low-quality scanning system), such as a teacher module.

[0054] In step 215, a high quality image can be obtained of the same product for which a low quality image was obtained in step 210. The high quality image can be obtained using a high quality scanning system.

[0055] In step 240, defect detection can be applied to the high quality image obtained in step 215. In some exemplary embodiments, the defect detection can be the same defect detection utilized in step 230, a different defect detection mechanism, etc. A list of defects detected in step 230 can be stored in List l ={d1,d2,…,d n}, and the list of defects detected in step 240 can be represented as List h ={d1,d2,…d m Each list can be written as a different defect (d i ) Additionally or alternatively, the list may be an empty set.

[0056] In step 250, the defects identified in steps 230 and 240 may be compared. If identical defects are detected (List l =List h ), the method may be terminated, including if no defects were detected in both images. However, if there are differences in the list (List l ≠List h ), step 260 can be performed.

[0057] In step 260, the pair of images obtained in steps 210 and 215 may be added to a training dataset. Because the defect detection system detected different defects, the two images are added to the training dataset and can be used during retraining to improve the predictive model and enable more accurate AI-based defect detection in the future. Note that the retraining may utilize the original training dataset, may utilize a portion of it, or may discard it entirely.

[0058] In step 270, for evaluation purposes, it may be determined whether there is a substantial difference between the defects detected in both cases. In some exemplary embodiments, any difference may be considered a substantial difference. In such cases, step 270 may be omitted (given the determination made in step 250). Additionally or alternatively, a substantial difference may be the difference when one defect list is empty and the other is not empty (e.g., List l =φ∧List h ≠φ, or List h =φ∧List l Additionally or alternatively, some defects may be considered to be of the same class, and two lists containing defects with the same class may be considered to be substantially identical (e.g., ∀d i ∈List l ,∃d j ∈List h ,stclass(di )=class(d j )∧∀d j ∈List h ,∃d i ∈List l ,stclass(d i )=class(d j Additionally or alternatively, if two lists contain the same number of defects at the same (x,y) locations in the image space of the PCB panel space (|List h |=|List l Only if two lists contain a similar number of defects within a threshold (||List h |-|List l The lists can be considered substantially identical only if ||≦threshold). Additional metrics for determining substantial similarity may be utilized, such as based on a combination of the above examples, additional metrics, etc. In some exemplary embodiments, substantial differences between the two lists may correspond to different results regarding the product or actions performed regarding the product. As such, the AI-based detection produces results that differ from those given based on high-quality images. In some exemplary embodiments, substantial differences may simply be considered false-negative type differences. In a false-negative scenario, the AI-based detection may erroneously indicate that a product is not defective, even though high-quality images indicate that the product has at least one defect. In such cases, the AOI system may not be able to prevent the use of the defective product. In some cases, false positives (e.g., erroneously indicating a non-defective product as defective) may be considered acceptable in some scenarios or at some rate, while false negatives may be completely unacceptable or only at a low rate.

[0059] In some exemplary embodiments, if there is a substantial difference, step 280 may be performed. A counter may be incremented and a decision may be made whether to retrain the predictive model. In some exemplary embodiments, after identifying a substantial difference threshold, the predictive model may be retrained to prevent erroneous results. Retraining may be performed using the method of FIG. 1 or a similar method.

[0060] Referring now to FIG. 3A, an apparatus is shown in accordance with some exemplary embodiments of the subject matter of this disclosure.

[0061] In some demonstrative embodiments, device 300a, also referred to as a teacher module, may include one or more processors 302. Processor 302 may be a central processing unit (CPU), a microprocessor, an electronic circuit, an integrated circuit (IC), etc. Processor 302 may be utilized to perform calculations required by device 300a or any of its subcomponents.

[0062] In some exemplary embodiments of the subject matter of this disclosure, the device 300a may include an input / output (I / O) module (not shown). The I / O module may be utilized to provide output to a user. Additionally or alternatively, the I / O module may be utilized to receive input from a user. Additionally or alternatively, the I / O module may be utilized to communicate with other devices, such as the student module 300b of FIG. 3B, other teacher modules, remote servers, etc.

[0063] In some exemplary embodiments, apparatus 300a may include a high-quality scanning system 360 and a low-quality scanning system 370. In some exemplary embodiments, scanning systems 360, 370 may be configured to scan electronic products to be inspected by an AOI process and provide high-quality and low-quality images, respectively. Note that the terms “high-quality” and “low-quality” may be relative to one another. In some exemplary embodiments, the scanning systems may be the same type of system, e.g., both optical camera sensors, both video cameras, etc. Additionally or alternatively, the scanning systems may be different types of systems, e.g., high-quality scanning system 360 may be an HD video camera while low-quality scanning system 370 may be an optical camera.

[0064] In some demonstrative embodiments, the device 300a may include a memory unit 307. The memory unit 307 may be a hard disk drive, a flash disk, a random access memory (RAM), a memory chip, or the like. In some demonstrative embodiments, the memory unit 307 may retain program code operable to cause the processor 302 to perform operations associated with any of the subcomponents of the device 300a. The memory unit 307 may include one or more components implemented as executable files, libraries, static libraries, functions, or any other executable components, as described in more detail below. In some demonstrative embodiments, the memory unit 307 may be configured for the predictive model 315, a training data set, a set of pairs of low-resolution and high-resolution images of the same product obtained during a learning phase, during production, etc., the improved quality images 378 generated thereby, etc. In some demonstrative embodiments, the high-quality images 365 obtained by the high-quality scanning system 360 may be retained by the memory unit 307. Additionally or alternatively, the low quality image 375 obtained by the low quality scanning system 370 may be retained by the memory unit 307 .

[0065] In some demonstrative embodiments, the model generator 310 may be utilized to obtain a training set including a set of paired images of the same product having different qualities (e.g., high-quality image 365, low-quality image 375) to generate the predictive model 315. In some demonstrative embodiments, the model generator 310 may be configured to train the predictive model 315 given the training data set. Additionally or alternatively, the model generator 310 may utilize machine learning-based techniques to train the predictive model 315.

[0066] In some exemplary embodiments, the predictive model 315, after training, can be configured to predict improved quality images 378 based on low quality images 375.

[0067] In some exemplary embodiments, defect detector 320 may be configured to detect defects in images of a product. In some exemplary embodiments, defect detector 320 may be applied to high quality images 365, low quality images 375, improved quality images 378, etc. In some exemplary embodiments, defect detector 320 may utilize proprietary, user-specified algorithms, classification engines, AI-based classification techniques, machine learning models, etc. to detect defects.

[0068] In some exemplary embodiments, the result comparator 330 may be configured to compare defects detected by the defect detector 320 against two images of the same product having different qualities, such as a high quality image 365 and an improved quality image 378.

[0069] In some demonstrative embodiments, in response to the model generator 310 generating the predictive model 315, the predictive model 315 may be distributed to other devices, such as other teacher modules, student modules, etc.

[0070] Referring now to FIG. 3B, an apparatus is shown in accordance with some exemplary embodiments of the subject matter of this disclosure.

[0071] In some exemplary embodiments, device 300b, also referred to as a student module, may include a processor 302, a low-quality scanning system 370, and a memory unit 307. In some exemplary embodiments, student module 300b may include a single, relatively low-quality scanning system (370), in contrast to the dual scanning system of teacher module 300a. In some exemplary embodiments, student module 300b may receive a predictive model 315 trained based on data collected by teacher module 300a and utilize the predictive model 315 to increase the quality of a scanned image from a low-quality image 375 to an improved-quality image 378. Defects may be detected by applying defect detector 320 to the improved-quality image 378 rather than the low-quality image 375.

[0072] Referring now to FIG. 4, a diagram of a computerized environment is shown, in accordance with some example embodiments of the subject matter of this disclosure.

[0073] 3A, includes a teacher module 410, such as 300a, and multiple student modules 420, such as 300b. Note that there may be multiple teacher modules 410. In some exemplary embodiments, the number of teacher modules 410 may be less than the number of lower-cost student modules 420 utilized in the same environment.

[0074] In some demonstrative embodiments, the teacher module 410 can be utilized to obtain training data that is utilized to generate the predictive model. The predictive model can be generated using pairs of images that are locally available on the teacher module 410. In some demonstrative embodiments, other teacher modules (not shown) can transmit data collected by them to the teacher module 410 for utilization in the generation of the predictive model. Additionally or alternatively, the generation of the predictive model can be performed by a server 415, which can be a local server or a remote server. The server 415 can receive the data collected by the teacher module 410, such as via the network 405, and utilize such data to generate the predictive model.

[0075] In some demonstrative embodiments, after a predictive model is generated (e.g., by teacher module 410 or by server 415), the predictive model may be distributed to other modules in the environment, such as other teacher modules, student module 420, etc.

[0076] In some exemplary embodiments, during the production process, an evaluation of the quality of the predictions may be performed by some of the teacher modules 410, all of the teacher modules 410, etc. In some exemplary embodiments, discrepancies between the detected images using the high-quality images and the improved-quality images may be retained for training purposes. In some exemplary embodiments, retraining may be invoked after certain conditions are met, such as determining substantial differences between the detected defects in both cases, finding a percentage above a threshold of the number of products with identified substantial differences compared to the number of products analyzed, or finding an absolute number of products with substantial differences between the detected defects in both cases.

[0077] In some exemplary embodiments, the teacher module 410 and the student module 420 may be deployed at the same customer site. Additionally or alternatively, all modules 410, 420 may be utilized as part of an AOI process for the same type of electronic product.

[0078] The present invention may be a system, a method, and / or a computer program product, which may include a computer-readable storage medium having computer-readable program instructions for causing a processor to perform aspects of the present invention.

[0079] A computer-readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded devices such as punch cards or grooved structures having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as being ephemeral signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or electrical signals transmitted over electrical wires.

[0080] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to an individual computing / processing device or to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium within the individual computing / processing device.

[0081] The computer-readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer as a standalone software package, partly on the user's computer and partly on a remote computer, or all on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., via the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer readable program instructions to individualize the electronic circuitry by utilizing state information in the computer readable program instructions to perform aspects of the present invention.

[0082] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0083] These computer-readable program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to make a machine, such that the instructions, which execute on the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium, directing a computer, programmable data processing apparatus, and / or other device to function in a particular manner, such that the computer-readable storage medium having the instructions stored thereon comprises an article of manufacture including instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0084] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to create a computer-implemented process and cause the computer, other programmable apparatus, or other device to perform a series of operable steps, such that the instructions executing on the computer, other programmable apparatus, or other device perform the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0085] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may, in fact, be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or actions or executes a combination of dedicated hardware and computer instructions.

[0086] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise. It should be further understood that the terms "comprises" and / or "comprising," when used herein, disclose the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0087] The corresponding structure, material, acts, and equivalents of all means or step-plus-function elements in the following claims are intended to include any structure, material, or acts for performing that function in combination with other specifically claimed claim elements. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the inventive form disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the invention. The embodiments were chosen and described to best explain the principles and practical application of the invention and to enable others skilled in the art to understand the invention in various embodiments with various modifications that are suitable for the particular uses contemplated.

Claims

1. one or more teacher modules for obtaining a predictive model, the teacher modules including a dual scanning system including a low-quality scanning system and a high-quality scanning system configured to obtain low-quality and high-quality images of a scanned product, respectively; obtaining a prediction model comprising: a plurality of student modules including the low-quality scanning system and not including the high-quality scanning system, the prediction model being configured to predict an improved-quality image of a product based on a low-quality image of the product, the prediction model being generated based on a pair of images obtained by the low-quality scanning system and the high-quality scanning system of the teacher module; utilizing the low quality scanning system of the student module to capture a low quality image of a product; predicting an improved quality image of the product using the predictive model based on the low quality image of the product, the improved quality image having a higher quality than a quality of the low quality image; performing defect detection on the improved quality image, thereby detecting defects without utilizing the high quality scanning system; A method comprising:

2. 2. The method of claim 1, wherein the low-quality scanning system is faster than the high-quality scanning system, thereby detecting defects in a shorter time compared to defect detection based on high-quality images obtained using the high-quality scanning system.

3. 2. The method of claim 1, wherein said utilizing, said predicting, and said performing said defect detection are performed by said student module.

4. 2. The method of claim 1, wherein said utilizing, said predicting, and said performing said defect detection are performed by said teacher module.

5. 5. The method of claim 4, further comprising the teacher module performing a result evaluation of the defect detection, wherein the performing a result evaluation comprises: utilizing said high quality scanning system to capture high quality images of said product; performing defect detection on the high-quality image; comparing the results of performing defect detection on the high-quality image and performing defect detection on the improved-quality image; A method comprising:

6. 6. The method of claim 5, wherein the comparing of results includes identifying substantial differences between defects detected using the high-quality image and defects detected using the improved-quality image.

7. 7. The method of claim 6, wherein identifying a substantial difference includes determining that there is no substantial difference in response to detecting two different non-empty sets of defects.

8. 6. The method of claim 5, further comprising, in response to determining a difference between the results, adding the low-quality images and the high-quality images to a training data set used to retrain the predictive model.

9. 10. The method of claim 1, wherein the obtaining the predictive model comprises: obtaining a set of pairs of low quality and high quality images of the product obtained using the dual scanning system, wherein said obtaining said set of pairs is performed at a customer site; training the predictive model using the set of pairs of low quality and high quality images of the product, thereby generating the predictive model; wherein said utilizing said low quality scanning system to capture said low quality image of said product is performed at said customer site.

10. 2. The method of claim 1, wherein the improved quality image has a lower quality than the quality of the image obtained by the high quality scanning system.

11. one or more teacher modules, each including a dual scanning system including a low-quality scanning system and a high-quality scanning system configured to obtain low-quality and high-quality images, respectively, of a product being scanned; a plurality of student modules, each student module including the low quality scanning system and not including the high quality scanning system; a model generator configured to generate a predictive model, the predictive model being configured to predict an improved-quality image of a product based on a low-quality image of the product using a pair of images obtained by the low-quality scanning system and the high-quality scanning system of the teacher module, the improved-quality image having a quality higher than that of the low-quality image; a defect detector configured to detect defects using automated optical inspection of images of a product, the defect detector configured to detect defects in improved quality images predicted by the predictive model from low quality images obtained by the low quality scanning system of the student module; A system comprising:

12. 12. The system of claim 11, wherein the number of said one or more teacher modules is less than the number of said plurality of student modules.

13. 12. The system of claim 11, wherein the one or more teacher modules and the plurality of student modules are located at a customer site.

14. 12. The system of claim 11, wherein the low quality scanning system is faster than the high quality scanning system.

15. 12. The system of claim 11, wherein the one or more teacher modules are configured to be utilized to collect training data sets used by the model generator, and the plurality of student modules are configured to be utilized to perform the automated optical inspection using images obtained by the low-quality scanning system.

16. 16. The system of claim 15, wherein the one or more teacher modules are configured to be utilized to perform the automated optical inspection using images obtained by the low-quality scanning system without utilizing the high-quality scanning system.

17. 1. A computer program product including a non-transitory computer-readable storage medium bearing program instructions, the program instructions, when read by a processor, causing the processor to one or more teacher modules for obtaining a predictive model, the teacher modules including a dual scanning system including a low-quality scanning system and a high-quality scanning system configured to obtain low-quality and high-quality images of a scanned product, respectively; obtaining a prediction model comprising: a plurality of student modules including the low-quality scanning system and not including the high-quality scanning system, the prediction model being configured to predict an improved-quality image of a product based on a low-quality image of the product, the prediction model being generated based on a pair of images obtained by the low-quality scanning system and the high-quality scanning system of the teacher module; utilizing the low quality scanning system of the student module to capture a low quality image of a product; predicting an improved quality image of the product using the predictive model based on the low quality image of the product, the improved quality image having a higher quality than a quality of the low quality image; performing defect detection on the improved quality image, thereby detecting defects without utilizing the high quality scanning system; and A computer program product that causes the computer to perform the steps described above.

18. 20. The computer program product of claim 17, wherein the low-quality scanning system is faster than the high-quality scanning system, thereby detecting defects in a shorter time compared to defect detection based on high-quality images obtained using the high-quality scanning system.

19. 20. The computer program product of claim 17, wherein said utilizing, said predicting, and said performing said defect detection are performed by a student module.

20. 20. The computer program product of claim 17, wherein the utilizing, predicting, and performing the defect detection are performed by a teacher module.

21. 18. The computer program product of claim 17, wherein obtaining the predictive model comprises: obtaining a set of pairs of low quality and high quality images of the product obtained using the dual scanning system, wherein said obtaining said set of pairs is performed at a customer site; training the predictive model using the set of pairs of low-quality and high-quality images of the product, thereby generating the predictive model; wherein said utilizing said low-quality scanning system to capture said low-quality image of said product is performed at said customer site.

22. 20. The computer program product of claim 17, wherein the improved quality image has a lower quality than the quality of an image obtained by the high quality scanning system.

Images