System and method for automatic defect identification

Abstract

An image inspection computing device (104) is provided. The device (104) includes a memory device (310, 410) and at least one processor (305, 405). The at least one processor (305, 405) is configured to receive at least one sample image (202) of a first part (202), wherein the at least one sample image (202) of the first part does not include a defect (258), store the at least one sample image (202) in the memory (310, 410), and receive an input image (206, 252) of a second part. The at least one processor (305, 405) is further configured to generate an encoding array based on the input image (206, 252) of the second part, perform a random data sampling process on the encoding array, generate a decoding array, and generate a reconstructed image (254) of the second part, the reconstructed image being derived from the random data sampling process and the decoding array. The at least one processor (305, 405) is further configured to produce a residual image (210, 256) and to identify a defect (258) of the second part.

Description

[0001] Priority Information

[0002] This application claims priority to U.S. Patent Application No. 16 / 845,324, filed April 10, 2020. TECHNICAL FIELD

[0003] The field of the invention relates generally to systems and methods of inspecting components, and in particular to systems and methods of automatically detecting defects in components. BACKGROUND

[0004] At least some known methods of inspecting components require human (e.g., human expert) detection, which is subject to the subjective interpretation of the human expert. Further systems and methods of detecting defects utilize and require a large amount of sample data to enable the inspection system to recognize what a defect in a component looks like. In other words, some systems learn what a defect in a component looks like by receiving a large amount of sample data and discriminating defects in defect samples of the sample data. Typically, a classifier is assigned to a defect in a component so that the system can learn what the defect looks like. However, the amount of defect samples is typically much less than the amount of conforming samples, and it can be relatively difficult to find a sample of every possible defect that can occur in a component because the various possible defects that can occur in a component can vary greatly. Moreover, known training solutions will only detect defects represented in the training data (e.g., defect samples). Defects not represented in the training data will not be detected by the known solutions.

[0005] Accordingly, there is a need for inspection systems that can be trained without requiring a large amount of defect samples, while also improving the efficiency of detecting defects in components. SUMMARY

[0006] In one aspect, a method of inspecting a component using an image inspection controller is provided, the image inspection controller including a processor communicatively coupled with a memory. The method includes receiving, at the processor, at least one sample image of a first component, wherein the at least one sample image of the first component does not include a defect, storing the at least one sample image in the memory, and receiving, at the processor, an input image of a second component. The method further includes generating, by the processor, an encoding array based on the input image of the second component, performing, by the processor, a random data sampling process on the encoding array, and generating, by the processor, a decoding array from the random data sampling process based at least in part on the at least one sample image. The method further includes generating, by the processor, a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image, comparing, by the processor, the reconstructed image of the second component to the input image of the second component, producing, by the processor, a residual image, the residual image including at least one difference between the reconstructed image of the second component and the input image of the second component if the second component has a defect, and identifying, by the processor, whether the second component has a defect based on the residual image.

[0007] In another aspect, an image inspection computing device is provided. The image inspection computing device includes a memory device and at least one processor communicatively coupled with the memory device. The at least one processor is configured to receive at least one sample image of a first component, wherein the at least one sample image of the first component does not include a defect, store the at least one sample image in the memory, and receive an input image of a second component. The at least one processor is further configured to generate an encoding array based on the input image of the second component, perform a random data sampling process on the encoding array, generate a decoding array from the random data sampling process based at least in part on the at least one sample image, and generate a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image. The at least one processor is further configured to compare the reconstructed image of the second component to the input image of the second component, produce a residual image, the residual image including at least one difference between the reconstructed image of the second component and the input image of the second component if the second component has a defect, and identify a defect of the second component based on the residual image.

[0008] In another aspect, a non-transitory computer-readable storage medium having computer-executable instructions embedded thereon is provided. When executed by a computing device comprising at least one processor in communication with a memory, the computer-executable instructions cause the computing device to receive at least one sample image of a first part, wherein the at least one sample image of the first part does not include a defect, store the at least one sample image in the memory, receive an input image of a second part, and generate an encoding array based on the input image of the second part. The computer-executable instructions further cause the computing device to perform a random data sampling process on the encoding array, generate a decoding array from the random data sampling process based at least in part on the at least one sample image, and generate a reconstructed image of the second part derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image. The computer-executable instructions further cause the computing device to compare the reconstructed image of the second part to the input image of the second part, produce a residual image that includes at least one difference between the reconstructed image of the second part and the input image of the second part if the second part has a defect, and identify a defect of the second part based on the residual image. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a schematic diagram illustrating an example of an example image inspection (II) computing system.

[0010] Figure 2A is a diagram illustrating an example data flow for detecting defects as implemented by the II computing system shown in Figure 1

[0011] Figure 2B is a diagram of an example defect detection process as implemented by the II computing system shown in Figure 1

[0012] Figure 3 is a block diagram of an example user computing device that can be used in the II computing system shown in Figure 1

[0013] Figure 4 is a block diagram of an example server computing system that can be used in the II computing system shown in Figure 1

[0014] Figure 5 is a flowchart of an example method of automatically detecting defects in accordance with the present disclosure.

[0015] Figure 6 is a diagram of components of one or more example computing devices that can be used in the II computing system shown in Figure 1 DETAILED DESCRIPTION

[0016] ​​​​​This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the present invention is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent

[0017] Described herein are systems and methods for automatic defect identification. The systems and methods described herein automatically detect defects by receiving sample input images of components that are free of defects. These sample input images and normal data derived from the sample images are used to reconstruct input images (e.g., two-dimensional or three-dimensional) of components to be inspected. The reconstructed images are then compared to input images of components to be inspected to create residual images that highlight defects in the input images of the components. Rather than requiring many sample images of defective components and comparing these images to input images of components to detect defects in the components, the systems and methods described herein utilize at least one sample input image of a component that is free of defects. This greatly reduces the time, cost, and inaccuracy associated with trying to find sample input images for every possible defect that can occur in a component.

[0018] Technical problems addressed by the present invention include at least one of: (a) subjectivity of human inspection of components; (b) labor-intensive processing to design and incorporate data augmentation strategies during defect detection; (c) time-consuming processing to label defect data to train defect detection systems; (d) large amounts of defect samples for training defect detection systems; (e) heavy data augmentation that limits variability of known systems and methods to detect defects; (f) need for pre-processing of images, such as denoising; and (g) detection of only defects represented as defects in training data or defect samples.

[0019] The technical effects achieved by the systems and methods described herein include at least one of: (a) receiving, at a processor, at least one sample image of a first component, wherein the at least one sample image of the first component does not include a defect; (b) storing, in a memory, the at least one sample image; (c) receiving, at the processor, an input image of a second component; (d) generating, by the processor, an encoding array based on the input image of the second component; (e) performing, by the processor, a random data sampling process on the encoding array; (f) generating, by the processor, a decoding array from the random data sampling based at least in part on the at least one sample image; (g) generating, by the processor, a reconstructed image of the second component derived from the random data sampling and the decoding array, the reconstructed image generated based at least in part on the at least one sample image; (h) comparing, by the processor, the reconstructed image of the second component to the input image of the second component; (i) producing, by the processor, a residual image, the residual image including at least one difference between the reconstructed image of the second component and the input image of the second component if the second component has a defect; and (j) discriminating, by the processor, whether the second component has a defect based on the residual image.

[0020] The technical benefits achieved by the systems and methods of the present disclosure include at least one of: (a) automatically inspecting components; (b) eliminating labor-intensive processes of designing and integrating data augmentation strategies during defect detection; (c) eliminating time-consuming processes of annotating defect data to train a defect detection system; (d) not requiring defect samples to train a defect detection system; (e) not requiring heavy data augmentation, thereby creating variability of the systems and methods described herein; (f) not requiring denoising or image preprocessing; and (g) detecting all defects and not requiring defect samples.

[0021] Figure 1 FIG. 1 is a schematic diagram illustrating an example image inspection (II) computing system 100 for automatically detecting defects in components.

[0022] The II computing system 100 includes a server 102 that includes at least one II computing device 104 and a database server 106, where the II computing device 104 (i.e., an image inspection controller) is in communication with at least one database 108 and at least one user computing device 110.

[0023] In example embodiments, the user computing device 110 (e.g., a smartphone, a tablet, a laptop, etc.) is configured to transmit inputs, such as sample component images, to the II computing device 104. In example embodiments, the sample component images include images of components that do not have defects. The user computing device 110 is also configured to receive outputs, such as a residual image 210 (as shown in Figure 2A

[0024] ​The database server 106 can be in communication with a database 108 containing information about various matters, as described in greater detail below. In one embodiment, the database 108 is stored on the server 102 and can be accessed by the user computing device 110 logging into the server 102 and / or the II computing device 104. In another embodiment, the database 108 can be stored remotely from the server 102 and can be non-centralized. In some embodiments, the database 108 can be distributed across multiple locations (e.g., the server 102, the database 108, etc.) and / or exist in a cloud environment.

[0025] In example embodiments, the II computing device 104 is configured to receive images of sample parts and / or subject parts and determine defects in any of the received subject parts, as described in greater detail below. The images received by the II computing device 104 and used to determine defects can be two-dimensional or three-dimensional images. Notably, the II computing device 104 is configured to determine defects using normal data (e.g., conforming data) and does not require any manipulation of the input images, manual enhancement of the image data, and / or defect annotation of the image data.

[0026] Figure 2A is a diagram illustrating use of Figure 1 A diagram of example data flow 200 of defect detection implemented by the II computing system 100 shown.

[0027] In example embodiments, the user computing device 110 is configured to transmit a first part sample image 202 of a first part to the II computing device 104. The first part sample image 202 includes a sample image of a part that is free of defects. In some embodiments, multiple sample images of parts that are free of defects can be transmitted from the user computing device 110 to the II computing device 104. Using multiple sample images (as opposed to a single sample image) can be beneficial by accounting for minor differences between sample images that are not defects, but can include minor manufacturing variations that are not in fact defects. Upon receiving the first part sample image 202 from the user computing device 110, the II computing device 104 is configured to generate a normal data sampling 204 of the first part sample image and transmit the normal data sampling 204 of the first part sample image to the database 108 such that the normal data sampling 204 of the first part sample image is stored in the database 108.

[0028] In an example embodiment, the II computing device 104 is configured to then receive a second component input image 206 of a second component from the user computing device 110. The second component input image 206 is an image of a component similar to the first component sample image 202. However, the second component associated with the second component input image 206 may include defects. Therefore, the II computing device 104 is configured to automatically detect defects in the second component input image 206.

[0029] In an example embodiment, the II computing device 104 is configured to generate a reconstructed image of the second component input image 206, such that the reconstructed image can be compared with the second component input image 206 to detect defects in the second component. The II computing device 104 uses normal data sampling 204 of a first component sample image to generate the reconstructed image. Therefore, the II computing device 104 is configured to, upon receiving the second component input image 206, generate a first component sample image request 208 and transmit the first component sample image request 208 to a database 108. The database 108 is configured to then transmit normal data sampling 204 of the first component sample image to the II computing device 104.

[0030] Figure 2B Is using Figure 1 The diagram illustrates an example defect detection process 250 implemented by the II computing system 100. In this example embodiment, the II computing device 104 is configured to use normal data sampling 204 of a first component sample image, based on the second component input image 252 (and...). Figure 2A The second component input image 252 (corresponding to the "second component input image 206") is used to generate a reconstructed image 254 of the second component through a random process. For example, the second component input image 252 includes a defect 258 in the second component. To detect the defect 258, the II computing device 104 generates the reconstructed image 254 of the second component by randomly sampling 204 using normal data from the first component sample image to correct the defect 258 in the second component input image 252. The reconstructed image 254 is then compared with the second component input image 252 to detect the defect 258. It is worth noting that the reconstructed image 254 does not contain the defect.

[0031] In an example embodiment, the II computing device 104 determines defects in the second component input image 252 in a two-step process. First, by comparing the second component input image 252 with the reconstructed image 254, a residual image 256 (also referred to as a similarity map and / or similarity index, compared with...) is generated. Figure 2A (Corresponding to "residual image 210" in the text). For example, a comparison can be made using the following formula:

[0032] S_m = SIMILARITY(I,g(I))

[0033] where I represents the second component input image 252, g(I) represents the reconstructed image 254, SIMILARITY is an operator used to estimate the similarity between the second component input image 252 and the reconstructed image 254, and S m represents the residual image / similarity map 256. The residual image 256 is then used by the II computing device 104 to determine whether there are any defects 258 in the second component input image 252. In some examples, the second component input image 252 can not include defects. In Figure 2B In the example embodiment shown, the second component input image 252 includes at least seven defects, for example as shown in the residual image 256, three of which are indicated as defects 258. The II computing device 104 determines the defects in the second component input image 252 by classifying the portions of the residual image 256 that are“high” (i.e., brighter areas) as possibly containing defects, while the portions of the residual image 256 that are“low” (i.e., darker areas) do not contain defects. To determine which of the high (i.e., brighter) areas contain defects, a quality index can be generated by the II computing device 104.

[0034] The quality index can be generated by the II computing device 104, for example, by using the following equation:

[0035] Q_I = ImageQI(S_m)

[0036] where S m is the previously generated similarity map, ImageQI is an operator used to estimate the quality index, and QI is the quality index. The quality index can include, for example, a value in the range of zero to ten, with a low value representing a lower quality and a higher value representing a higher quality. A lower value of the quality index indicates a higher likelihood that the corresponding portion of the residual image 256 contains a defect.

[0037] In some embodiments, the II computing device 104 can determine an acceptance threshold to use with the quality index. The acceptance threshold is determined for the component such that portions of the residual image with a quality index score above the acceptance threshold are determined by the II computing device 104 to be free of defects, while portions of the residual image with a quality index score below the acceptance threshold are determined by the II computing device 104 to be defective. In some embodiments, if the quality index score is equal to a predetermined threshold, the associated portion of the residual image can be determined to be defective. In other embodiments, if the quality index score is equal to a predetermined threshold, the associated portion of the residual image can be determined to be free of defects.

[0038] A quality index score can be determined for the entire residual image, each individual pixel in the residual image, and / or portions of pixels in the residual image. In some embodiments, a quality index score can first be determined for the entire image, and if the quality index score is below a certain threshold, a quality index score is determined for portions of the residual image to determine which particular areas of the residual image contain defects. This narrowing process can be repeated until particular portions of the residual image are identified as containing defects.

[0039] The threshold to be used with the quality index score can be manually input by a user on the user computing device 110 in communication with the II computing device 104, or automatically determined by the II computing device 104. In embodiments where the II computing device 104 determines the threshold, the II computing device 104 can determine the threshold by comparing the quality index scores of portions of the sample image 202 that do not have defects to the quality index scores of portions of the image that have defects. In these embodiments, the II computing device 104 is configured to determine the threshold so that the quality scores associated with pixels and / or groups of pixels that have relatively minor defects will be greater than the threshold. Thus, minor manufacturing defects will not be determined to be defects by the II computing device 104 because the quality scores associated with the pixels and / or groups of pixels will be greater than the threshold. In some examples, if the sample image 202 has a generally low quality score associated with it, the threshold can be lower because the sample image 202 does not have defects but still has a low quality score. In other examples, if the sample image 202 has a generally high quality score associated with it, the threshold can be higher.

[0040] Referring back to Figure 2A In determining whether the residual image 210 contains defects, the II computing device 104 is configured to transmit the residual image 210 to the user computing device 110 and the database 108. In some embodiments, the II computing device 104 can be configured to mark the residual image 210 to identify defects, as shown in the residual image 256 in Figure 2B In some embodiments, the II computing device 104 can transmit a signal to the user computing device 110 indicating that the residual image 210 contains defects.

[0041] Figure 3 An example configuration of a user system 302 that can be operated by a user 301 is illustrated. In example embodiments, the user system 302 is similar to the user computing device 110 (as shown in Figure 1 and can be used by the user 301 to communicate with the II computing device 104 (also in Figure 1The user system 302 can be used by the user 301 to, for example, transmit the first component sample image 202 and the second component input image 206 (both shown in FIG. 2). In example embodiments, the user system 302 includes a processor 305 for executing instructions. In some embodiments, the executable instructions are stored in a memory area 310. The processor 305 can include one or more processing units, such as a multi-core configuration. The memory area 310 can include one or more computer-readable media.

[0042] The user system 302 also includes at least one media output component 315 for presenting information to the user 301, such as the residual image 210. The media output component 315 is any component capable of conveying information to the user 301. In some embodiments, the media output component includes an output adapter, such as a video adapter and / or an audio adapter. The output adapter is operatively coupled with the processor 305 and operatively coupled with an output device, such as a display device, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or an "electronic ink" display, or an audio output device, a speaker, or headphones.

[0043] In some embodiments, the user system 302 includes an input device 320 for receiving input from the user 301. The input device 320 can include, for example, a keyboard, a pointing device, a mouse, a stylus, a touch-sensitive panel, a touchpad, a touchscreen, a gyroscope, an accelerometer, a position detector, or an audio input device. A single component, such as a touchscreen, can function simultaneously as an output device of the media output component 315 and as the input device 320. The user system 302 can also include a communication interface 325 that is operatively coupled with a remote device, such as the II computing device 104. The communication interface 325 can include, for example, a wired or wireless network adapter or a wireless data transceiver for use with a mobile telephone network, a Global System for Mobile Communications (GSM), 3G, or other mobile data network, or Worldwide Interoperability for Microwave Access (WIMAX).

[0044] Stored in the memory area 310 are, for example, computer readable instructions for providing a user interface to the user 301 via the media output component 315 and, optionally, receiving and processing input from the input device 320. The user interface can include a web browser and a client application, among other possibilities. The web browser enables a user, such as the user 301, to display and interact with media and other information typically embedded in a web page or website often from the II computing system 100.

[0045] Figure 4 An example configuration of the server system 401 is illustrated. The server system 401 can include, but is not limited to, the II computing device 104 (as shown in FIG. 1).Figure 1 The server system 401 includes a processor 405 for executing instructions. For example, the instructions can be stored in a memory area 410. The processor 405 can include one or more processing units (e.g., in a multi-core configuration) for executing the instructions. The instructions can be executed within various different operating systems on the server system 401, such as UNIX, LINUX, Microsoft Windows®, and the like. It should also be understood that, upon startup of a computer-based method, various instructions can be executed during initialization. Some operations can be needed for performing one or more processes described herein, while other operations can be more general and / or specific to a particular programming language (e.g., C, C#, C++, Java, or other suitable programming language, etc.).

[0046] The processor 405 is operatively coupled with a communication interface 415, enabling the server system 401 to communicate with remote devices (such as the user system 302 (shown), or another server system 401). For example, the communication interface 415 can receive requests from the user computing device 110 via the Internet. Figure 3

[0047] The processor 405 can also be operatively coupled with a storage device 425. The storage device 425 is any computer-operated hardware suitable for storing and / or retrieving data. In some embodiments, the storage device 425 is integrated in the server system 401. For example, the server system 401 can include one or more hard disk drives as the storage device 425. In other embodiments, the storage device 425 is external to the server system 401, and can be accessed by multiple server systems 401. For example, the storage device 425 can include multiple storage units such as hard disks or solid state disks in a redundant array of independent disks (RAID) configuration. The storage device 425 can include a storage area network (SAN) and / or a network attached storage (NAS) system.

[0048] In some embodiments, the processor 405 is operatively coupled with the storage device 425 via a storage interface 420. The storage interface 420 is any component enabling the processor 405 to access the storage device 425. The storage interface 420 can include, for example, an advanced technology attached (ATA) adapter, a serial ATA (SAT A) adapter, a small computer system interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and / or any component enabling the processor 405 to access the storage device 425.

[0049] ​The memory region 410 can include, but is not limited to, a random access memory (RAM) such as dynamic RAM (DRAM) or static RAM (SRAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and non-volatile RAM (NVRAM). The above memory types are exemplary only, and are thus not limiting as to the type of memory used to store a computer program.

[0050] Figure 5 is a flowchart of an example method 500 for automatically detecting defects in accordance with the present disclosure.

[0051] In example embodiments, the method 500 includes receiving 502 at least one sample image of the first component 202, wherein the at least one sample image of the first component 202 does not include a defect, and storing 504 the at least one sample image 202 in a memory (e.g., the database 108). The method 500 further includes receiving 506 an input image of the second component 206 / 252, generating 508 an encoded array based on the input image of the second component 206 / 252, performing 510 a random data sampling process on the encoded array, and generating 512 a decoded array from the random data sampling.

[0052] In some embodiments, generating 508 the encoded array based on the input image can be performed by an encoder component included in the II computing device 104 by using a lower dimensional image than the input image (e.g., using a two-dimensional representation of an input three-dimensional image). A random data sampling process is then introduced to modify the encoded array such that the encoded array is properly formatted for machine learning techniques described below (e.g., for backpropagation). After performing the random data sampling process, a latent variable can be estimated which is used by a decoding component included in the II computing device 104 in order to generate 512 the decoded array. Thus, at least in part because of the random data sampling process described above, the systems and methods described herein can include active machine learning techniques (e.g., as described below) rather than known systems and methods that require memorization of defect data from defect sample images.

[0053] The method 500 also includes generating 514 a reconstructed image 254 of the second component derived from the random data sampling and decoding array, the reconstructed image 254 generated based at least in part on the at least one sample image 202, and comparing 516 the reconstructed image 254 of the second component to the input image of the second component 206 / 252. The method 500 further includes producing 518 a residual image 256 including at least one difference between the reconstructed image 254 of the second component and the input image of the second component 206 / 252 if the second component is defective, and identifying 520 whether the second component is defective based on the residual image 256.

[0054] In some embodiments of the method 500, comparing 516 the reconstructed image 254 of the second component to the input image 206 / 252 of the second component further includes comparing pixels of the reconstructed image 254 of the second component to pixels of the input image 206 / 252 of the second component. In these embodiments, the method 500 further includes calculating a similarity index representing an estimated similarity between the pixels of the reconstructed image 254 and the pixels of the input image 206 / 252 based on comparing the pixels of the reconstructed image 254 and the pixels of the input image 206 / 252, and generating a quality index including a quality score of at least one pixel in the residual image 256 from the similarity index. Further in the method 500, identifying 520 whether the second component is defective can include identifying at least one pixel of the residual image 256 having a quality score below a threshold value, and determining the threshold value by comparing a quality score of at least one reference pixel of the at least one sample image 202 to a quality score of at least one corresponding reference pixel in the image having at least one defect. In these embodiments, the method 500 can also include determining the threshold value such that a quality score of a pixel associated with a relatively minor defect will be greater than the threshold value. In some embodiments, the method 500 includes applying a trained model to the residual image to determine whether the second component is defective.

[0055] Figure 6 is a computer program product that can be used in Figure 1 FIG. 600 is a diagram of components of one or more example computing devices that can be used in the II computing system 100 shown. In some embodiments, the computing device 610 can be used to implement the II computing device 104. The computer components can be used, for example, to identify 520 whether the second component is defective (as shown in Figure 5 An operator 602 (e.g., a user operating the II computing device 104) can access the computing device 610 in order to initiate various operations and / or service the computing device 610. In some embodiments, the database 620 is similar to the database 108, as shown in Figure 1The database 620 can be implemented on one or more data storage devices 630 and coupled to several components within the computing device 610 that perform specific tasks. In example embodiments, the database 620 includes residual image data 622 and normal data sampling 624 of images.

[0056] In example embodiments, the computing device 610 includes a communication component 640 programmed to, for example, receive 502 at least one image of the first component 202, receive 506 the input image of the second component 206, and transmit the residual image 210 (as shown, for example). Figure 2A and 5 as shown).

[0057] In example embodiments, the computing device 610 further includes an analysis component 650 to, for example, discern 520 whether the second component is defective based on the residual image 256.

[0058] Example embodiments of systems and methods for discerning component defects are described in detail above. The systems and methods are not limited to the specific embodiments described herein but, rather, the components of the systems and / or the operation of the methods can be utilized independently and separately from other components and / or methods described herein. Further, the described components and / or operations can also be defined in, or used in combination with, other systems, methods, and / or devices, and are not limited to practice with only the systems as described herein.

[0059] The processor or processing element can employ artificial intelligence and / or be trained using supervised or unsupervised machine learning, and the machine learning program can employ a neural network, which can be a convolutional neural network, a deep learning neural network, or a comprehensive learning module or program that learns in two or more domains or fields of interest. Machine learning can involve discerning and recognizing patterns in existing data in order to make predictions on subsequent data. Models can be created based on example inputs in order to make valid and reliable predictions on new inputs.

[0060] Additionally or alternatively, the machine learning program can be trained by inputting a sample data set or certain data into the program, such as image data, text data, report data, and / or numerical analysis. The machine learning program can utilize deep learning algorithms, which can primarily focus on pattern recognition and can be trained after processing multiple examples. The machine learning program can include Bayesian program learning (BPL), speech recognition and synthesis, image or object recognition, optical character recognition, and / or natural language processing— which can be used individually or in combination. The machine learning program can also include natural language processing, semantic analysis, automated reasoning, and / or machine learning.

[0061] In supervised machine learning, a processing element can be provided with example inputs and their associated outputs, and can seek to discover general rules that map inputs to outputs, so that when a new input is subsequently provided, the processing element can accurately predict the correct output based on the discovered rules. In unsupervised machine learning, a processing element can be tasked with finding structure in unlabeled example inputs on its own. In one embodiment, machine learning techniques can be used to extract data related to a computer device, a user of a computer device, a computer network hosting a computer device, a service executing on a computer device, and / or other data.

[0062] Based on these analyses, the processing element can learn how to recognize features and patterns that can then be applied to a trained model, as an example, a model that determines whether a component is defective or not.

[0063] As can be appreciated from the foregoing description, embodiments of the foregoing disclosure can be implemented using computer programming or engineering techniques, including computer software, firmware, hardware, or any combination or subset thereof. Any such resulting program(s), having computer readable code means, can be embedded in or provided in connection with one or more computer readable media, thereby making a computer program product or products, i.e., a manufacturing product. The computer readable media can be, for example but not limited to, fixed (hard) drives, diskettes, optical disks, magnetic tape, semiconductor memory (e.g., Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM)) and / or any transmission or data storage medium such as the Internet or other communication network or link. The resulting program(s) can be implemented in a high level procedural or object oriented programming and / or scripting languages, tool software, firmware, operating system or combination thereof. Computer program or programs can also be embedded in a computer program product, which is provided as a manufacturing product, e.g., a computer readable storage medium having tangibly embodied computer readable code thereon, which is an article of manufacture including programs fixed in computer readable medium. As such, the manufacturing product (computer program product) can be made and / or used, for example, by producing (manufacturing) a computer readable storage medium to contain a set of instructions of the computer program, and / or by transferring and / or streaming the computer program from one computer readable medium to another, such that the computer program can be executed by one or more computer processors.

[0064] These computer programs (also known as programs, software, software applications, “apps”, or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming and / or scripting languages, and / or tools software, firmware, operating system or combination thereof. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. However, a “machine-readable medium” and “computer-readable medium” do not include transitory signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0065] As used herein, the term "database" can refer to a data body, a relational database management system (RDBMS), or both. A database can include any collection of data including hierarchical databases, relational databases, flat file databases, object- relational databases, object databases, and any other structured collections of records or data stored in a computer system. The above examples are merely illustrative, and thus are not intended to limit or restrict the definition and / or meaning of the term database in any way. Examples of RDBMSs include, but are not limited to including Oracle® databases, MySQL, IBM® DB2, Microsoft® SQL Server, Sybase®, and PostgreSQL. However, any database implementation (e.g., relational, file-based) can be used to implement the systems and methods described herein. (Oracle is a registered trademark of Oracle Corporation, Redwood Shores, California; IBM is a registered trademark of International Business Machines Corporation, Armonk, New York; Microsoft is a registered trademark of Microsoft Corporation, Redmond, Washington; Sybase is a registered trademark of Sybase, Dublin, California).

[0066] As used herein, a processor can include any programmable system including systems using microcontrollers, reduced instruction set circuits (RISC), application specific integrated circuits (ASICs), logic circuits, and any other circuit or processor capable of executing the functions described herein. The above examples are merely illustrative, and thus are not intended to limit or restrict the definition and / or meaning of the term processor in any way.

[0067] As used herein, the terms "software" and "firmware" are interchangeable, and include any computer program stored in memory for execution by a processor, including RAM memory, ROM memory, EPROM memory, EEPROM memory, and nonvolatile RAM (NVRAM) memory. The above memory types are exemplary only, and are thus not limiting as to the types of memory usable for storage of a computer program.

[0068] In some embodiments, the system includes multiple components distributed among multiple computer devices. One or more components can be in the form of computer executable instructions embedded in a computer readable medium. These systems and processes are not limited to the specific embodiments described herein. Further, components of each system and each process can be practiced independent and separate from other components and processes described herein. Each component and process can also be used in combination with other components and processes described herein. The present embodiments are therefore not to be limited to the specific embodiments described herein.

[0069] As used herein, an element or step recited in the singular and preceded with "a" or "an" should be understood as not excluding plural elements or steps, unless explicitly stated otherwise. Furthermore, references to "example embodiments," "example implementations," or "one implementation" of the present disclosure, are not intended to be interpreted as excluding the existence of additional embodiments or implementations that also incorporate those features.

[0070] The patent claims at the end of this document are not intended to be interpreted under 35 U.S.C. § 112(f) unless traditional means-plus-function language is explicitly recited in the claim(s). Such shall only apply to claims expectations to be interpreted under 35 U.S.C. § 112(f) will specifically be recited as such.

[0071] Although specific features of various embodiments of the present application can be shown in some drawings and not others, this is for convenience only as each feature can be combined with any or all of the other features of any of the drawings in accordance with the principles of the present application.

[0072] Further directions of the present application are provided by the subject matter of the following clauses:

[0073] 1. A method of inspecting a part using an image inspection controller, the image inspection controller comprising a processor communicatively coupled with a memory, the method comprising: receiving, at the processor, at least one sample image of a first part, wherein the at least one sample image of the first part does not include a defect; receiving, at the processor, an input image of a second part; generating, by the processor, an encoding array based on the input image of the second part; performing, by the processor, a random data sampling process on the encoding array; generating, by the processor, a decoding array from the random data sampling process based at least in part on the at least one sample image; generating, by the processor, a reconstructed image of the second part derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image; and comparing, by the processor, the reconstructed image of the second part to the input image of the second part to produce a residual image that includes at least one difference between the reconstructed image of the second part and the input image of the second part if the second part has a defect.

[0074] 2. The method of any preceding clause, wherein comparing the reconstructed image of the second part to the input image of the second part further comprises comparing, by the processor, pixels of the reconstructed image of the second part to pixels of the input image of the second part.

[0075] 3. The method of any preceding clause, further comprising calculating a similarity index based on comparing the pixels of the reconstructed image and the pixels of the input image, the similarity index representing an estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

[0076] 4. The method of any preceding clause, further comprising generating a quality index from the similarity index, the quality index comprising a quality score for at least one pixel in the residual image.

[0077] 5. The method of any preceding clause, wherein identifying a defect of the second component further comprises identifying at least one pixel of the residual image having a quality score below a threshold value.

[0078] 6. The method of any preceding clause, further comprising determining the threshold value by comparing a quality score of at least one reference pixel of the at least one sample image to a quality score of at least one corresponding reference pixel in an image having at least one defect.

[0079] 7. The method of any preceding clause, further comprising applying a trained model to the residual image to determine whether the second component has a defect.

[0080] 8. An image inspection computing device comprising: a memory device; and at least one processor communicatively coupled with the memory device, wherein the at least one processor is configured to: receive at least one sample image of a first component, wherein the at least one sample image of the first component does not include a defect; receive an input image of a second component; generate an encoding array based on the input image of the second component; perform a random data sampling process on the encoding array; generate a decoding array from the random data sampling process based at least in part on the at least one sample image; generate a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image; and compare the reconstructed image of the second component to the input image of the second component to produce a residual image, the residual image including at least one difference between the reconstructed image of the second component and the input image of the second component if the second component has a defect.

[0081] 9. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to, when comparing the reconstructed image of the second component to the input image of the second component, compare pixels of the reconstructed image of the second component to pixels of the input image of the second component.

[0082] 10. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to compute a similarity index based on comparing the pixels of the reconstructed image and the pixels of the input image, the similarity index representing an estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

[0083] 11. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to generate a quality index from the similarity index, the quality index comprising a quality score for at least one pixel in the residual image.

[0084] 12. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to identify, in discriminating the defect of the second component, at least one pixel of the residual image having a quality score below a threshold value.

[0085] 13. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to determine the threshold value by comparing a quality score of at least one reference pixel of the at least one sample image to a quality score of at least one corresponding reference pixel in an image having at least one defect.

[0086] 14. The image inspection computing device of any preceding clause, wherein the at least one processor is further configured to determine the threshold value such that a quality score of a pixel associated with a relatively minor defect will be greater than the threshold value.

[0087] 15. A non-transitory computer-readable storage medium having computer- executable instructions embedded therein, wherein, when executed by a computing device comprising at least one processor in communication with a memory, the computer-executable instructions cause the computing device to: receive at least one sample image of a first component, wherein the at least one sample image of the first component does not include a defect; receive an input image of a second component; generate an encoding array based on the input image of the second component; perform a random data sampling process on the encoding array; generate a decoding array from the random data sampling process based at least in part on the at least one sample image; generate a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image generated based at least in part on the at least one sample image; and compare the reconstructed image of the second component to the input image of the second component to produce a residual image that includes at least one difference between the reconstructed image of the second component and the input image of the second component if the second component has a defect.

[0088] 16. The non-transitory computer-readable storage medium of any preceding clause, wherein the computer-executable instructions further cause the computing device, in comparing the reconstructed image of the second component to the input image of the second component, to compare pixels of the reconstructed image of the second component to pixels of the input image of the second component.

[0089] 17. The non-transitory computer-readable storage medium of any preceding clause, wherein the computer-executable instructions further cause the computing device, based on comparing the pixels of the reconstructed image and the pixels of the input image, to compute a similarity index that represents an estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

[0090] 18. The non-transitory computer-readable storage medium of any preceding clause, wherein the computer-executable instructions further cause the computing device to generate a quality index from the similarity index, the quality index including a quality score for at least one pixel in the residual image.

[0091] 19. The non-transitory computer-readable storage medium of any preceding clause, wherein the computer-executable instructions further cause the computing device, in identifying a defect of the second component, to identify at least one pixel of the residual image having a quality score below a threshold value.

[0092] 20. The non-transitory computer-readable storage medium of any preceding clause, wherein the computer-executable instructions further cause the computing device to determine the threshold value by comparing a quality score of at least one reference pixel of the at least one sample image to a quality score exponential value of at least one corresponding reference pixel in the image having at least one defect.

[0093] This written description uses examples to disclose the disclosure, including the best mode, and also to enable any person skilled in the art to practice the disclosure, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the disclosure is defined by the claims, and can include other examples that occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent

Claims

1. A method for inspecting components using an image inspection controller, the image inspection controller including a processor communicatively coupled to a memory, characterized in that, The method includes: At the processor, at least one sample image of a first component is received, wherein the at least one sample image of the first component does not include defects; At the processor, the input image from the second component is received; The processor generates an coded array using an image of a lower dimension than the input image of the second component, based on the input image of the second component; The processor performs random data sampling on the encoding array to modify the encoding array; The processor generates a decoding array from the random data sampling process, based at least in part on the at least one sample image. The processor generates a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image being generated at least in part based on the at least one sample image; and The processor compares the reconstructed image of the second component with the input image of the second component to generate a residual image. If the second component has a defect, the residual image includes at least one difference between the reconstructed image of the second component and the input image of the second component.

2. The method according to claim 1, characterized in that, The comparison of the reconstructed image of the second component with the input image of the second component further includes the processor comparing the pixels of the reconstructed image of the second component with the pixels of the input image of the second component.

3. The method according to claim 2, characterized in that, The method further includes calculating a similarity index based on comparing the pixels of the reconstructed image with the pixels of the input image, the similarity index representing the estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

4. The method according to claim 3, characterized in that, It further includes generating a quality index from the similarity index, the quality index including the quality score of at least one pixel in the residual image.

5. The method according to claim 4, characterized in that, The identification of defects in the second component further includes identifying at least one pixel in the residual image whose quality score is below a threshold.

6. The method according to claim 5, characterized in that, The threshold is further determined by comparing the quality score of at least one reference pixel in the at least one sample image with the quality score of at least one corresponding reference pixel in an image having at least one defect.

7. The method according to claim 1, characterized in that, The method further includes applying the trained model to the residual image to determine whether the second component is defective.

8. An image inspection computing device, characterized in that, include: Memory devices; as well as At least one processor, the at least one processor being communicatively connected to the memory device, wherein the at least one processor is configured to: Receive at least one sample image of a first component, wherein the at least one sample image of the first component does not include defects; Receive the input image from the second component; Based on the input image of the second component, an image of a lower dimension than the input image of the second component is used to generate a coding array; Random data sampling is performed on the encoded array to modify it; A decoding array is generated from the random data sampling process, based at least in part on the at least one sample image; Generate a reconstructed image of the second component derived from the random data sampling process and the decoding array, the reconstructed image being generated at least in part based on the at least one sample image; and The reconstructed image of the second component is compared with the input image of the second component to produce a residual image. If the second component has a defect, the residual image includes at least one difference between the reconstructed image of the second component and the input image of the second component.

9. The image inspection computing device according to claim 8, characterized in that, The at least one processor is further configured to compare the pixels of the reconstructed image of the second component with the pixels of the input image of the second component when comparing the reconstructed image of the second component with the input image of the second component.

10. The image inspection computing device according to claim 9, characterized in that, The at least one processor is further configured to calculate a similarity index based on a comparison of the pixels of the reconstructed image and the pixels of the input image, the similarity index representing the estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

11. The image inspection computing device according to claim 10, characterized in that, The at least one processor is further configured to generate a quality index from the similarity index, the quality index including the quality score of at least one pixel in the residual image.

12. The image inspection computing device according to claim 11, characterized in that, The at least one processor is further configured to, when identifying defects in the second component, identify at least one pixel of the residual image whose quality score is below a threshold.

13. The image inspection computing device according to claim 12, characterized in that, The at least one processor is further configured to determine the threshold by comparing the quality score of at least one reference pixel in the at least one sample image with the quality score of at least one corresponding reference pixel in an image having at least one defect.

14. The image inspection computing device according to claim 12, characterized in that, The at least one processor is further configured to determine the threshold such that the quality score of a pixel associated with a relatively minor defect will be greater than the threshold.

15. A non-transitory computer-readable storage medium, wherein computer-executable instructions are embedded thereon, characterized in that, The computer-executable instructions, when executed by a computing device including at least one processor communicating with memory, cause the computing device to: Receive at least one sample image of a first component, wherein the at least one sample image of the first component does not include defects; Receive the input image from the second component; Based on the input image of the second component, an image of a lower dimension than the input image of the second component is used to generate a coding array; Random data sampling is performed on the encoded array to modify it; A decoding array is generated from the random data sampling process, based at least in part on the at least one sample image; A reconstructed image of the second component, derived from the random data sampling process and the decoding array, is generated based at least in part on the at least one sample image; as well as The reconstructed image of the second component is compared with the input image of the second component to produce a residual image. If the second component has a defect, the residual image includes at least one difference between the reconstructed image of the second component and the input image of the second component.

16. The non-transitory computer-readable storage medium according to claim 15, characterized in that, The computer-executable instructions further cause the computing device to compare the pixels of the reconstructed image of the second component with the pixels of the input image of the second component when comparing the reconstructed image of the second component with the input image of the second component.

17. The non-transitory computer-readable storage medium according to claim 16, characterized in that, The computer-executable instructions further cause the computing device to calculate a similarity index based on a comparison of the pixels of the reconstructed image and the pixels of the input image, the similarity index representing an estimated similarity between the pixels of the reconstructed image and the pixels of the input image.

18. The non-transitory computer-readable storage medium according to claim 17, characterized in that, The computer-executable instructions further cause the computing device to generate a quality index from the similarity index, the quality index including the quality score of at least one pixel in the residual image.

19. The non-transitory computer-readable storage medium according to claim 18, characterized in that, The computer-executable instructions further cause the computing device, when identifying defects in the second component, to identify at least one pixel in the residual image whose quality score is below a threshold.

20. The non-transitory computer-readable storage medium according to claim 19, characterized in that, The computer-executable instructions further enable the computing device to determine the threshold by comparing the quality score of at least one reference pixel in the at least one sample image with the quality score index value of at least one corresponding reference pixel in an image having at least one defect.

Images