Method and system for anomoly detection

Abstract

Provided is a method and system for detecting anomalies to determine if a target part is conforming. The method includes applying a keeper model to an inspection image of the target part to obtain a keeper output image. The keeper model retains non-anomalous areas of the inspection image and identified anomalies of the inspection image in the keeper output image. The method further includes applying a remover model to the inspection image to obtain a remover output image. The remover model retains non-anomalous areas and removes anomalies from the inspection image in the remover output image. The method further includes comparing the keeper output image and the remover output image to detect anomalies based on features in the keeper output image that are not present in the remover output image. The keeper model and the remover model are trained models.

Description

METHOD AND SYSTEM FOR ANOMOLY DETECTIONTechnical Field

[0001] The following relates generally to anomaly detection and particularly a method and system for detecting anomalies to determine if a good is quality conforming.Introduction

[0002] Anomaly detection is used to determine if a goods or products are quality conforming, also referred to as “OK” or good. Detecting anomalies enables comparison of the part against anomalies of parts previously identified as good (G) or not good (NG) to determine if the inspected good is able to pass as OK.

[0003] In some existing systems machine vision is used to inspect an inspection image of, for example, a target part and detect anomalies. In some existing systems all known defect classes and unknown anomalies are detected. These defect classes and anomalies are used to determine if a part is good (G) or not good (NG). Detecting anomalies so broadly can be data and processing power intensive.

[0004] Some existing systems employ methods known as generative methods to determine if a part is G or NG. Generative methods typically compare detected anomalies of the target part against anomalies of one or more samples predetermined to be good. Where an anomaly is determined to correspond to a known good anomaly (i.e. a preaccepted anomaly) the anomaly is passed over, otherwise, the anomaly is determined to be NG. There are typically no tolerances for acceptable variation from the predetermined good anomalies. Therefore, these methods will determine target parts NG if any anomalies of the target part are previously unknown even if that anomaly is acceptable (i.e. a false positive). This determination of NG based on previously unknown anomalies even if these anomalies would be acceptable if known can be costly and wasteful. Furthermore, comprehensive predetermining of acceptable anomalies, without including unacceptable anomalies or combinations of anomalies can be difficult if not impossible. Therefore, some waste is usually accepted. Some existing systems employ machine learning techniques to generate acceptable anomaly data and mitigate this waste. However, data required to define such anomalies comprehensively is often so extensivethat the comparison with the target sample is slow and the processing burden is high. Depending on the application these burdens may be prohibitive leading to quality control avoidance or batch testing at larger intervals. This can lead to nonconforming parts slipping by inspection and remaining in production; an issue both in quality and safety.

[0005] Some existing systems employ methods known as discriminative methods to determine if a part is G or NG. Discriminative methods typically compare detected anomalies of the target part against anomalies of one or more samples predetermined to be NG. Where an anomaly of the target part is found to correspond with a NG anomaly the part may be determined to be NG. In some existing systems, a certain amount or quality of NG anomalies must be found to warrant a NG determination of the target part. As anomalies not corresponding to anomalies of the NG data set will pass inspection, this method requires any potential NG anomaly to be predetermined to avoid passing target parts with unknown unacceptable anomalies. This is often difficult and unknown unacceptable anomalies may be inadvertently excluded from the NG set. As above this can lead to the production of NG parts which may be unsafe depending on the application. Mitigating these issues often requires a substantial volume of data classifying NG parts and anomalies. Aside from the data and processing issues described above, this requires the NG parts data to be available or generated. Generating or obtaining this NG parts data is typically costly, particularly if physical samples modified or intentionally produced to be NG are required to generate the NG anomalies data set. Generating these scrap NG parts is also wasteful.

[0006] In some systems, generative and discriminative methods may both be used to determine if a target part is G or NG. For example, a golden sample an illustrating acceptable including most common anomalies predetermined to be acceptable may be generated and subtracted from the target part to eliminate from consideration the most prevalent acceptable anomalies. The remaining anomalies may be identified and classified using discriminative methods to determine if NG anomalies exist. Applying both methods this way reduces the burden and need for robustness at the generative stage as this stage is used to reduce the load at the discriminative stage rather than make a determination. Applying both methods also typically reduces the quantity of anomalies processed at the discriminative stage. However, the specificity of the golden samplemitigates the extent it eliminates acceptable anomalies leaving a substantial amount of ultimately acceptable anomalies to be processed at the discriminative stage. Also, these systems still suffer from issues associated with generating and applying the NG anomalies data set described above.

[0007] Regardless of the method used, the variation between a G and NG anomaly can be obscured or vary from predetermined discriminations due to low contrast at the anomaly, small (i.e. sub-millimeter) sizes of anomalies, and texture variations of the parts. Therefore, predetermined parameters defining G or NG anomalies may require narrowing in the case of G anomalies and overbreadth in the case of NG anomalies to be conservative in avoiding false G determinations.

[0008] Accordingly, systems and methods for detecting anomalies are desired which overcome issues in existing anomaly detection system and methods.Summary

[0009] Provided is a method for detecting anomalies to determine if a target part is conforming. The method includes applying a keeper model to an inspection image of the target part to obtain a keeper output image and applying a remover model to the inspection image to obtain a remover output image. The keeper model is configured to retain non-anomalous areas of the inspection image and identified anomalies of the inspection image in the keeper output image. The remover model configured to retain non-anomalous areas and remove anomalies from the inspection image in the remover output image. The method includes comparing the keeper output image and the remover output image to detect anomalies based on features in the keeper output image that are not present in the remover output image. The keeper model and the remover model are models trained to keep and remove anomalies respectively.

[0010] One or more of the remover model and the keeper model may be an autoencoder model.

[0011] The autoencoder model may include skip connections for bypassing hidden layers of the autoencoder model.

[0012] One or more of the remover model and the keeper model may be trained by augmenting an okay (OK) inspection image with simulated anomaly data to obtain an not okay (NOK) inspection image and adjusting post processing parameters based on a comparison of the simulated anomaly data with detected anomalies and corresponding anomaly data obtained by the applying the method to the obtained NOK inspection image.

[0013] The method may further include segmenting the inspection image. The segmentation may include segmenting the image into a plurality of areas and masking areas determined not of interest.

[0014] The method may further include generating an anomalous area heatmap based on the comparison of the keeper output image and the remover output image.

[0015] The method may further include conducting a shape analysis and binarization on the results of the comparison of the keeper output image and the remover output image to generate a shape analysis and binarization map or obtain anomaly data corresponding to each detected anomaly.

[0016] The method may further include plotting the detected anomalies to obtain a plotted anomalies image. The plotted anomalies image may include the inspection image augmented with an indication of each anomaly.

[0017] Plotting the detected anomalies may further include plotting anomaly data corresponding to each detected anomaly.

[0018] The keeper model and the remover model may retain non-anomalous areas based on a golden sample.

[0019] In another aspect provided is a system for detecting anomalies to determine if a target part is conforming. The system includes a keeper module configured to apply a keeper model to an inspection image of the target part to obtain a keeper output image and a remover module configured to apply a remover model to the inspection image to obtain a remover output image. The keeper model configured to retain non-anomalous areas of the inspection image and identified anomalies of the inspection image in the keeper output image. The remover model configured to retain non-anomalous areas and remove anomalies from the inspection image in the remover output image. The systemincludes a comparison module configured to compare the keeper output image and the remover output image to detect anomalies based on features in the keeper output image that are not present in the remover output image. The keeper model and the remover model are models trained to keep and remove anomalies, respectively.

[0020] One or more of the remover model and the keeper model may be an autoencoder model.

[0021] The autoencoder model may further include skip connections for bypassing hidden layers of the autoencoder model.

[0022] One or more of the remover model and the keeper model may be trained by augmenting an okay (OK) inspection image with simulated anomaly data to obtain an not okay (NOK) inspection image and adjusting post processing parameters based on a comparison of the simulated anomaly data with detected anomalies and corresponding anomaly data obtained by the applying the method to the obtained NOK inspection image.

[0023] The system may further include a segmentation module configured to segment the inspection image, the segmentation comprising segmenting the image into a plurality of areas and masking areas determined not of interest.

[0024] The comparison module may be configured to generate an anomalous area heatmap based on the comparison of the keeper output image and the remover output image.

[0025] The system may include a shape analysis and binarization module configured to conduct a shape analysis and binarization on the results of the comparison of the keeper output image and the remover output image to generate a shape analysis and binarization map or obtain anomaly data corresponding to each detected anomaly.

[0026] The system may include an an anomaly plotting module configured to plot the detected anomalies to obtain a plotted anomalies image. The plotted anomalies image may include the inspection image augmented with an indication of each anomaly.

[0027] Plotting the detected anomalies may include plotting anomaly data corresponding to each detected anomaly.

[0028] The keeper model and the remover model may retain non-anomalous areas based on a golden sample.

[0029] Other aspects and features will become apparent to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0030] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0031] Figure 1 is a flow diagram of an anomaly detection method, according to an embodiment;

[0032] Figure 2 is a flow diagram depicting example image inputs and outputs of the anomaly detection method of Figure 1 , according to an embodiment;

[0033] Figure 3 is a block diagram of an anomaly detection system for implementing the method of Figure 1 , according to an embodiment;

[0034] Figure 4A block diagram illustrating an autoencoder network architecture of the keeper model of Figure 3, according to an embodiment;

[0035] Figure 4B is a block diagram illustrating hidden layers and skip connections of the autoencoder network architecture of Figure 4A, according to an embodiment;

[0036] Figure 5A is an inspection image of Figure 2, according to an embodiment;

[0037] Figure 5B is an image depicting the keeper model output of Figure 2, according to an embodiment;

[0038] Figure 5C is an image depicting the remover model output of Figure 2, according to an embodiment;

[0039] Figure 6A is an inspection image of Figures 1 and 3, according to an embodiment;

[0040] Figure 6B is an image depicting the keeper model output of Figures 1 and 3, according to an embodiment;

[0041] Figure 6C is an image depicting the remover model output of Figures 1 and 3, according to an embodiment;

[0042] Figure 7A is an image depicting the anomalous area heatmap of Figure 2, according to an embodiment;

[0043] Figure 7B is an image depicting an area including an anomaly of the anomalous area heatmap of Figure 7A; according to an embodiment;

[0044] Figure 8A is an image depicting the Shape Analysis & Binarization Map of Figure 2, according to an embodiment;

[0045] Figure 8B is an image depicting an area including an anomaly of the Shape Analysis & Binarization Map of Figure 8A; according to an embodiment;

[0046] Figure 9A is a plotted anomalies image of Figure 2, according to an embodiment;

[0047] Figure 9B is an image depicting an area including an anomaly of the plotted anomalies image of Figure 9A, according to an embodiment;

[0048] Figure 10 is a flow chart of a method of training the keeper and remover models of Figures 1 and 3, according to an embodiment;

[0049] Figure 11 is a flow chart depicting example inputs and outputs of the image segmentation of Figure 11 , according to an embodiment;

[0050] Figure 12 is a flow chart depicting example inputs and outputs of the data augmentation of Figure 10, according to an embodiment;

[0051] Figure 13A is a flow chart depicting example inputs and outputs of the keeper network training of Figure 10, according to an embodiment;

[0052] Figure 13B is a flow chart depicting example inputs and outputs of the masked data synchronization of Figure 10, according to an embodiment;

[0053] Figure 14A is a flow chart depicting example inputs and outputs of the remover network training of Figure 10 with a not okay (NOK) sample, according to an embodiment;

[0054] Figure 14B is a flow chart depicting example inputs and outputs of the remover network training of Figure 10 with an okay (OK) sample, according to an embodiment;

[0055] Figure 15 is a flow chart depicting example inputs and outputs of the detection and plotting anomalies of Figure 1 , according to an embodiment; and

[0056] Figure 16 is a compilation of various plotted anomalies images of Figure 1 , according to an embodiment.Detailed Description

[0057] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0058] A description of an embodiment with several components in contact or connection with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0059] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0060] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device / article. Similarly, where more than one device or article is describedherein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0061] The present disclosure provides a method and system for detecting anomalies.

[0062] Referring to Figures 1 and 2, shown therein are flow diagrams of a method 100 for identifying anomalies including blocks and example depictions respectively, of inputs and outputs in block, according to an embodiment. The method 100 may be implemented by the work hardening apparatus 300 of Figure 3 described below.

[0063] The anomaly detection method 100 is directed to inputting an inspection image 102 of a target part into a keeper model and a remover model and comparing the outputs to detect and plot anomalies. Comparing the outputs of the keeper model and remover model, as further described below, enables the model to be trained based on images with higher complexity over existing systems, is more efficient over existing systems due to remover and keeper model synergy, accommodates normal variations over existing systems, achieves faster and more linear processing times over existing systems, and is not dependent on defective samples for training.

[0064] At 102, in some embodiments the anomaly detection 100 includes segmenting an inspection image 102, to obtain a masked image 106. Segmentation, at 102, may include segmenting the model into multiple areas. The segmentation may be based on areas of the part recognized to be distinct from other areas of the part. The distinctions may be identified via image recognition, for example based on features of the part such as a segment indicated by a change in shading.

[0065] Segmentation, at 102, may also include masking. Masking removes areas of the image from consideration. Masking may be to exclude for example areas depicting elements of the parts where anomalies are acceptable or areas of the inspection image attributable to factors other than the part being inspected, such as areas depicting an element of stand or holding apparatus that secured the target part when the inspection image was obtained.

[0066] At 108, the anomaly detection 100 includes processing masked image 106 through a keeper model. The keeper model may be the keeper model 309 of Figure 3 further described below. Processing the masked image 106 through the keeper model is to obtain a keeper output image 110. The processing through the keeper model retains in the keeper out image 110 areas matching a sample predetermined as acceptable (i.e. a golden sample), and anomalies of the masked image 106. The retention is based on training of the keeper model such as the training at 1006 of Figure 10 described below. Anomalies are captured and represented explicitly without significant loss of detail.

[0067] At 112, the anomaly detection 100 includes processing the masked image 106 through a remover model. The remover model may be the remover model 313 of Figure 3, further described below. The processing of the masked image 106 through the remover model is to obtain a remover output image 114. In the remover output image 114 anomalies are removed from the masked image 106. Anomalies are removed based on training of the remover model such as the training 1010 of Figure 10, further described below. Based on the training of the remover model, the remover output image 114 represents a “clean” image, free from anomalies. The removal, at 112, may include removing anomalies, by compressing and removing anomalous areas from the masked image 106.

[0068] At 116, the anomaly detection 100 includes comparing the keeper output image 110 to the remover output image 114. The comparison, at 116, includes highlighting areas where the keeper output image 110 differs from the remover output image 114. These highlighted areas may be represented in an anomalous area heatmap 118. “Hot” areas indicate areas where anomalies are prevalent.

[0069] At 120, in some embodiments, the anomaly detection 100 includes conducting a shape analysis and binarization on areas identified as including anomalies, at 116. The shape analysis and binarization detects and binarizes each anomaly to isolate each anomaly from other anomalies. Each anomaly may be assigned identification (ID) information such as a unique identifier or ID number. Each anomaly is further analyzed to obtain information about the anomaly such as shape. The anomalies may be displayedon in a shape analysis & binarization map 122 depicting the anomalies only (i.e. without non-anomalous aspects of the inspection image 102).

[0070] At 124, in some embodiments, the anomaly detection 100 includes plotting the anomalies are plotted. Plotting the anomalies, at 124, includes indicating the anomalies on the inspection image 102 to obtain a plotted anomalies image 126. The indication of an anomaly may be, for example, via a border around the anomaly. Plotting the anomaly may further include providing anomaly information obtained, at 120, in the plotted anomalies image 126. The anomaly information may include information about each anomaly such as size, shape, and the like. The anomaly information may also include the ID information assigned, at 120.

[0071] At 128, in some embodiments, the anomaly detection 100 includes adjusting post processing parameters. Adjusted post processing parameters include parameters such as thresholding and morphological parameters. The adjustment is based on a comparison of predetermined augmented data applied at 116 with the anomalies and corresponding data detected and analyzed at 120 and 124. As augmented data applied is predetermined, any and all anomalies detected should be attributable to the augmented data. Therefore, any discrepancies such as omitted or additional anomalies in whole or in part indicate error in the models for detecting anomalies. Adjusting post processing parameters based on this identified error further beneficially improves the accuracy of the method 100 to extract and detect anomalous areas only.

[0072] Referring to Figure 3, shown therein is a block diagram of an anomaly detection system 300, according to an embodiment. The anomaly detection system 200 may be used to implement the method 100 of Figure 1 .

[0073] The anomaly detection system 300 includes a processor 352. The processor 252 is configured to detect anomalies.

[0074] The anomaly detection system 300 includes memory 354. The memory 354 is communicatively connected to the processor 352. The memory 354 stores computerexecutable instructions which, when executed by the processor 352, cause the anomaly detection system 200 to perform the functions and provide the functionalities described herein, such as detecting anomalies. The memory 354 also stores data such as imagesand models used by the processor 352 to perform the functions and provide the functionalities described herein, such as detecting anomalies. The memory 354 may also store data which is output when the executable instructions are executed by the processor 352, such as a plotted anomalies image 326, further described below.

[0075] The anomaly detection system 300 further includes an input device 356. The input device 356 is configured to receive an input into the system 300. The input device may be an interface such as network connection or data port configured to receive or obtain inputs to the system 300 for example from an external system, memory, or network. The inputs may be provided by a user. The input device 356 may be a device such as a camera or sensor for obtaining input from, for example, a target part. The processor 352 may generate input data in response to the input received via the input device 356 and store the input data in memory 354. The input data may then be used by the processor 352 to perform the functions and provide the functionalities described herein, such as detecting anomalies.

[0076] The input device 356 is configured to obtain an inspection image 302. The inspection image 302 may be obtained, for example, by input of a user or generated from data collected via an image sensor such as a camera. The obtained inspection image 302 may be stored in memory 304.

[0077] The system 300 may include a display device 358. The display device 358 is configured to display an output generated by the processor 352. In some embodiments, the display device 358 displays a graphical user interface configured to receive input such that a user can interact with the system 300 and view outputs generated thereby.

[0078] The processor 302 includes a segmentation module 304. The segmentation module 304 is configured to segment the inspection image 302 into distinct areas. The segmentation module 204 may also be configured to mask areas of the inspection image. Masking an area excludes the area from anomaly detection. The segmentation module 304 is configured to generate a masked image 306 based on the segmentation and masking.

[0079] The processor 300 includes a remover module 312. The remover module 312 is configured to receive the masked image 306 from the segmentation module 304.The remover module 312 is configured to generate a remover output image 314 by applying a remover model 314 to the masked image 306. The remover model 313 may also be referred to as remover network 313 or remover architecture 313. The remover model 313 is configured to remove anomalies from the masked image 106. Based on the training of the remover model 313, the remover output image 314 represents a “clean” image, free from anomalies. The removal may include removing anomalies, by compressing and removing anomalous areas from the masked image 306. In some embodiments, the remover model 313 is a model with an autoencoder network architecture such as the remover model 400 further described below.

[0080] Referring to Figure 4A shown therein is a block diagram illustrating a remover model 400 with an autoencoder network architecture, according to an embodiment. The autoencoder network of the remover model 400 is an artificial neural network. The artificial neural network can learn (i.e. be trained) unsupervised. With the autoencoder network architecture, the remover model 400 learns to represent data in a compressed form and uses that learned representation to recreate a recreation of the original data. The recreation is close to the original data.

[0081] The remover model 400 includes an input layer 410. The input layer is configured to receive input inspection images such as the inspection image 302 of Figure 3.

[0082] The remover model 400 includes hidden layers 420. The hidden layers are configured to compress and then reconstruct the input data received at the input layer 410.

[0083] The hidden layers 420 include one or more encoder layers 430. The encoder layers 430 are configured to compress the input data. The compression may be referred to as contracting or convolution and pooling. It will be appreciated that while one encoder layer 430 is shown, the encoder may include multiple encoder layers 430.

[0084] The hidden layers 420 further include a code layer 440. The code layer 440 processes the compressed input data from the encoder layers 430 to provide the functionality of the remover model 400.

[0085] The hidden layers 420 include one or more decoder layers 450. The decoder layers 450 are configured to reconstruct the processed data from the code layer 440. The reconstruction may be referred to as expansion or transpose convolution. It will be appreciated that while one decoder layer 450 is shown, the decoder may include multiple decoder layers 450.

[0086] The remover model 400 includes an output layer 460. The output layer 460 outputs the reconstructed data received from the hidden layers 420.

[0087] Referring also to Figure 4B, shown therein is a block diagram illustrating the hidden layers 420 and skip connections 470 of the remover model 400 according to an embodiment. Each encoder layer 430a, 430b, and 430c corresponds to a decoder layer 450a, 450b, and 450c, respectively. Skip connections 470a, 470b, and 470c enable the bypass of one or more layers in the neural network. The skip connections 470a, 470b, and 470b directly connect the earlier layer to the corresponding later one. The output of the skip connection is added to the main path’s output. These skip connections 470a, 470b, and 470c mitigate vanishing gradient issues and degradation problems in deep networks of the keeper model 400. This enhances training and enables deeper networks beneficially improving model performance.

[0088] Each encoder layer 430 includes multiple encoder blocks 432. Similarly, each decoder layer 450 includes multiple decoder blocks 452. Each block 432, 452 are fundamental units of the keeper network 400. Each block augments the data. Each encoder layer 430 is responsible for down sampling the data. Each decoder layer 450 is responsible for up sampling the data.

[0089] Referring specifically to Figure 3, the processor 302 includes a keeper module 308. The keeper module 308 is configured to receive the masked image 306 from the segmentation module 304. The keeper module 308 is configured to generate a keep output image 310 by applying the keeper model 309 to the masked image 306. The keeper model 309 may also be referred to as keeper network 309 or keeper architecture 309. The keeper model 309 is configured to retain areas matching a sample predetermined as acceptable (i.e. a golden sample), and anomalies of the masked image 306. The keeper model 309 is configured to capture and represent anomalies explicitly.

[0090] In some embodiments, the keeper model 309 is a model with an autoencoder network architecture. Where the keeper model 309 is a model with an autoencoder network architecture, the keeper model 309 is configured similarly to the remover model 400 of Figure 4A. In some embodiments, the hidden layers of the keeper model 309 are configured similarly to the hidden layers 420 of Figure 4B and include skip connections such as the skip connections 470a, 470b, and 470c of Figure 4B.

[0091] The processor 302 further includes a comparison module 316. The comparison module 316 is configured to receive and compare the keeper output image 310 and the remover output image 314 to detect anomalies. In some embodiments, the comparison module 316 is configured to generate and anomalous areas heatmap 318, such as the anomalous area heatmap 120 further described in Figure 7A below.

[0092] In some embodiments, the processor 302 includes a shape analysis and binarization module 320. The shape analysis and binarization module 320 is configured to receive the output of the comparison module 316 and binarize and analyze detected anomalies. The shape analysis and binarization module 320 may generate a shape analysis & binarization map 322, further described in Figure 8A below.

[0093] In some embodiments, the processor 302 includes an anomaly plotting module 324. The anomaly plotting module 324 is configured to generate a plotted anomalies image 326, further described in Figure 9A below.

[0094] The processor 302 includes a data augmentation module 364. The data augmentation module 364 is configured to apply simulated anomaly data 362 to an inspection image 302 and a masked image to obtain non-masked NOK Image 366 and a masked NOK Image 368. Images referred to herein as OK are images indicating quality conforming parts and images referred to as NOK are images indicating quality nonconforming parts. The data augmentation module 364 is also configured to pass the inspection image 302 through without augmentation as a non-masked OK image 367. It will be appreciated that the inspection image 302 provided to the data augmentation module is predetermined to be OK. The NOK images 366 and 368 serve as training images for the keeper model 309 and remover model 313 without the need to produce anomalous parts that would be scrap waste. Furthermore, as the anomalies are basedon known simulated anomaly data 362, the expected result of the anomaly detection system 300 is also known without independent inspection. This beneficially enables the keeper model 309 and remover model 313 to be trained unsupervised such as by the method 1000 of Figure 10.

[0095] The processor 302 further includes a post processing adjustment module 374. The post processing adjustment module is configured to adjust post processing parameters of the anomaly detection system 300 and specifically of the comparison module 316. For example, the post processing adjustment module 374 may adjust parameters such as thresholding and morphological parameters. Adjusting post processing parameters beneficially improves the capability of the system 300 to extract and detect anomalous areas only. The improved extraction beneficially improves the quality of the anomalous area heatmap 318.

[0096] Referring to Figures 5A, 5B and 5C, shown therein are an example inspection image 102, keeper output image 110, and remover output image 114, respectively, according to an embodiment. The inspection image 102, keeper output image 110, and remover output image 114 may be the inspection image 102, keeper output image 110, and remover output image 114 of Figure 2. The inspection image 102 includes areas of interest 530a, 530b, and 530c. The areas of interest 530a, 530b, 530c represent areas where detection of anomalies may affect whether a part is quality conforming. The inspection image 102 further includes an anomalous area 532. The anomalous area 532 is an area where at least one anomaly is present.

[0097] The keeper output image 110 is masked such that only the areas of interest 530a, 530b, and 530c remain. It will be appreciated that in the keeper output image 110 the anomalous area 532 is retained.

[0098] The remover output image 114 is similarly masked. However, in the remover output image 114 the anomalous area 532 of Figure 5A is removed. The removed area is replaced to create a “clean” image.

[0099] Referring to Figures 6A, 6B and 6C, shown therein are an example inspection image 102, keeper output image 110, and remover output image 114, respectively, according to an embodiment. The inspection image 102, keeper outputimage 110, and remover output image 114 are embodiments of the inspection image 102, keeper output image 110, and remover output image 114 of Figure 1. The inspection image 102 includes area of interest 630. The inspection image 102 further includes an anomalous area 632.

[0100] The keeper output image 110 is masked such that only the areas of interest 630 remains. It will be appreciated that in the keeper output image 110 the anomalous area 632 is retained.

[0101] The remover output image 114 is similarly masked. However, in the remover output image 114 the anomalous area 632 of Figure 5A is removed. The removed area is replaced to create a “clean” image.

[0102] Referring to Figures 7A and 7B shown therein is an anomalous area heatmap 120 and hot area 932 respectively, according to an embodiment. The anomalous area heatmap 120 may be the anomalous area heatmap 120 of Figure 2. The anomalous area heatmap 120 includes a “hot” area 732. The hot area 732 includes an anomaly 734 indicated by the colours at the red end of the spectrum.

[0103] Referring to Figures 8A and 8B shown therein is a shape analysis and binarization map 122 and an anomalous area 832 respectively, according to an embodiment. The shape analysis and binarization map 122 may be the shape analysis and binarization map 122 of Figure 2. The shape analysis and binarization map 122 includes an anomalous area 832. The anomalous area 832 includes an anomaly 834. It will be appreciated that anomaly 834 is readily apparent in the shape analysis and binarization map 122 as only detected anomalies are depicted in the map 122.

[0104] Referring to Figures 9A and 9B shown there is a plotted anomalies image 126 and an anomalous area 932, according to an embodiment. The plotted anomalies image 126 may be the plotted anomalies image 126 of Figure 2. The plotted anomalies image 126 includes an anomalous area 932. The anomalous area 932 includes a plotted anomaly 934. The anomaly 934 is plotted by an outline 936 of the detected anomaly . The anomaly 934 is further plotted with anomaly information 938.

[0105] Referring to Figure 10, shown therein is a flow diagram of a method 1000 of training a keeper model and remover model, according to an embodiment. The keeper model and remover model may be the keeper model 309 and remover model 313 of Figure 3 respectively.

[0106] At 1002, the method 1000 includes segmenting a zero padded input image.

[0107] Referring also to Figure 11 , shown therein is a flow chart 1100 depicting example inputs and outputs of the image segmentation, at 1002 of Figure 10, according to an embodiment. Segmenting the zero padded input image 1102 includes applying a section model to the zero padded input image 1102 to obtain a masked image 1106. The zero padded input image 1102 is an embodiment of the inspection image 102 of Figure 1. The zero padded input image 1102 is predetermined to be OK. The masked image 1106 is an embodiment of the masked image 106 of Figure 1 .

[0108] Referring specifically to Figure 10, at 1004, the method 1000 includes augmenting the data.

[0109] Referring also to Figure 12, shown therein is a flow chart depicting example inputs and outputs of the data augmentation, at 1004 of Figure 10, according to an embodiment. Data augmentation at 1004, includes applying simulated anomaly data to the zero padded original image 1102 to obtain a non-masked NOK image 1266. Data augmentation, at 1004, further includes applying the simulated anomaly data to the masked image 1106 to obtain a masked NOK image 1268. Data augmentation, at 1004 further includes passing the zero padded original image 1102 through without augmentation to obtain a non-masked OK image 1267. It will be appreciated that images referred to as OK are also known as good images or conforming images and images referred to as NOK are also known as defective images or non-conforming images.

[0110] Referring specifically to Figure 10, at 1006, the method 1000 includes training the keeper network.

[0111] Referring also to Figure 13A shown therein is a flow chart 1300 depicting example inputs and outputs of the keeper network training, at 1006, of Figure 10 according to an embodiment. Training the keeper network, at 1006, includes applying thekeeper model to the non-masked NOK image 1266 to obtain a NOK output image 131 Oa. The NOK keeper output image 131 Oa is an embodiment of the keeper output image 110 of Figure 1 .

[0112] Referring specifically to Figure 10, at 1008, the method 1000 includes synchronizing masked data.

[0113] Referring also to Figure 13B shown therein is a flow chart 1350 depicting example inputs and outputs of the masked data synchronization, at 1008 of Figure 10, according to an embodiment. Synchronization of the masked data, at 1008, includes applying the keeper model to the non-masked OK image 1267 to obtain an output OK image 1310b. The output OK image 1310b is an embodiment of the keeper output image 110 of Figure 1.

[0114] Referring specifically to Figure 10, at 1010, the method 1000 includes training the remover network.

[0115] Referring also to Figures 14A and 14B, shown therein are flow charts 1400 and 1450, respectively, depicting example inputs and outputs of the remover network training, at 1010 of Figure 10 with the non-masked NOK image 1266 and the non-masked OK image 1267, respectively, according to an embodiment. Training the remover network at 1010 includes applying the remover network to each of the non-masked NOK image 1266 and the non-masked OK image 1267 to obtain OK output images 1414a and 1414b, respectively. The OK output images 1414a and 1414b are embodiments of the remover output image 114 of Figure 1 .

[0116] Referring specifically to Figure 10, at 1012, the method 1000 includes providing the trained keeper network and the trained remover network to an anomaly detection system such as the anomaly detection system 300 of Figure 3.

[0117] Referring to Figure 15 is a flow chart 1500 depicting example inputs and outputs of the detection and plotting anomalies of Figure 1 , according to a further embodiment. Detecting and plotting anomalies includes comparing at 116 the keeper output image 1510 and the remover output image 114. Anomalies 1534 are referred to herein generically as anomaly 1534 and collectively as anomalies 1534. Anomalies 1534are detected based on features identified in the NOK output image 1310a that are not present in the OK output image 1414a.

[0118] At 120, in some embodiments, detecting and plotting anomalies 1500includes conducting a shape analysis and binarization. The shape analysis and binarization includes binarizing the anomalies 1534. Binarization includes discretizing and vectorizing each anomaly 1534. The binarized anomalies 1534 are analyzed to obtain information about each anomaly 1534 such as shape and size. The anomalies 1532 may be represented in a shape analysis and binarization map 1522. The shape analysis and binarization map 1522 is an embodiment of the shape analysis and binarization map 122 of Figure 1 .

[0119] At 124, in some embodiments, detecting and plotting anomalies 1500further includes plotting the detected anomalies 1534. The plotted anomalies generate a plotted anomalies image 1526. The plotted anomalies image 1526 is an embodiment of the plotted anomalies image 126 of Figure 1. Generating the plotted anomalies image 1526 includes indicating the detected anomalies 1534 on an inspection image such as the inspection image 102 of Figure 1 . In some embodiments, each anomaly 1534 is indicated by an outline 1536 surrounding the detected anomaly 1534. In some embodiments (not shown), anomaly information corresponding to each anomaly 1534 may be plotted on the plotted anomalies image 1526.

[0120] Referring to Figure 16, shown therein is a compilation 1600 of various plotted anomalies images 1626a through 1626g as a result of an anomaly detection method such as the anomaly detection method 100 of Figure 1. Each plotted anomaly image 1626a through 1626g includes a plotted anomaly 1634a through 1634. Each plotted anomalies 1634a through 1634g is indicated similarly to the plotted anomaly 934 of Figure 9.

[0121] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art.

Claims

Claims:1 . A method for detecting anomalies to determine if a target part is conforming, the method comprising: applying a keeper model to an inspection image of the target part to obtain a keeper output image, the keeper model configured to retain non-anomalous areas of the inspection image and identified anomalies of the inspection image in the keeper output image; applying a remover model to the inspection image to obtain a remover output image, the remover model configured to retain non-anomalous areas and remove anomalies from the inspection image in the remover output image; and comparing the keeper output image and the remover output image to detect anomalies based on features in the keeper output image that are not present in the remover output image, wherein the keeper model and the remover model are models trained to keep and remove anomalies respectively.

2. The method of claim 1 , wherein one or more of the remover model and the keeper model is an autoencoder model.

3. The method of claim 2 wherein the autoencoder model further comprises skip connections for bypassing hidden layers of the autoencoder model.

4. The method of claim 1 , wherein one or more of the remover model and the keeper model is trained by augmenting an okay (OK) inspection image with simulated anomaly data to obtain an not okay (NOK) inspection image and adjusting post processing parameters based on a comparison of the simulated anomaly data with detected anomalies and corresponding anomaly data obtained by the applying the method to the obtained NOK inspection image.

5. The method of claim 1 further comprising segmenting the inspection image, the segmentation comprising segmenting the image into a plurality of areas and masking areas determined not of interest.

6. The method of claim 1 further comprising generating an anomalous area heatmap based on the comparison of the keeper output image and the remover output image.

7. The method of claim 1 further comprising conducting a shape analysis and binarization on the results of the comparison of the keeper output image and the remover output image to generate a shape analysis and binarization map or obtain anomaly data corresponding to each detected anomaly.

8. The method of claim 1 further comprising plotting the detected anomalies to obtain a plotted anomalies image, the plotted anomalies image including the inspection image augmented with an indication of each anomaly.

9. The method of claim 8 wherein plotting the detected anomalies further comprises plotting anomaly data corresponding to each detected anomaly.

10. The method of claim 1 wherein the keeper model and the remover model retain non-anomalous areas based on a golden sample.

11. A system for detecting anomalies to determine if a target part is conforming, the system comprising: a keeper module configured to apply a keeper model to an inspection image of the target part to obtain a keeper output image, the keeper model configured to retain non-anomalous areas of the inspection image and identified anomalies of the inspection image in the keeper output image; a remover module configured to apply a remover model to the inspection image to obtain a remover output image, the remover model configured to retain non-anomalous areas and remove anomalies from the inspection image in the remover output image; and a comparison module configured to compare the keeper output image and the remover output image to detect anomalies based on features in the keeper output image that are not present in the remover output image, wherein the keeper model and the remover model are models trained to keep and remove anomalies respectively.

12. The system of claim 11 , wherein one or more of the remover model and the keeper model is an autoencoder model.

13. The system of claim 12 wherein the autoencoder model further comprises skip connections for bypassing hidden layers of the autoencoder model.

14. The system of claim 11 , wherein one or more of the remover model and the keeper model is trained by augmenting an okay (OK) inspection image with simulated anomaly data to obtain an not okay (NOK) inspection image and adjusting post processing parameters based on a comparison of the simulated anomaly data with detected anomalies and corresponding anomaly data obtained by the applying the method to the obtained NOK inspection image.

15. The system of claim 11 further comprising a segmentation module configured to segment the inspection image, the segmentation comprising segmenting the image into a plurality of areas and masking areas determined not of interest.

16. The system of claim 11 wherein the comparison module is further configured to generate an anomalous area heatmap based on the comparison of the keeper output image and the remover output image.

17. The system of claim 11 further comprising a shape analysis and binarization module configured to conduct a shape analysis and binarization on the results ofthe comparison of the keeper output image and the remover output image to generate a shape analysis and binarization map or obtain anomaly data corresponding to each detected anomaly.

18. The system of claim 11 further comprising an anomaly plotting module configured to plot the detected anomalies to obtain a plotted anomalies image, the plotted anomalies image including the inspection image augmented with an indication of each anomaly.

19. The system of claim 18 wherein plotting the detected anomalies further comprises plotting anomaly data corresponding to each detected anomaly.

20. The system of claim 11 wherein the keeper model and the remover model retain non-anomalous areas based on a golden sample.

Images