Feature weighting for knn-based anomaly detection

Abstract

To provide a method and system for detecting an anomaly from object images.SOLUTION: A feature extractor provides feature data characterizing part images, including non-defective parts and defective parts. Training data for non-defective parts is used to create a k-nearest neighbors (k-NN) model core set. The feature data includes hundreds of feature vectors, each having hundreds of filter dimensions. A weight value of one is initially assigned to each filter, and test data comprising some non-defective and some defective parts is evaluated by a weighted k-NN module to determine an anomaly score from the weighted feature data. After all test images are evaluated, non-defective and defective data points nearest a threshold are selected and a gradient ascent computation is performed to update the filter weights. Anomaly scoring and gradient ascent are performed iteratively until filter weights are identified which maximize the separation between non-defective and defective scores, thereby eliminating missed detections and false anomalies.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates generally to learning methods for anomaly detection, and more specifically to an anomaly detection method that uses a feature extractor in combination with a weighted k-Nearest Neighbor (k-NN) scoring technique, where each filter is assigned a weight value for feature vector distance calculation, and a gradient ascent method is used to find an optimal combination of filter weights that increases the difference between good scores and anomaly scores, thereby improving both recall and precision in anomaly detection. [Background technology]

[0002] Many industries require the inspection of parts or other objects. This can include inspection for proper fit and / or finish, surface quality, etc. In recent years, these types of inspections are increasingly being performed by computer-based image analysis systems rather than being performed manually by human inspectors. In a typical computer-based image analysis application, images of objects are provided to a computer and an algorithm is run on each object image to perform anomaly detection.

[0003] Anomaly detection systems often include a feature extractor, such as a machine learning system that takes an image as input and outputs a set of feature vectors that characterize the image. Feature extractors dramatically reduce the amount of data required for anomaly detection by replacing images with millions of pixels with a set of feature vectors defined by potentially hundreds of numerical values.

[0004] After feature extraction, another algorithm or calculation is used to evaluate the feature vector data and provide an anomaly score. One effective technique for determining an anomaly score from feature data is known as k-nearest neighbor (k-NN). The k-NN technique is a supervised learning method that determines attribute values ​​from the "k" nearest training examples in a dataset. In anomaly detection applications, the dataset contains feature vector data of the training samples, and the attribute values ​​provided by k-NN become the anomaly scores.

[0005] A fundamental challenge in anomaly detection includes data imbalance between input data representing "good" and "bad" objects. That is, the number of good objects used to train a feature extractor typically far exceeds the number of bad objects. This can make it difficult for the feature extractor to build a model that accurately distinguishes between the characteristics of good and bad objects.

[0006] Techniques are known in the art that attempt to improve the effectiveness of anomaly detection systems. Such techniques range from simple adjustment of thresholds between good and bad scores, to adapting neural network classifiers, to weighting filters one at a time in feature vector calculations. However, none of these existing techniques have been found to be flexible in adaptation and effective in improving anomaly detection results.

[0007] Considering the above circumstances, improved methods are needed for adaptive and learning image-based anomaly detection. Summary of the Invention

[0008] The following disclosure describes a method and system for detecting anomalies from object images. A feature extractor neural network is used to provide feature data characterizing a plurality of images, including images of many good parts and a smaller number of bad parts. A training data set including feature data of only the good parts is used to create a k-Nearest Neighbor (k-NN) model core data set. The feature data includes hundreds of feature vectors, each with hundreds of filter dimensions. Each filter is initially assigned a weight value of 1, and then a test data set including feature data of some good parts and some bad parts is evaluated by a weighted k-NN module to determine an anomaly score from the weighted feature data of each image. After evaluating all test images, the good and bad data points closest to a threshold are selected, and a gradient ascent calculation is performed to update the filter weights to increase the separation between the good and bad scores. The weighted k-NN anomaly scoring and gradient ascent are performed iteratively until a set of filter weights that maximizes the separation between the good and bad scores is identified, allowing for the setting of a threshold that eliminates missed anomalies and minimizes false anomaly classifications.

[0009] Additional features of the present disclosure will become apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the basic structure of an anomaly detection system known in the art. [Diagram 2] FIG. 2 is an illustration of a visually inspected part that includes a flaw in one part that needs to be identified using anomaly detection methods and systems. [Diagram 3] 3 is a block diagram of an anomaly detection system having an offline training portion and an online testing portion, where the online testing portion includes a k-nearest neighbor (k-NN) module for determining anomaly scores from feature data, as known in the art. [Figure 4A]FIG. 4A is a diagram of feature data clusters illustrating the concept of thresholds in k-NN-based anomaly detection according to an embodiment of the present disclosure. [Figure 4B] FIG. 4B is a diagram of feature data clusters illustrating the concept of model adaptation in k-NN based anomaly detection according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a block diagram of a k-NN based anomaly detection system including threshold and model adaptation during online test data processing in accordance with an embodiment of the present disclosure. [Figure 6] FIG. 6 is a flowchart diagram of a method for threshold and model adaptation in a k-NN based anomaly detection system according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a flowchart diagram of a method for threshold and model adaptation during k-NN based anomaly detection, including calculations used to update the k-NN model core set and calculate new thresholds, in accordance with an embodiment of the present disclosure. [Figure 8] FIG. 8 is a diagram of a k-NN model core set and false anomalous data points, showing the calculations used to update the core set in the method of FIG. 7, along with the updated core set, according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a series of plots of part anomaly scores from feature data illustrating the effect of threshold and model adaptation on k-NN based anomaly detection using the method of FIGS. 6 and 7 in accordance with an embodiment of the present disclosure, improving both false anomaly and false negative performance. [Figure 10] FIG. 10 is a schematic block diagram of a convolutional neural network-based (CNN-based) feature extractor system known in the art. [Figure 11] FIG. 11 is a graph of anomaly scores for multiple part images including good and bad parts, illustrating the selection of a set of points to use in performing gradient ascent calculations for feature weighting in accordance with an embodiment of the present disclosure. [Figure 12] FIG. 12 is a block diagram of a k-NN based anomaly detection system including feature filter weighting during online test data processing in accordance with an embodiment of the present disclosure. [Figure 13] FIG. 13 is a flowchart diagram of a method for feature weighting in a k-NN based anomaly detection system according to an embodiment of the present disclosure. [Figure 14] FIG. 14 is a pair of graphs of anomaly scores for multiple part images, including good and bad parts, illustrating the improvement in porosity resulting from the feature weighting method of FIG. 13 in accordance with an embodiment of the present disclosure. [Figure 15] FIG. 15 is a graph of porosity between good and bad parts plotted against iteration number resulting from the feature weighting method of FIG. 13 in accordance with an embodiment of the present disclosure. [Figure 16] FIG. 16 is a graph of filter weights at the optimal number of iterations of FIG. 15 resulting from the feature weighting method of FIG. 13 in accordance with an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following discussion of embodiments of the present disclosure relating to methods for feature weighting in k-NN-based anomaly detection is merely exemplary in nature and is not intended to limit the disclosed technology or its applications or uses.

[0012] 1 is a block diagram showing the basic structure of an anomaly detection system 100 as known in the art. Block 110 provides input. The input of block 110 is typically visual input, such as an image of a part or workpiece. In some applications, the input is a graphical or data input, such as data from an accelerometer or other sensor that characterizes the operation of the equipment. In either case, the input of block 110 is used to determine whether the item being analyzed (part or workpiece, or equipment) is normal (a.k.a. "good", "OK", "nominal") or abnormal ("bad", "defective").

[0013] The input from block 110 is provided to an algorithm 120 that determines an anomaly score 130. Based on the anomaly score 130, the item under analysis is classified as either normal / good or abnormal / bad.

[0014] Figure 1 is intended merely to illustrate the basic concepts and components of an anomaly detection system, which serve as the basis for further discussion below.

[0015] Figure 2 is a diagram of a visually inspected part that includes a flaw in one portion that needs to be identified using an anomaly detection method and system. In the example of Figure 2, the anomaly detection system is used to classify the screw head as good or bad based on an image of the screw head. The screw head may have just been manufactured (such as on a header machine) or the screw may have just been installed in an assembly. In yet another situation, the screw is part of an assembled product, and an electrical connection or performance test is performed by contacting an external probe to the screw head.

[0016] The screw head 200 on the left side of Fig. 2 is a normal (good) part without any flaws that would be rejected as an anomaly. However, the image showing the screw head 200 contains some distinctive features that make the task of anomaly detection difficult. For example, shadows 202 and reflections 204 often appear in the image due to imperfect lighting conditions, surrounding objects, camera angles, etc. For an object with a shiny and uneven surface such as the screw head 200, it is very difficult to provide an image completely free of shadows and reflections.

[0017] The screw head 200 includes a recess 206, in this case shaped to receive a Phillips head screwdriver bit. The recess 206 is clearly a required geometric feature of the screw head 200 and is not a defect or anomaly. However, the multiple surfaces and shadows, among other features, of the recess 206 make evaluation of an image of the screw head 200 quite complicated.

[0018] The screw head 220 on the right side of Figure 2 is an abnormal (bad) part that has several blemishes and should be rejected as such. The image of the screw head 220 has the same shadows and reflections as the image of the screw head 200, and the same recess 206. The image of the screw head 220 is rotated approximately 45 degrees relative to the image of the screw head 200, illustrating another difficulty associated with visual anomaly detection: shadows, reflections, and other geometric features may appear in different locations and orientations from part to part, so simply comparing a new image to a known good part image is insufficient to reliably detect anomalies.

[0019] The screw head 220 includes blemishes 222, 224, and 226. The blemishes 222-226 may be of one or more types, including blemishes from the head preparation machine, flats or other defects, or burns caused by arcing when an external electrical probe contacts the screw head 220. Any one of the blemishes 222-226 is sufficient to cause the screw head 220 to be classified as abnormal. Three different blemishes are included to illustrate that blemishes may appear on different parts of the screw head 220 and may appear in different locations relative to shadows and reflections. The blemishes 222-226 themselves may have different shapes, different lighting and reflective properties.

[0020] Figure 2 and the above discussion are intended to provide additional context for the discussion of anomaly detection. Furthermore, the techniques of the present disclosure have been applied to anomaly detection in screw heads of substantially the same type as that shown in Figure 2. The effectiveness of the disclosed methods and systems is discussed later in this disclosure.

[0021] As discussed above, due to the complexities encountered in real-world anomaly detection, various techniques have been developed for anomaly detection with self-learning capabilities and advanced anomaly score analysis.

[0022] 3 is a block diagram of an anomaly detection system 300 having an offline trainer and an online tester, as known in the art. The system 300 includes an offline trainer 302 and an online tester 304. The offline trainer 302 is used to pre-train a database to recognize the features of good parts, and the online tester 304 includes a k-nearest neighbor (k-NN) module to evaluate parts with unknown classification and determine an anomaly score from the feature data.

[0023] The offline learner 302 provides training data 310, where the training data 310 includes images of parts whose classification (good or abnormal) is known. Each image in the training data 310 can be of any of the types discussed above, such as an image of a screw head, an image of a painted surface to determine the quality of a paint finish, or other types. Each image in the training data 310 is provided to a feature extractor module 320, which may be a convolutional neural network (CNN) that extracts feature vectors from the images, as known in the art. As features 330 are identified as being extracted from good or abnormal parts in the training data 310, the features 330 are used to build a memory bank 340 that distinguishes between good and abnormal parts. In some applications, only good parts are provided in the training data 310.

[0024] In the online testing unit 304, the input 350 typically includes multiple images from parts whose classification (good or abnormal) is unknown. Each image from the input 350 is provided to a feature extractor module 360, which may be the same CNN-based feature extractor module 320 as the offline training unit 302. For each image, the feature extractor module 360 ​​provides a feature 370. The * in element 370 simply indicates that the feature 370 is different from the feature 330 because the input 350 is different from the training data 310. The feature 370 is provided to a k-NN module 380, which determines an anomaly score based on the feature 370 for each individual part. The k-NN module 380 compares the features 370 of each part to the memory bank 340 that was pre-trained in the offline training unit 302. The anomaly score for each part is determined by the k-NN module 380 based on the "distance" (in feature vector space) between the features of the part and the features of a core set of good parts in the memory bank 340. Parts with features that closely match the core set (i.e., very short distance) will have an anomaly score close to 0. Parts with features that do not closely match the core set (i.e., large distance) will have a high anomaly score (e.g., greater than 1).

[0025] At block 390, a final part classification is provided. The final classification is typically either -1 (for a good part) or 1 (for a bad or abnormal part). The criteria for assigning the final classification value based on the anomaly score varies by application, usually by comparison to a threshold value.

[0026] Despite its relatively sophisticated feature extractor module and k-NN-based anomaly score determination capability, system 300 still has difficulty distinguishing between good and abnormal parts in all cases. In most anomaly detection applications, there are many parts that are clearly good and have low anomaly scores, and a few parts that are clearly bad and have high anomaly scores. However, there are typically some parts that receive anomaly scores (e.g., 0.95, 1.02) that are close to a cutoff or threshold value (e.g., 1.0), and it is not clear whether such parts are good or bad.

[0027] The difficulty of reliably detecting anomalies in existing systems such as the system 300 of Fig. 3 is due in part to the complexities of the real world as illustrated in Fig. 2 and discussed above. Another reason is that the distribution of good data in the input 350 (during online testing) may differ from the distribution of good data in the training data 310 (during offline learning). For example, some features of the good data in the online testing phase may never occur in the offline training phase. Thus, the pre-training performed in the offline learning unit 302 is necessarily incomplete.

[0028] The techniques of this disclosure have been developed to overcome limitations of existing anomaly detection methods and systems, including systems such as system 300 of FIG. 3 discussed above.

[0029] A first technique for overcoming the limitations of existing anomaly detection methods and systems includes threshold and model adaptation, which directly addresses the problem of system 300 discussed above, namely, the inability of the training data to fully and accurately represent the complete set of features of good parts.

[0030] 4A and 4B are diagrams of feature data clusters illustrating the concepts of threshold and model adaptation, respectively, in k-NN based anomaly detection according to an embodiment of the present disclosure. In FIG. 4A, a number of feature data points are depicted as locations according to anomaly scores and plotted on a vertical axis 402. Ellipse 410 includes feature data points for items (parts or input images) that are determined to be good (e.g., by human inspection). Ellipse 420 includes feature data points for items (parts or input images) that are determined to be bad (anomaly or defect).

[0031] The threshold line 430 is the initial threshold used by the k-NN anomaly score evaluator to distinguish between good and bad parts. That is, points with an anomaly score above the threshold are classified as bad (anomaly) and vice versa. Using the threshold line 430, feature point 412, known to belong to a good part, is erroneously classified as an anomaly. This situation is known as a false anomaly classification. Furthermore, using the threshold line 430, feature point 422, known to belong to a bad / abnormal part, is erroneously classified as a good part. This is known as a missed anomaly. While both false anomalies and missed anomalies are undesirable, criteria for anomaly detection systems often dictate that missed anomalies are not tolerated (i.e., bad parts are not allowed to pass undetected).

[0032] To eliminate the false negative detection of point 422, a new lower threshold line 432 is defined. Although threshold line 432 does in fact eliminate all false negative detections in the data set shown in Figure 4A, a lower threshold exacerbates the problem of false anomaly classification because other feature data points of good parts exceed threshold line 432. Figure 4A illustrates an important lesson about k-NN-based anomaly detection: threshold tuning alone cannot overcome the interrelated problems of false anomaly classification and false negative detection.

[0033] In Figure 4B, model 450 (left) shows multiple feature data points of good parts along with feature data point 460 of a part classified as anomalous. Boundary 452 defines a boundary around a core set of feature points of the good parts (the feature points of the good parts are inside boundary 452 because only a portion of model 450 is shown, and are to the right of boundary 452 in the figure). Feature data point 460 was classified as anomalous, but is near the core set of good parts in model 450.

[0034] Adaptation of model 450 may be necessary if the part or image associated with feature data point 460 is determined (such as by human inspection) to be a good part rather than an anomaly. On the right, model 470 is defined by boundary 472. Model 470 is an adaptation of model 450 that includes feature point 460 in a core set of good parts. This adaptation is shown here as simply redrawing the boundary. However, techniques are described below for continually incrementally adapting the feature points included in the model core set, including removing feature points from the core set as new points are added. The techniques described below combine model adaptation and threshold adaptation to significantly improve anomaly detection performance.

[0035] 4A and 4B (and subsequent figures) are depicted in two dimensions to simplify discussion of the concepts involved: as anyone familiar with feature extraction knows, each feature "point" is actually a feature vector with dimensions that may be in the hundreds, and calculations of location and distance between points are actually performed in vector space, not in two-dimensional space.

[0036] The threshold and model adaptation techniques of the present disclosure use human inspection in combination with specially designed computational logic to gradually adapt both the k-NN model core set and thresholds within the anomaly detection system, and are discussed in more detail below.

[0037] 5 is a block diagram of a k-NN based anomaly detection system 500 including threshold and model adaptation during online test data processing, according to an embodiment of the present disclosure. System 500 has a structure generally similar to system 300 of FIG 3, including an offline training portion and an online testing portion. System 500 adds threshold and model adaptation computations in conjunction with determining the k-NN anomaly scores, as described below.

[0038] The offline trainer 302A is essentially the same as the offline trainer 302 of Figure 3, except that training data 310A is provided to the feature extractor module 320 and features 330 are contained in a memory bank 340A. In a preferred embodiment, the training data 310A of the system 500 contains only images of good parts and no bad / abnormal parts, as the goal is to initially populate the memory bank 340A with a core set of known good feature data, define a low initial threshold, and adapt the memory bank 340A and the threshold during online test data processing.

[0039] In the online testing portion 504, the input 510 includes multiple images from parts whose classification is unknown (i.e., each part is not known to be good or abnormal). Each image from the input 510 is provided, one at a time, to a feature extractor module 520, which is preferably the same type of CNN-based feature extractor module discussed above. Features 530 are provided from the feature extractor module 520 for each image. Again, the features 530 characterize the parts whose classification is unknown (including both good and bad / abnormal parts). The features 530 are provided to a k-NN module 540, which determines an anomaly score based on the features of the individual parts.

[0040] The k-NN module 540 compares the features 530 of each part to the memory bank 340A pre-trained in the offline learner 302A. The anomaly score for each part is determined by the k-NN module 540 based on the "distance" (in feature vector space) between the features of that part and the features of the nearest parts in the core set of good parts in the memory bank 340A. Parts with features that closely match the core set (i.e., the distance is very short) will have a low anomaly score. Parts with features that do not closely match the core set (i.e., the distance is large) will have a high anomaly score, e.g., greater than 1.

[0041] A threshold and model adaptation module 550 is added to the system 500. The threshold and model adaptation module 550 performs a series of evaluations and calculations after the anomaly scores for each image are provided by the k-NN module 540. Functions performed by the threshold and model adaptation module 550 include determining whether items with an anomaly score above a threshold are false anomalies, updating a core set of good part features in the memory bank 340A, updating the thresholds used for final part classification, and updating the anomaly library 560 used in the calculations. The dashed lines to the memory bank 340A and the anomaly library 560 indicate the adaptations and updates performed on such elements, with the threshold values ​​maintained within the threshold and model adaptation module 550 itself. Details of such evaluations and calculations are discussed below in conjunction with Figures 6 and 7.

[0042] Block 570 provides a final part classification. The final classification is either -1 (for a good part) or 1 (for a bad or abnormal part). The final classification value is based on the anomaly score compared to a threshold, where the model core set used by the k-NN module 540 to determine the score and the threshold used to perform the final classification both evolve as more images are processed by the Threshold and Model Adaptation module 550.

[0043] A preferred embodiment of system 500 uses a relatively large number of good part images and a relatively small number of bad part images, such as several hundred good part images and several tens of bad part images. In one non-limiting embodiment, one-third of the good part images (and no bad part images) are used for training data 310A of offline learning portion 302A, one-third of the good part images and half of the bad part images are used for adaptation at input 510 of online testing portion 504, and the remaining one-third of the good part images and the remaining half of the bad part images are used for testing at input 510 of online testing portion 504. In this embodiment, after each adaptation step (one image is processed and thresholds and models are adapted), the entire set of test images is processed to evaluate the performance of system 500.

[0044] 6 is a flowchart diagram 600 of a method for threshold and model adaptation in a k-NN based anomaly detection system according to an embodiment of the present disclosure. Following the start oval 602, the threshold α is initialized in box 604. The threshold α is initialized after performing online learning using only good part data. That is, the threshold α is initialized to a relatively low value (less than 1.0) before starting adaptation using the online testing portion 504. The idea is to initialize the threshold α to a value high enough to classify most good parts as good, but low enough to classify all bad parts as anomalies (no missed anomalies).

[0045] In box 606, new feature data (for a single image) is provided. This is the data of features 530 in FIG. 5, which is provided to the k-NN module 540, which in box 608 provides an anomaly score based on a comparison of the features with the model core set from memory bank 340A. In decision diamond 610, it is determined whether the anomaly score is greater than a threshold α. This determination and the remaining steps of FIG. 6 are performed in threshold and model adaptation module 550 in FIG. 5. If the anomaly score is not greater than the threshold α, in box 612, memory bank 340A is updated, including adding features of the current (good) image to the model core set if certain criteria are met. The criteria include determining the distance from each feature point in the current feature set to the closest feature point in the core set, and replacing the existing feature point with a feature point from the current feature set if the distance is greater than the minimum inter-point distance of the existing feature points in the core set. This is discussed further below and illustrated in FIG. 8.

[0046] From decision diamond 610, if the anomaly score is greater than a threshold α, indicating an anomaly, then the part or image corresponding to the current feature data is inspected by a human at box 618. The human inspector classifies the part image as either good or bad (anomaly). If the human inspector classifies the part image as good, then at decision diamond 620 the image feature data is determined to be a false anomaly.

[0047] In the case of a false anomaly, the process moves to box 622 where the core set (of good part feature data) is adapted to memory bank 340A. This involves evaluating current feature data known from human inspection to belong to good part images to determine whether to add the current feature data to the core set. This evaluation is described in more detail with reference to Figures 7 and 8. After model adaptation in box 622, threshold α is adapted to increasingly higher values ​​in box 624. Threshold adaptation is also described in more detail in the discussion of Figure 7.

[0048] If the human inspector determines that the part / image is indeed anomaly, meaning it is not a false anomaly, decision diamond 620 moves the process to box 626 where anomaly library 560 is updated to include the current feature data. The suitability of anomaly library 560 in adaptively updating threshold α is discussed below in connection with FIG. 7.

[0049] After model and threshold adaptation in boxes 622 and 624, or anomaly library updating in box 626, the process returns to box 606 where new feature data is provided (for the next part image in input 510). If, at decision diamond 614, there are no more parts to analyze, the process ends at end point 616.

[0050] It was noted above that a set of adaptation data images is provided to the online testing unit 504 and, after each adaptation data image is processed, the entire set of test data images is processed to determine the system anomaly detection performance. For this reason, the steps of FIG. 6 are performed only on the adaptation data images and not on the test data images. For this reason, the system 500 of FIG. 5 is preferably configured with a "switch" that indicates whether the system is operating in an adaptation mode or a test mode. The threshold and model adaptation steps of FIG. 6 (and FIG. 7) are performed when the system is in the adaptation mode. When the system is in the test mode, the module 550 simply classifies each part image based on its anomaly score compared to the current value of the threshold α.

[0051] 7 is a flowchart diagram 700 of a method for threshold and model adaptation during k-NN based anomaly detection, including calculations used to update the k-NN model core set and calculate new thresholds, according to an embodiment of the present disclosure. FIG. 7 and the accompanying discussion provide details of the threshold and model adaptation calculation steps included in the method of FIG.

[0052] The process begins at start oval 702 upon detecting a false anomaly at decision diamond 620 of Figure 6. In box 704, data from memory bank 340A (containing model core set M of good part feature data) is provided along with anomaly library 560 (containing anomaly / fault part feature data). In addition, box 704 provides a minimum distance vector d that contains the minimum distance between feature vectors in model core set M. Additionally, an incremental step l is performed. r and discount factor γ are provided in box 704, where such parameters are used in the threshold update calculations discussed below.

[0053] In box 706, feature data for false abnormal parts is provided. These are parts / images that have an anomaly score above the threshold in decision diamond 610 and have been determined by a human inspector to be good parts rather than abnormal. The calculations described here are performed in threshold and model adaptation module 550 of FIG. 5. In one embodiment, the feature data consists of a set of 784 feature vectors (a 28×28 grid), each with a dimension of 384. The exact composition of the feature data will vary depending on the design of the feature extractor module 520, as will be appreciated by those skilled in the art.

[0054] After the offline learning phase (see FIG. 5), memory bank 340A contains a model core set M that contains feature vectors of only good parts. Model core set M may initially contain 784 feature vectors for each of dozens (or hundreds) of part images, where the number of feature vectors can be hundreds of thousands. To reduce inference time during the online testing phase, model core set M is reduced to an initial 10 6 Instead of 10 feature vectors, 5 The downsampling may be performed in any suitable manner to include a feature vector number of an order of magnitude.

[0055] Next, it is determined whether to include one or more feature vectors from the current (false-abnormal) part in the model core set M. The idea behind this is that the current false-abnormal part had feature data that caused its anomaly score to be higher than a threshold, but human inspection determined it to be a good part. Thus, including features from the current false-abnormal part image in the model core set M may potentially make the core set more robust to various image characteristics (recall the example in FIG. 2). In a preferred embodiment, if a feature vector from the current false-abnormal part is included in the model core set M, the feature vector is also removed from the model core set M to maintain a constant core set size.

[0056] In box 708, for each of the feature vectors of the current erroneous abnormal part (e.g., 784 in total), the distance from the feature vector to the closest feature vector in the model core set M is calculated. Then, in decision diamond 710, it is determined whether the distance calculated in box 708 is greater than the minimum distance (provided in d) between feature vectors in the model core set M. If the answer is no, the process proceeds to decision diamond 714 and returns to box 708 to calculate the distance of the next feature vector.

[0057] At decision diamond 710, if the distance calculated in box 708 is greater than the minimum distance between the feature vectors in the model core set M, then at box 712 one of the two feature vectors involved in the minimum distance is removed from the model core set M and replaced with the feature vector currently being calculated. The model core set M and the minimum distance vector d are then updated in memory bank 340A.

[0058] FIG. 8 is a diagram of a k-NN model core set M and a false anomalous data point 820, visually representing the calculations used to update the core set M in the method of FIG. 7 according to an embodiment of the present disclosure. The core set M, indicated at 810A, includes feature vectors ("points") 812, 814, and 816, etc. As discussed above, calculate the distance 830 between the false anomalous feature vector (point) 820 and the feature vector 816, which is the closest feature vector in the core set M. The distance 832 is the minimum distance between any pair of feature vectors in the core set M. All distances are calculated in the feature vector space. Since the distance 830 is greater than the distance 832, the logic of FIG. 7 removes one of the feature vector points 812 / 814 from the core set M and replaces it with the feature vector 820.

[0059] 8 shows an updated core set 810B where feature vector point 814 has been removed from core set M and feature vector 820 has been included in core set M. Again, the goal of this model core set adaptation is to increase the diversity of feature vectors (of known good parts) included in core set M. This, along with subsequent threshold adaptations, will reduce the number of false anomaly classifications.

[0060] In the above manner, all feature vectors of the current false abnormal part are evaluated, some of which may be included in the model core set M as replacements for the dense core set feature vectors. This continues in the method of FIG. 7 until no feature vectors are present for the current false abnormal part at decision diamond 714.

[0061] The process then moves to box 716 to begin the steps involved in adapting the threshold based on an evaluation of the current false abnormal part. The steps of boxes 716-720 of FIG. 7 are implemented in box 624 of FIG. 6. In box 716, a revised anomaly score s' is calculated in k-NN module 540 based on the feature vector of the current false abnormal part and the revised model core set M from memory bank 340A. In box 718, an upper bound score s' is calculated based on all anomaly data in anomaly library 560 compared to the revised model core set M from memory bank 340A. u is calculated by the k-NN module 540.

[0062] In box 720, a new value for threshold α is calculated that is incrementally higher than the existing value of threshold α. The new value for threshold α is calculated in box 720 as follows:

[0063]

number

[0064] Here, the first term is determined by choosing the higher of the revised anomaly score s′ and the old value of the threshold α, and then increasing the incremental step l r Add this to the second term, which is calculated by multiplying by the discount factor γ. The sum of the terms and the upper score s u The lower of these is selected as the new value of the threshold α. The logic embodied in equation (1) is that if the revised anomaly score s′ is greater than the old threshold because the current false anomaly part is actually a good part, then the revised anomaly score s′ is accepted as the starting point for the new threshold. Regardless of whether s′ or α is chosen as the starting point, the term γ·l r adds an increment to the calculation of the new threshold, which is the upper score s u The upper limit is determined by.

[0065] After the adaptive calculation of the threshold in box 720, the method of Figure 7 ends at terminal 722. This corresponds to the method of Figure 6 completing box 624 and returning to processing new data (data for another image from input 510). Each time a false anomaly is encountered, steps of elements 618-624 of Figure 6 are performed along with the entire threshold and model adaptation process of Figure 7.

[0066] The foregoing discussion of Figures 6-7 describes several evaluations and calculations that are performed in the process of threshold and model adaptation. With reference to the system block diagram of Figure 5, it will be understood that the various blocks and modules are all embodied in algorithms running on a processor (or multiple processors in communication with each other) such that the necessary data is available to each computational module as needed for feature extraction, k-NN scoring, threshold and model adaptation, updating memory bank 340A and anomaly library 580, etc. This is visually represented by the arrows in Figure 5.

[0067] FIG. 9 is a series of plots of part anomaly scores from feature data illustrating the effect of threshold and model adaptation on k-NN based anomaly detection using the method of FIGS. 6 and 7 in accordance with an embodiment of the present disclosure, improving both false anomaly and false negative performance.

[0068] A set of feature vector points is shown in the initial situation 900 (left side), with the points arranged according to their respective anomaly scores plotted on the vertical axis. The feature vector points shown in 900 show the result of sending multiple test data images to the system 500 after only offline learning from portion 302A has been performed and no adaptation has yet been performed. Set 910 contains feature data points for items (parts or input images) that have been determined to be good (e.g., by human inspection). Set 920 contains feature data points for items (parts or input images) that have been determined to be bad (anomaly). The threshold line 902 is set to a low value to prevent missed anomalies. With a low threshold and no model adaptation performed, a large number of false anomalies will occur, as shown in box 904.

[0069] After processing a number of adaptation data images in system 500, some threshold and model adaptation has occurred, as shown in situation 930. In situation 930, set 940 contains feature data points of good items from set 910, where the data points have generally moved toward lower anomaly scores due to model adaptation. Set 950 contains feature data points of the same bad items as set 920. Additionally, in situation 930, threshold adaptation has resulted in a gradual upward movement of threshold line 932. With a slightly higher threshold and some model adaptation, the number of false anomalies is reduced, as shown in box 934. There are still no missed anomalies in situation 930, because the threshold adaptation involves the calculations discussed above, limiting the upward movement of the threshold to prevent missed detections.

[0070] After processing additional adaptation data images in system 500, threshold and model adaptation has reached a final or near-final state as shown in situation 960. In situation 960, set 970 contains the same feature data points of good items from sets 910 and 940, where the data points have been further moved in the direction of lower anomaly scores due to model adaptation. Set 980 contains the same feature data points of bad items as sets 920 and 950. Additionally, in situation 960, threshold adaptation has resulted in further upward movement of threshold line 962. Due to the higher threshold and additional model adaptation, no false anomalies have been introduced in situation 960. Furthermore, in situation 960, no anomalies are still missed for the reasons discussed above. It can be seen that once threshold and model adaptation is in effect, the amount of human inspection required will be dramatically reduced, and ultimately, all or nearly all of the parts classified as anomalies will actually be true anomalies.

[0071] The above threshold and model adaptation techniques have been shown through experiments to separate good and bad data point sets in an anomaly detection system while continuously setting the threshold to an optimal value. Such techniques achieve an ideal combination of anomaly detection results, i.e., zero missed anomaly detections and zero or near zero false anomaly classifications. This ideal combination of results is not possible with conventional anomaly detection methods.

[0072] Another technique disclosed herein for overcoming limitations of existing anomaly detection methods and systems includes feature weighting, which involves assigning weights to filters used to calculate distances between feature vectors when calculating anomaly scores, and optimizing the weights to achieve optimal separation of good and bad data (e.g., part images). More details about feature weighting techniques are discussed below.

[0073] Figure 10 is a schematic block diagram of a convolutional neural network-based (CNN-based) feature extractor system known in the art. The feature extractor module has been discussed repeatedly previously in this disclosure, including its use in the anomaly detection system using threshold and model adaptation of Figure 5. Figure 10 is provided as a basis for discussing some of the feature extractor details important to the feature weighting techniques described below.

[0074] As shown in the previous figures and discussed above, an input image 1010 is provided to the feature extractor module 1020. As will be appreciated by those skilled in the art, a convolutional neural network (CNN) is a common structure used for the feature extractor module, and in fact, pre-trained feature extractor CNNs are available, where the network structure, including the number of layers and the size of each layer output, is fixed and the network parameters are trained. The feature extractor module 1020 is an example of such a pre-trained feature extractor CNN. Other examples of feature extractor structures with different numbers of layers and feature vectors may also be used in the feature weighted anomaly detection techniques disclosed herein.

[0075] In the example of FIG. 10, an input image 1010 is provided to a CNN 1020 that includes three layers (L1, L2, L3) at a particular pixel resolution. Each layer provides an output that is input to the next layer and / or is provided as a usable output from the CNN 1020. In this example, based on the predefined structure and training of the CNN 1020, layer L2 includes 128 filters, each of which results in a 28×28 feature map. Layer L3 includes 256 filters, each of which results in a 14×14 feature map. Through interpolation, the L3 output is converted to a 28×28 feature map, which matches the size of the output from L2. Such feature maps and filter numbers are visually shown at 1030.

[0076] The final output of the CNN 1020 used in k-NN anomaly detection is shown at 1040. The output shown at 1040 is an array of 28x28 feature vectors 1042, each with dimension {384x1}. The total number of feature vectors is 784 (28x28). The dimension of each feature vector (384) represents 128 filters at the L2 output and 256 filters at the L3 output. The output shown at 1040 (784 feature vectors with 384 filter components each) is used in the k-NN based anomaly detection computation.

[0077] Returning to Figure 3 and our earlier discussion of feature extraction and anomaly detection, the anomaly score is calculated by calculating the distance in vector space between each feature vector of an image and its nearest neighbor in a core set of feature vectors of good images. The smaller the average distance of an image's feature vectors, the smaller the anomaly score (more likely a good part) and the larger the average distance, the larger the anomaly score (i.e., if the score exceeds a threshold, it is a bad / anomaly part).

[0078] When calculating the distance between the feature vector of the part / image currently being evaluated and the feature vectors of the good parts in the model core set, it is common to calculate the unweighted distance between one vector and another, giving equal weight to all of the vector components (each vector has dimensions of, for example, {384×1}). In accordance with the techniques of this disclosure, a weighted distance is used by assigning a weight to each filter component of the vector being evaluated. Each of the 384 filter components is assigned an initial weight value of 1.0, and an iterative optimization process is employed to recalculate the anomaly scores using weighted k-NN calculations and adjust the filter weights by gradient ascent calculations to maximize the gap between the anomaly scores of the good and bad parts. This technique is discussed in more detail in the remaining figures.

[0079] 11 is a graph 1100 of anomaly scores for multiple part images, including good and bad parts, illustrating the selection of a point set to use in performing a feature weighting gradient ascent calculation, according to an embodiment of the present disclosure. The data points on the graph 1100 are positioned vertically according to the anomaly scores of the data points (part images) and simply positioned horizontally according to the image number.

[0080] To optimize the filter weights for anomaly detection, multiple part images need to be processed at each iteration step. Specifically, set 1110 contains data points for items (part images) that are determined to be bad (e.g., by human inspection). Set 1120 contains data points for items (part images) that are determined to be good. A threshold line 1102 can be plotted at the anomaly score value (approximately 1.75) to distinguish between good and bad parts. It can be seen that using the threshold line 1102, there are no missed anomalies and no false anomaly classifications, but there is little vertical separation or gap between the good part data point set 1120 and the bad part data point set 1110. In other words, without knowing in advance whether a part image is good or bad, it would be very difficult to establish a threshold that would reliably classify a part as good or bad based on the anomaly score. The value of the threshold line in this example is approximately 1.75. In other examples using different images and / or different feature extractors and core sets, the good / bad part thresholds may be lower or higher. The actual value of the threshold is not important for this discussion. What is important is to separate the anomaly scores of good and bad parts so that a threshold can be established that reliably distinguishes between the two.

[0081] The objective of the techniques disclosed herein is to increase the separation or difference between the anomaly scores of good and bad parts using an optimization-based feature weighting calculation. With such optimized filter component weighting values, processing additional images of the same object (e.g., screw head) benefits from the increased difference and the resulting improved anomaly detection performance.

[0082] As explained above, multiple part images need to be processed at each iteration step. In the actual implementation considered here and shown in the figures following FIG. 11, 56 part images were included in the test data set, as sometimes indicated by the scale on the horizontal (image number) axis. This set of 56 images was processed at each iteration of the weighted optimization algorithm considered below. Another concept illustrated in FIG. 11 is related to the gradient ascent optimization calculation. Specifically, the gradient ascent calculation is implemented using the good and bad test data points that are closest to the threshold line 1102. This provides a kind of leverage for the gradient ascent calculation to increase the difference between the anomaly scores of good and bad parts. In FIG. 11, which is the initial graph of the anomaly scores before feature weighting, ellipse 1112 contains some bad data points that should be included in set P, and ellipse 1122 contains some good data points that should be included in set P. 11 includes three good and three bad data points, individual implementations may use more or less than three, and may actually use just one of each (one good and one bad). The selection and use of points in set P is discussed further below.

[0083] Figure 12 is a block diagram of a k-NN based anomaly detection system 1200 including feature filter weighting during online test data processing according to an embodiment of the present disclosure. System 1200 has a similar structure to the systems of Figures 3 and 5, including an offline learning portion and an online testing portion. System 1200 includes feature weighting calculation in conjunction with k-NN anomaly scoring, as described below.

[0084] The offline learner 302 is the same as the offline learner 302 of Figure 3, where training data 310 is provided to a feature extractor module 320 and features 330 are included in a memory bank 340. In system 1200, the training data 310 only includes images of good parts and no bad / abnormal parts, as the goal is to create a memory bank 340 with a core set of known good feature data and use the model core set throughout the feature weighting calculations.

[0085] In the online testing portion 1204, the input 1210 includes a number of images from a known classification of parts (each part image is labeled as either good or abnormal). In this example, there are 56 images in the input 1210. Each image from the input 1210 is provided to a feature extractor module 1220, which is a CNN-based feature extractor module of the type described above with respect to FIG. 10. Features 1230 are provided from the feature extractor module 1220 for each image. Additionally, the features 1230 in this example include 784 feature vectors for each part image, with each feature vector including 384 filter components. The features 1230 for each of the 56 part images, along with the label of each image (good or bad), are provided to a weighted k-NN module 1240, which determines an anomaly score for each individual part image.

[0086] The weighted k-NN module 1240 compares the features 1230 of each part to a model core set from the memory bank 340 that was pre-trained in the offline learner 302. The anomaly score of each part is determined by the weighted k-NN module 1240 based on the weighted distance (in feature vector space) between the features of the part and the features of the nearest parts in the core set of good parts in the memory bank 340. Parts with features that closely match the core set (i.e., the distance is very small) will have a low anomaly score. Parts with features that do not closely match the core set (i.e., the distance is large) will have a high anomaly score. In the weighted k-NN module 1240, the distance is calculated using weight values ​​assigned to each of the 384 filter components. The weight values ​​are contained in a weight vector w of size {384×1}, with all values ​​of w initially set to 1.0.

[0087] The system 1200 includes a filter weight optimization module 1250, which calculates optimal values ​​for the weight vector w in an iterative loop with the k-NN module 1240. Functions performed by the filter weight optimization module 1250 include defining a set P based on the anomaly scores (e.g., 56 scores) of all part images, and performing a gradient ascent calculation to calculate new values ​​for the weight vector w and update the weight library 1260. The model core set (of good part feature data) is unchanged and provided from the memory bank 340 to the weighted k-NN module 1240. The updated weight vector w is also provided from the weight library 1260 to the weighted k-NN module 1240, which calculates new anomaly scores for all part images based on the new filter weights. The iterative weight optimization loop of modules 1240 and 1250 continues for a prescribed number of iterations or until it is determined that the weight values ​​have been optimized. Further details of such calculations are discussed below with reference to FIG. 13.

[0088] A final part classification is provided for each part image at block 1270. The final part classification at block 1270 is not performed during the iterative weight optimization loop of modules 1240 and 1250. However, after the filter weights have been optimized as discussed above, the online testing portion 1204 can be used to process the part images and perform anomaly detection on the unlabeled images (i.e., each image is not known to be a good or bad part). This portion includes the final part classification at block 1270. Using the optimal weights and a threshold assigned to a value in the gap between good and bad parts, the system 1200 will reliably classify the part images without missing an anomaly or falsely classifying an anomaly.

[0089] The final classification is typically either -1 (good part) or 1 (bad or abnormal part), although other conventions may be defined. The final classification value is based on the weighted k-NN anomaly score compared to a threshold, where the weight values ​​used by the weighted k-NN module 1240 to determine the score are optimized in an iterative manner as discussed above, and the threshold used to make the final classification is established based on the score distribution after weight optimization.

[0090] Fig. 13 is a flowchart diagram 1300 of a method for weighting features in a k-NN based anomaly detection system according to an embodiment of the present disclosure. The method of Fig. 13 is implemented in the system 1200 of Fig. 12, specifically in the online testing component 1204, with the iterative steps being specifically implemented in the weighted k-NN module 1240 and the filter weights optimization module 1250.

[0091] The process begins at start oval 1302 when online testing portion 1204 is used in training mode. In box 1304, feature data for labeled or classified part images is provided, including images of known good parts and images of known bad parts. In the example discussed above and continued here, 56 part images are provided for training the feature weighted anomaly detection system. Of course, more or less than 56 part images may be used in the test data set. If more input test images are required than the number of available part images, data augmentation may be achieved by horizontally and vertically flipping each available part image. For purposes of this computational discussion, bad part images (i.e., anomalies or defects) are defined as those included in the defect data set D, and good part images are defined as those included in the good part data set G.

[0092] In box 1306, the model core set M of good part feature data from memory bank 340 is provided along with certain input parameters. The input parameters required for the calculation include a score function f that determines an anomaly score for the part image using a weighted k-NN calculation (based on the feature vector data of the part image, the model core set M, and the weight vector w), a label y that is defined to have a value of 1 for a defective part image and −1 for a good part image, a function g that is defined as g=yf (i.e., the value of g is the value of the anomaly score from f for a defective part and the value of g is the negative of the value of the anomaly score from f for a good part), and a learning rate coefficient α.

[0093] In box 1308, a weight vector w is initialized with a value of 1 (i.e., all 384 filter weights are set to 1). In box 1310, a weighted k-NN calculation is used to calculate anomaly scores for all part images. In this example, all 56 part images, some of which are in defect data set D and some of which are in good data set G, are analyzed to determine anomaly scores using score function f. The first time box 1310 is executed, all filter weights in vector w have a value of 1. In subsequent executions of box 1310, all filter weights in vector w have different values. After execution of box 1310, all parts in the test data set have anomaly scores. This is the situation shown in FIG. 11 and discussed above.

[0094] In box 1312, a set of points P is determined. As discussed above and shown in FIG. 11, set P includes one or more good data points with the highest anomaly scores and one or more bad data points with the lowest anomaly scores. The particular data points included in set P may change with each iteration because every data point (part image) is assigned a new anomaly score each time the weight vector w is updated. The following is an example of a calculation that may be used in box 1312 to determine the points in set P:

[0095]

number

[0096] Here, equation (2) can be read as follows: find point x among the elements of defect data set D that has the minimum value of score function f, and find point x among the elements of good parts data set G that has the maximum value of score function f. In equation (2), the "points" are each identified part image (e.g., #1-56) defined by its extracted feature vector, and the score function f is evaluated from x's feature vector, the model core set M, and the weight vector w.

[0097] In box 1314, a gradient ascent calculation is performed to update the individual filter weight values ​​of the weight vector w. As known in the art, gradient ascent is an iterative technique that can be used to evaluate the influence of a set of input variables on the value of a function and maximize the function according to its gradient. In this case, the gradient ascent calculation performed in box 1314 is defined as follows:

[0098]

number

[0099] Here, equation (3) updates the weight vector w by adding a term that is the gradient ∇ of the function g evaluated at the point set P multiplied by a learning rate coefficient α. Recall that the value of g is the value of the anomaly score from f for the defective parts, and the value of g is the negative of the value of the anomaly score from f for the good parts. Thus, the value of the function g is maximized when the defective parts in P have high anomaly scores and the good parts in P have low anomaly scores. At each iteration, we establish a local value of the gradient ∇, and at the next iteration, we use the value of the gradient to calculate the next iteration of the weight vector w according to equation (3). As a result, the weight vector w is updated in the direction of the positive gradient, ultimately finding the optimal weight vector w that maximizes g.

[0100] At decision diamond 1316, it is determined whether to continue iterating. The iterations continue until the gradients converge to a predefined convergence criterion or a predefined maximum number of iterations is reached. The convergence criterion and the maximum number of iterations are provided among the input parameters in box 1306. If the iterations continue at decision diamond 1316, the process returns to box 1310, where the anomaly scores of all parts are recalculated using the updated weight vector w. A new set P is then determined, a new gradient ascent calculation is performed, and so on.

[0101] Once the iterations have ended at decision diamond 1316, an optimal weight vector w for separating and distinguishing between good and bad parts in the test data set has been found (or may be found by analyzing the data from all iterations). This means that the system 1200 of FIG. 12 is now trained to perform anomaly detection and produce the desired results. Thus, in box 1318 of FIG. 13, the weighted k-NN module 1240 is used with the final (optimal) weight vector w to perform anomaly detection in a "production" mode where input part images have not been classified into good and bad parts. As long as the input part images in the production mode are the same objects (such as the screw heads in FIG. 2) as the test images used for training, the system 1200 will accurately perform anomaly detection on the unclassified part images. The process ends at end point 1320.

[0102] The foregoing discussion of Figure 13 described numerous evaluations and calculations that are performed in the process of weighting the feature filters. With reference to the system block diagram of Figure 12, it will be understood that the various blocks and modules are all embodied in algorithms running on a processor (or multiple processors communicating with each other) such that the necessary data is available to each computational module when needed for feature extraction, k-NN scoring, weight vector optimization, etc. This is visually indicated by the arrows in Figure 12.

[0103] FIG. 14 is a pair of graphs of anomaly scores for multiple part images, including good and bad parts, illustrating the improvement in porosity resulting from the feature weighting method of FIG. 13, in accordance with an embodiment of the present disclosure. The graph 1400 on the left shows the initial anomaly scores for the 56 test part images discussed above. As previously described, the test part images are grouped into a set of bad (abnormal or defective) parts 1410 and a set of good parts 1420. The anomaly scores are obtained from actual experiments in initial conditions where the filter weights are both set to a value of 1, which corresponds to the first iteration of the flow chart diagram 1300 of FIG. 13. A threshold line 1402 can be drawn that separates the set of bad parts 1410 from the set of good parts 1420, but there is little separation between the sets.

[0104] The graph on the right 1430 shows the anomaly scores for the test part images when the features are weighted using the optimal weight vector w determined using the method of FIG. 13. A threshold line 1432 can be drawn that separates the set of bad parts 1410 from the set of good parts 1420, and it can be seen that the separation between the sets is much larger. The gap 1440 is defined as the difference in anomaly scores between the lowest scoring bad part and the highest scoring good part. The range 1450 is defined as the difference between the highest and lowest overall scores. The porosity is then defined as the gap 1440 divided by the range 1450.

[0105] In the first iteration (initially, all feature filter weights are set to 1), the experimental porosity was 3.99%. In the iteration with the optimal weight vector w, the porosity was 26.98%. This dramatic increase in porosity is visually confirmed by the difference in separation between graphs 1400 and 1430, clearly demonstrating the effectiveness of the disclosed feature weighting technique for k-NN based anomaly detection. In addition, the gradient of the function g over P (∇g| atP Note that an increase in σ has a mathematical meaning similar to an increase in porosity, as shown in FIG.

[0106] 15 is a graph 1500 of porosity between good and bad parts plotted against iteration number resulting from the feature weighting method of FIG. 13 in accordance with an embodiment of the present disclosure. Trace 1510 plots the porosity for each iteration. It can be seen that the porosity increases dramatically at first, peaks after about 70 iterations, and then decreases and meanders. This is the nature of gradient ascent methods, which will continue to explore within the space of possible states to see if a better optimum can be found.

[0107] A line 1520 is drawn at the iteration number at which a peak porosity was achieved. This peak porosity corresponds to graph 1430 in FIG. 14. As previously mentioned, iterations of the method of FIG. 13 may be allowed to continue for a sufficiently large number of iterations to identify a maximum porosity after all iterations (as shown in FIG. 15), or may be terminated when a convergence criterion is met. The convergence criterion may be defined in terms of improvement over a series of iterations to avoid premature termination at a local maximum such as seen near iteration number 10 in FIG. 15.

[0108] FIG. 16 is a graph 1600 of filter weights at the optimal iteration number of FIG. 15 resulting from the feature weighting method of FIG. 13 according to an embodiment of the present disclosure. FIG. 16 plots the individual filter weight values ​​for each of the 384 filter components in the weight vector w. Recall that all of the filter weights were initially set to a value of 1. After gradient ascent optimization of the weight vector w, it can be seen that most of the individual filter weights remain fairly close to 1, but some are significantly higher and some are significantly lower than 1. This distribution of filter weights illustrates the capabilities of the disclosed technique: using feature vectors extracted from part images of a particular object, a weighted k-NN anomaly scoring computation combined with a gradient ascent filter weight optimization computation provides a system trained to provide accurate anomaly detection results. Filter weighting is an effective means of separating good and bad part images of an object, and FIG. 16 illustrates that it would be impossible to effectively select the individual filter weight values ​​using manual methods. In addition, selecting filter weights one at a time is ineffective because hundreds of filter weights are interdependent when calculating feature vector distances.

[0109] Testing of an anomaly detection system trained using the feature filter weighting as described above produced desirable anomaly detection result characteristics in terms of recall (no missed anomalies) and precision (no incorrectly classified anomalies).

[0110] Various computers have been described and implied throughout the preceding discussion. It should be understood that such computer software applications and modules are executed on one or more computing devices having a processor and memory modules. In particular, this includes computers with processors configured with algorithms implementing the functions of the blocks of Figures 5 and 12, where the computer is in communication with an imaging system (e.g., a camera) that provides input images, and may communicate with other input / output devices as necessary to achieve a fully automated system.

[0111] The foregoing discussion discloses and describes merely exemplary embodiments of the present disclosure. Those skilled in the art will readily appreciate from such discussion and the accompanying drawings and claims that various changes, modifications, and variations can be made without departing from the spirit and scope of the present disclosure, as defined in the following claims.

Claims

1. 1. A method of feature weighting in anomaly detection, comprising: providing an input dataset comprising a plurality of labeled good part images and a plurality of labeled abnormal part images; providing a model core set comprising feature vectors of good part images; performing feature extraction on all of the images in the input dataset to generate feature vector data, the feature vector data including a first amount of feature vectors for each of the images, each feature vector including a second amount of filter component values; setting all components of a weight vector to a value of 1.0, said weight vector including a number of components equal to said second amount; and performing an iterative weight vector optimization calculation, the iterative weight vector optimization calculation comprising: calculating an anomaly score for each of the images using a weighted k-nearest neighbor (k-NN) algorithm running on a computer having a processor and a memory, the anomaly score including comparing the feature vector data of each of the images to the model core set in a weighted distance calculation using the weight vector; identifying a critical set comprising a subset of the good part images and a subset of the abnormal part images; calculating a score function value by subtracting the anomaly scores of the good part images in the critical set from the anomaly scores of the abnormal part images in the critical set; performing a gradient ascent calculation that adjusts the components of the weight vector to increase a value of a score function.

2. The method of claim 1 , wherein the iterative weight vector optimization calculation terminates when a predetermined maximum number of iterations is reached or a convergence criterion is met.

3. 3. The method of claim 2, further comprising: calculating anomaly scores for images of unknown good or abnormal classification manufactured parts using a weighted k-NN algorithm with a final set of component values ​​of the weight vector after the iterative weight vector optimization calculations are completed; accepting parts having anomaly scores below a threshold and rejecting parts having anomaly scores above the threshold.

4. The method of claim 1 , wherein the critical set includes one or more of the good part images having a highest anomaly score and one or more of the abnormal part images having a lowest anomaly score.

5. 2. The method of claim 1 , wherein performing feature extraction includes using a convolutional neural network having at least three layers, and combining filters output from two of the layers to generate a second quantity of filter component values ​​for each of the feature vectors.

6. 2. The method of claim 1 , wherein calculating the anomaly score for each of the images comprises calculating a weighted distance from each of the feature vectors in the feature vector data to the feature vectors in the model core set, the larger the weighted distance value, the higher the anomaly score.

7. 7. The method of claim 6, wherein the weighted distance is calculated with a vector distance calculation that multiplies, for each pair of feature vectors being measured, the difference between similarly indexed filter component values ​​by a correspondingly indexed component of the weight vector.

8. The method of claim 1 , wherein the gradient ascent calculation comprises determining a gradient of the score function with respect to changes in components of the weight vector.

9. 9. The method of claim 8, wherein the gradient ascent calculation adjusts the components of the weight vector by an amount equal to the gradient of the score function multiplied by a learning rate factor.

10. The method of claim 1 , wherein a good part image is an image of a part determined to have acceptable quality and a bad part image is an image of a part determined to have unacceptable quality.

11. The method of claim 1 , wherein the model core set is created by including feature vectors from images in a training dataset that includes a plurality of good part images.

12. 1. A method of feature weighting in anomaly detection, comprising: providing an input dataset comprising a plurality of labeled good part images and a plurality of labeled abnormal part images; providing a model core set comprising feature vectors of good part images; performing feature extraction on all of the images in the input dataset to generate feature vector data, the feature vector data including a first amount of feature vectors for each of the images, each feature vector including a second amount of filter component values; setting all components of a weight vector to a value of 1.0, said weight vector including a number of components equal to said second amount; and performing an iterative weight vector optimization calculation, the iterative weight vector optimization calculation comprising: calculating an anomaly score for each of the images using a weighted k-nearest neighbor (k-NN) algorithm running on a computer having a processor and a memory, the weight vector being used to calculate a weighted distance from each of the feature vectors in the feature vector data to the feature vectors in the model core set, the larger the weighted distance value, the higher the anomaly score; identifying a critical set comprising one or more of the good part images having the highest anomaly scores and one or more of the abnormal part images having the lowest anomaly scores; calculating a score function value by subtracting the anomaly scores of the good part images in the critical set from the anomaly scores of the abnormal part images in the critical set; performing a gradient ascent calculation to adjust the components of the weight vector to increase the value of the score function, the gradient ascent calculation comprising determining a gradient of the score function with respect to a change in the components of the weight vector and adjusting the components of the weight vector by an amount equal to a learning rate factor multiplied by the gradient of the score function.

13. 13. The method of claim 12, wherein the weighted distance is calculated with a vector distance calculation that multiplies, for each pair of feature vectors being measured, the difference between similarly indexed filter component values ​​by a correspondingly indexed component of the weight vector.

14. 1. An adaptive anomaly detection system, comprising: a computer configured with a processor and a memory; a feature extractor module that extracts feature vectors from images, including images in a training dataset that includes a plurality of good part images, the feature vectors from the training dataset being used to create a model core set; a weighted k-nearest neighbor (k-NN) algorithm; a weight vector optimization module, An input dataset is provided to the feature extractor module, the input dataset including a plurality of labeled good part images and a plurality of labeled abnormal part images; the feature extractor module performs feature extraction on all of the images in the input dataset to generate feature vector data, the feature vector data including a first amount of feature vectors for each of the images, each feature vector including a second amount of filter component values; performing an iterative weight vector optimization calculation after setting all components of a weight vector having a number of components equal to the second amount to a value of 1.0, the iterative weight vector optimization calculation comprising: calculating an anomaly score for each of the images using the weighted k-NN algorithm, the anomaly score including comparing the feature vector data of each of the images to the model core set in a weighted distance calculation using the weight vector; identifying a critical set using the optimization module, the critical set including a subset of the good part images and a subset of the abnormal part images; calculating, using the optimization module, a score function value by subtracting the anomaly scores of the good part images in the critical set from the anomaly scores of the abnormal part images in the critical set; and performing a gradient ascent calculation using the optimization module to adjust the components of the weight vector to increase the value of the score function.

15. 15. The system of claim 14, wherein the iterative weight vector optimization calculation is terminated when a predetermined maximum number of iterations is reached or a convergence criterion is met, and wherein the weighted k-NN algorithm is used with a final set of component values ​​of the weight vector after the iterative weight vector optimization calculation is terminated to calculate anomaly scores for images of production parts of unknown good or abnormal classification, and to accept parts with anomaly scores below a threshold and reject parts with anomaly scores above the threshold.

16. 15. The system of claim 14, wherein a critical set includes one or more of the good part images having the highest anomaly scores and one or more of the abnormal part images having the lowest anomaly scores.

17. 15. The system of claim 14, wherein the feature extractor module performs the feature extraction using a convolutional neural network having at least three layers and combines filters output from two of the layers to generate the second quantity of filter component values ​​for each of the feature vectors.

18. 15. The system of claim 14, wherein calculating the anomaly score for each of the images comprises calculating a weighted distance from each of the feature vectors in the feature vector data to the feature vectors in the model core set, where larger values ​​of the weighted distance indicate higher anomaly scores, the weighted distance being calculated with a vector distance calculation that multiplies, for each pair of measured feature vectors, the difference between similarly indexed filter component values ​​by a correspondingly indexed component of the weight vector.

19. 15. The system of claim 14, wherein the gradient ascent calculation includes determining a gradient of the score function with respect to changes in components of the weight vector, the gradient ascent calculation adjusting the components of the weight vector by an amount equal to the gradient of the score function multiplied by a learning rate factor.

20. 15. The system of claim 14, wherein a good part image is an image of a part determined to have acceptable quality and a bad part image is an image of a part determined to have unacceptable quality.

Images