Threshold and model adjustment for KNN-based anomaly detection methods

Abstract

A method for anomaly detection from an object image. The feature extractor provides feature data characterizing a component image including a good component and a bad component. A set of k-nearest neighbor (k-NN) model kernels is created using the training data of the good component. The adjustment data for some good and some bad components is evaluated by the k-NN module to determine an anomaly score from the feature data. A good-to-bad threshold of the anomaly score is initially set to a low value. When an anomaly score is detected that exceeds a threshold, the human examines the corresponding image to determine whether the subject component is good or bad. If the component is good and the anomaly is false, the k-NN model kernel set is updated to include the feature data of the subject component, and the threshold is adjusted higher. After each adjustment, the k-NN model is applied to the test data set to evaluate performance.

Description

Technical Field

[0001] The present disclosure relates generally to adaptive methods for anomaly detection, and more particularly to methods for anomaly detection using a feature extractor in combination with a k-nearest neighbor (k-NN) scoring technique, wherein a model of good object data and a threshold separating good object scores from anomaly scores are updated during an adaptation phase to improve both the number of missed anomaly detections and the number of false anomaly alarms. Background Art

[0002] In many industries, parts or other objects must be inspected, including checking for proper fit and / or finish, surface quality, etc. In recent years, these types of inspections are often performed by computer-based image analysis systems rather than manually by human inspectors. In a typical computer-based image analysis application, images of objects are provided to a computer running an algorithm that performs anomaly detection on each image of the object.

[0003] Anomaly detection systems typically include a feature extractor, such as a machine learning system, which takes an image as input and provides a set of feature vectors that characterize the image as output. Feature extractors significantly reduce the amount of data required for anomaly detection by replacing images with millions of pixels with a set of feature vectors that can be defined in hundreds of numbers.

[0004] After feature extraction, a separate algorithm or calculation is used to evaluate the feature vector data in order to provide an anomaly score. An effective technique for determining anomaly scores based on feature data is called k-nearest neighbor (k-NN). The k-NN technique is a supervised learning method in which attribute values ​​are determined based on the "k" closest training examples in a data set. When applied to anomaly detection, the data set contains feature vector data for the training samples, and the attribute value provided by k-NN is the anomaly score.

[0005] The fundamental challenge in anomaly detection is the data imbalance between input data representing "good" objects and input data representing "bad" objects. That is, the number of good objects used to train the feature extractor often far exceeds the number of bad objects. This can make it difficult for the feature extractor to build a model that accurately distinguishes the characteristics of good and bad objects.

[0006] Techniques that attempt to improve the effectiveness of anomaly detection systems are known in the art. These techniques range from simple adjustment of the threshold between good and bad scores to tuning of neural network classifiers to one-at-a-time filter weighting in feature vector calculations. However, none of these prior art techniques have proven to be flexible in terms of adjustment and effective in improving anomaly detection results.

[0007] In view of the above, improved methods for adaptive and learning image-based anomaly detection are needed. Summary of the invention

[0008] The following disclosure describes a method and system for anomaly detection from an image of an object. A feature extractor neural network is used to provide feature data characterizing a plurality of images, the plurality of images including images of a large number of good parts and a small number of bad parts. A k-nearest neighbor (k-NN) model kernel data set is created using training data including feature data only for good parts. Adaptation data including feature data for some good parts and some bad parts is then evaluated by the k-NN module to determine anomaly scores from the feature data for each image. A good-to-bad threshold for the anomaly score is initially set to a low value. When an anomaly score exceeding the threshold is detected, a human inspector evaluates the corresponding image to determine whether the subject part is good or bad. If the part is good and the anomaly is a false alarm, the k-NN model data set is updated to include feature data for the subject part, and the threshold is incrementally adjusted higher. After each adaptation, the k-NN model is applied to all parts in the test data set to evaluate performance.

[0009] Additional features of the present disclosure will become apparent from the following description and appended claims, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram illustration showing the basic architecture of an anomaly detection system as known in the art;

[0011] Figure 2 is a diagram of a part to be visually inspected, including a defect on one of the parts that must be identified using anomaly detection methods and systems;

[0012] Figure 3 is a block diagram illustration of an anomaly detection system having an offline learning portion and an online testing portion, wherein the online testing portion includes a k-nearest neighbor (k-NN) module for determining an anomaly score based on feature data, as known in the art;

[0013] Figure 4A and 4B are diagrams of feature data clusters respectively depicting concepts of threshold and model adjustment in k-NN based anomaly detection according to an embodiment of the present disclosure;

[0014] Figure 5 is a block diagram illustration of a k-NN based anomaly detection system including threshold and model adjustment during online test data processing according to an embodiment of the present disclosure;

[0015] Figure 6is a flowchart of a method for threshold and model adjustment in a k-NN based anomaly detection system according to an embodiment of the present disclosure;

[0016] Figure 7 is a flow chart of a method for threshold and model adjustment during k-NN based anomaly detection according to an embodiment of the present disclosure, including calculations for updating a k-NN model kernel set and calculating a new threshold;

[0017] Figure 8 is a diagram of a k-NN model kernel set and false anomaly data points according to an embodiment of the present disclosure, which depicts a method for Figure 7 The calculation of the updated kernel set in the method, and the updated kernel set;

[0018] Fig. 9 is a series of graphs of component anomaly scores from feature data according to an embodiment of the present disclosure, which depicts the use of Figure 6 and Figure 7 The impact of threshold and model tuning in k-NN based anomaly detection of our approach, where both false anomaly and missed detection performance are improved;

[0019] Fig.10 is a schematic block diagram illustration of a convolutional neural network (CNN-based) based feature extractor system as known in the art;

[0020] Fig.11 is a graph of anomaly scores for a plurality of part images including good parts and bad parts illustrating selection of a set of points used in performing a gradient ascent calculation for feature weighting according to an embodiment of the present disclosure;

[0021] Fig.12 is a block diagram illustration of a k-NN based anomaly detection system including feature filter weighting during online test data processing according to an embodiment of the present disclosure;

[0022] Fig.13 is a flowchart of a method for feature weighting in a k-NN based anomaly detection system according to an embodiment of the present disclosure;

[0023] Fig.14 is a pair of graphs of anomaly scores of a plurality of component images including good components and bad components according to an embodiment of the present disclosure, which shows that Fig.13 The gap ratio improvement obtained by the feature weighting method;

[0024] Fig.15 According to an embodiment of the present disclosure, Fig.13 a graph of the gap ratio between good parts and bad parts plotted against the number of iterations obtained using the feature weighting method of ; and

[0025] Fig.16 According to an embodiment of the present disclosure, Fig.13 The feature weighting method is obtained from Fig.15 Plot of the filter weights at the optimal number of iterations. DETAILED DESCRIPTION

[0026] The following discussion of embodiments of the present disclosure relates to methods for threshold and model adjustment in k-NN based anomaly detection, which is merely exemplary in nature and is in no way intended to limit the disclosed technology or its applications or uses.

[0027] Figure 1 1 is a block diagram illustration showing the basic architecture of an anomaly detection system 100 as known in the art. At box 110, an input is provided. The input at box 110 is typically a visual input, such as an image of a component or workpiece. In some applications, the input is a graphical or data input, such as data from an accelerometer or other sensor that characterizes the operation of the device. In any case, the input at box 110 is used to determine whether the item (component or workpiece or device) being analyzed is normal (also referred to as "good", "normal", "nominal") or abnormal ("bad", "defective").

[0028] Input from box 110 is provided to an algorithm 120 which determines an anomaly score 130. Based on the anomaly score 130, the item being analyzed is classified as normal / good or abnormal / bad.

[0029] Figure 1 It is only to illustrate the basic concepts and construction framework of the anomaly detection system in order to prepare for the further discussion below.

[0030] Figure 2 is a graphic representation of a part to be visually inspected, including defects on a part that must be identified using anomaly detection methods and systems. Figure 2 In an example of , an anomaly detection system is used to classify a screw head as good or bad based on an image of the screw head. The screw head may have just been manufactured (e.g., on a head machine), or the screw may have just been installed into an assembly. In yet another case, the screw is part of an assembled product and an electrical connectivity or performance test is performed by contacting an external probe with the screw head.

[0031] exist Figure 2The screw head 200 at the left side of the image is a normal (good) part with no defects that would cause it to be rejected as an anomaly. However, the image depicting the screw head 200 includes several characteristic parts, which makes the task of anomaly detection difficult. For example, shadows 202 and reflections 204 are common in the image due to imperfect lighting conditions, surrounding objects, camera angles, etc. Since objects such as screw heads 200 have shiny, uneven surfaces, it is very difficult to provide an image that is completely free of shadows and reflections.

[0032] The screw head 200 also includes a recess 206, which in this case is shaped to accommodate a Phillips screwdriver bit. The recess 206 is clearly a necessary shape feature of the screw head 200, rather than a defect or anomaly. However, the many surfaces and shades of the recess 206 and other characteristics add a lot of complexity to the evaluation of the image of the screw head 200.

[0033] exist Figure 2 The screw head 220 at the right side of is an abnormal (bad) part, having several defects that would cause it to be rejected as an anomaly. As with the image of screw head 200, the image of screw head 220 includes shadows and reflections, as well as the same grooves 206. The image of screw head 220 is rotated approximately 45° relative to the image of screw head 200, which illustrates another difficulty associated with visual anomaly detection. That is, shadows and reflections, as well as other shape features, may appear in different positions and orientations from one part to the next, making a simple comparison of a new image to an image of a known good part insufficient to reliably detect anomalies.

[0034] Screw head 220 includes flaws 222, 224, and 226. Flaws 222-226 may be of one or more types, including scratches, flat spots, or other defects from screw head machinery, or burn marks caused by arcing when an external electrical probe contacts screw head 220. Any one of flaws 222 and 226 may be sufficient to cause screw head 220 to be classified as abnormal. Three different flaws are included to illustrate that the flaws may appear on different parts of screw head 220, and in different locations relative to shadows and reflections. Flaws 222-226 themselves may also have different shapes, different lightness / darkness, and reflectivity characteristics.

[0035] include Figure 2 and the previous discussion to provide further context for the discussion of anomaly detection. In addition, the techniques of the present invention have been applied to similar Figure 2 Screw head anomaly detection of the type shown. The effects of the disclosed method and system will be discussed later in this disclosure.

[0036] Due to the complexity encountered in real-world anomaly detection, as mentioned above, various techniques for anomaly detection with self-learning capabilities and advanced anomaly score analysis have been developed.

[0037] Figure 3 3 is a block diagram illustration of an anomaly detection system 300 having an offline learning portion and an online testing portion as is known in the art. The system 300 includes an offline learning portion 302 and an online testing portion 304. The offline learning portion 302 is used to pre-train a database to identify features of good parts, while the online testing portion 304 evaluates parts of unknown classification and includes a k-nearest neighbor (k-NN) module for determining anomaly scores based on feature data.

[0038] In the offline learning portion 302, training data 310 is provided, wherein the training data 310 includes images of parts of known classification (good or abnormal). Each image in the training data 310 can be of any of the above types, or other types, such as images of screw heads, images of painted surfaces to determine the quality of painted surfaces, etc. Each image in the training data 310 is provided to a feature extractor module 320, which can be a convolutional neural network (CNN) that extracts feature vectors from the image, as known in the art. Features 330 are used to construct a memory bank 340 that distinguishes good parts from abnormal parts because features 330 are identified as extracted from good parts or abnormal parts in the training data 310. In some applications, only good parts are provided in the training data 310.

[0039] In the online testing portion 304, the input 350 includes multiple images from components of typically unknown classification (good or abnormal). Each image from the input 350 is provided to a feature extractor module 360, which can be the same CNN-based feature extractor module 320 as in the offline learning portion 302. Features 370 are provided for each image from the feature extractor module 360. Since the input 350 is different from the training data 310, the * in the element 370 simply indicates that the feature 370 is different from the feature 330. The features 370 are provided to the k-NN module 380, which determines an anomaly score based on the features 370 of each individual component. The k-NN module 380 compares the features 370 of each component to the memory bank 340 pre-trained in the offline learning portion 302. The anomaly score for each component is determined by the k-NN module 380 based on the "distance" (in feature vector space) between the features of the component and the features of the kernel set of good components in the memory bank 340. Components with features that closely match the kernel set (i.e., very small distance) will have an anomaly score close to zero. Parts with features that do not closely match the core set (ie, large distance) will have a high anomaly score, eg, greater than one.

[0040] A final part classification is provided at box 390. The final classification is typically -1 (for a good part) or 1 (for a bad or abnormal part). The criteria for assigning the final classification value based on the anomaly score, typically by comparison to a threshold, varies by application.

[0041] Even with its relatively sophisticated feature extractor module and k-NN based anomaly score determination, system 300 still has difficulty distinguishing good parts from anomalous parts in all cases. In most anomaly detection applications, there are many obviously good parts with low anomaly scores, and a few obviously bad parts with high anomaly scores. However, there are often some parts that receive anomaly scores (e.g., 0.95 or 1.02) close to a cutoff or threshold (e.g., 1.0), where it is unclear whether these parts are good or bad.

[0042] In such Figure 3 The difficulty in reliably detecting anomalies in existing systems 300 is due in part to Figure 2 The complexity of the real world shown in and discussed above. This is also due to the fact that the distribution of good data in the input 350 (during online testing) is sometimes different from the distribution of good data in the training data 310 (during offline learning). For example, some features of the good data in the online testing phase may never appear in the offline training phase. Therefore, the pre-training that occurs in the offline learning portion 302 is inevitably incomplete.

[0043] The disclosed techniques have been developed to overcome limitations of existing anomaly detection methods and systems, including those discussed above. Figure 3 Those limitations of the system 300.

[0044] The first technique to overcome the limitations of existing anomaly detection methods and systems is threshold and model adjustment.Threshold and model adjustment directly addresses the problem of system 300 discussed above, namely that the training data cannot fully and accurately represent the complete feature set of good parts.

[0045] Figure 4A and 4B is a diagram of feature data clusters respectively depicting the concepts of threshold and model adjustment in k-NN based anomaly detection according to an embodiment of the present disclosure. Figure 4A 4, a plurality of feature data points are depicted in positions according to anomaly scores plotted on a vertical axis 402. Ellipse 410 contains feature data points of items (parts or input images) that have been determined to be good (e.g., by manual inspection). Ellipse 420 contains feature data points of items (parts or input images) that have been determined to be bad (anomalies or defects).

[0046] Threshold line 430 is an initial threshold for distinguishing good and bad parts in the k-NN anomaly score evaluator. That is, points with anomaly scores exceeding the threshold are classified as bad (abnormal), and vice versa. Using threshold line 430, feature point 412 is incorrectly classified as an abnormality because it is known that the point belongs to a good part. This situation is called a false anomaly classification. In addition, using threshold line 430, when feature point 422 is known to belong to a bad / abnormal part, the point is incorrectly classified as a good part. This is called a missed anomaly detection. Although both false anomalies and missed detections are undesirable, it is generally established as a standard for anomaly detection systems that no missed anomaly detections are acceptable (i.e., no bad parts are allowed to pass undetected).

[0047] To eliminate missed detection of point 422, a new lower threshold line 432 is established. Threshold line 432 effectively eliminates Figure 4A All missed detections in the data set are shown, but the lower threshold exacerbates the problem of false anomaly classifications because another characteristic data point from a good part is now above the threshold line 432. Figure 4A An important lesson about k-NN based anomaly detection is illustrated; namely, threshold adjustment alone cannot overcome the related problems of false anomaly classification and missed detection of anomalies.

[0048] exist Figure 4B , a model 450 (on the left) depicts a plurality of feature data points of good parts, and a feature data point 460 of a part classified as an anomaly. Boundary line 452 defines the boundary of a core set of feature points around good parts (the feature points of good parts are within boundary line 452, which is to the right of line 452 in the figure, and only a portion of model 450 is shown). Although feature data point 460 is classified as an anomaly, it is close to the core set of good parts in model 450.

[0049] If the part or image associated with the feature data point 460 is determined (e.g., by manual inspection) to be a good part rather than an anomaly, adjustment of the model 450 may be required. At the right, the model 470 is bounded by a boundary line 472. The model 470 is an adjustment of the model 450 that includes the feature point 460 in a kernel set of good parts. This adjustment is illustrated here as simply redrawing the boundary line; however, a technique is described below for continuously and incrementally adjusting the feature points included in the model kernel set, including eliminating feature points from the kernel set as new points are added. In the technique described below, model adjustment and threshold adjustment are combined in a manner that significantly improves anomaly detection performance.

[0050] To simplify the concepts discussed, Figure 4A and 4B(and subsequent figures) are drawn in two dimensions. As those familiar with feature extraction will appreciate, each feature "point" is actually a feature vector with dimensions that may be several hundred, and the calculations of the positions and distances between points are actually performed in vector space rather than in two dimensions.

[0051] The threshold and model adjustment techniques of the present disclosure use manual inspection in combination with specially designed computational logic to incrementally adjust the k-NN model kernel set and thresholds in anomaly detection systems. These techniques will be discussed in detail below.

[0052] Figure 5 is a block diagram illustration of a k-NN based anomaly detection system 500 including threshold and model adjustment during online test data processing according to an embodiment of the present disclosure. The system 500 has Figure 3 The system 500 has a substantially similar architecture to the system 300, including an offline learning portion and an online testing portion. The system 500 adds threshold and model adjustment calculations coupled with the k-NN anomaly score determination, as described below.

[0053] Offline learning part 302A and Figure 3 The offline learning portion 302 of the system 500 is substantially the same, where the training data 310A is provided to the feature extractor module 320, and the features 330 are included in the memory bank 340A. In a preferred embodiment, the training data 310A of the system 500 includes images of only good parts, and not bad / anomalous parts. This is because the goal is to initially create the memory bank 340A with a core set of known good feature data, establish a low initial threshold, and adjust the memory bank 340A and the threshold during online test data processing.

[0054] In the online testing portion 504, the input 510 includes multiple images of parts from an unknown classification (i.e., it is not known whether each part is good or abnormal). Each image from the input 510 is provided one at a time to a feature extractor module 520, which is preferably the same type of CNN-based feature extractor module as previously discussed. Features 530 are provided for each image from the feature extractor module 520. Again, the features 530 characterize the parts of the unknown classification, including good parts and bad / abnormal parts. The features 530 are provided to the k-NN module 540, which determines an anomaly score based on the features of each individual part.

[0055] The k-NN module 540 compares the features 530 of each component to the memory bank 340A pre-trained in the offline learning portion 302A. The anomaly score of each component is determined by the k-NN module 540 based on the "distance" (in feature vector space) between the features of the component and the features of the nearest neighbors in the core set of good components in the memory bank 340A. Components with features that closely match the core set (i.e., very small distances) will have low anomaly scores. Components with features that do not closely match the core set (i.e., large distances) will have high anomaly scores, such as greater than one.

[0056] A threshold and model adjustment module 550 is added to the system 500. After the anomaly score for each image is provided by the k-NN module 540, the threshold and model adjustment module 550 performs a set of evaluations and calculations. The functions performed by the threshold and model adjustment module 550 include determining whether an item with an anomaly score exceeding a threshold is a false anomaly, updating the kernel set of good part features in the memory bank 340A, updating the thresholds used for the final part classification, and updating the anomaly library 560 used in the calculations. The dotted lines to the memory bank 340A and the anomaly library 560 indicate the adjustments and updates performed on these elements, while the value of the threshold is maintained within the threshold and model adjustment module 550 itself. The details of these evaluations and calculations will be discussed below in conjunction with Figure 6 and 7 Have a discussion.

[0057] A final part classification is provided at box 570. The final classification is either -1 (for a good part) or 1 (for a bad or abnormal part). The final classification value is based on the anomaly score compared to a threshold, where both the set of model kernels used by the k-NN module 540 to determine the score and the threshold used to perform the final classification evolve as the threshold and model adjustment module 550 processes more images.

[0058] In a preferred embodiment of the system 500, a relatively large number of images of good parts and a relatively small number of bad parts are used, such as hundreds of good part images and dozens of bad part images. In a non-limiting embodiment, one third of the good part images (and no bad part images) are used for training data 310A in the offline learning portion 302A, one third of the good part images and half of the bad part images are used for adjustment in the input 510 of the online testing portion 504, and the last one third of the good part images and the other half of the bad part images are used for testing in the input 510 of the online testing portion 504. In this embodiment, after each adjustment step (processing one image and adjusting the threshold and model), the entire set of test images is processed to evaluate the performance of the system 500.

[0059] Figure 6600 is a flow chart of a method for threshold and model tuning in a k-NN based anomaly detection system according to an embodiment of the present disclosure. After the start ellipse 602, at box 604, a threshold α is initialized. The threshold α is initialized after performing online learning using only good component data. That is, before tuning begins using the online testing portion 504, the threshold α is initialized to a relatively low value (lower than 1.0). The idea is to initialize the threshold α to a value that is high enough to allow most good components to be classified as good, but low enough to ensure that all bad components are classified as anomalies (no missed anomaly detections).

[0060] At block 606, new feature data (for a single image) is provided; this is Figure 5 The data in features 530 is provided to the k-NN module 540, which provides an anomaly score based on a comparison of these features with a set of model kernels from memory bank 340A at box 608. At decision diamond 610, a determination is made as to whether the anomaly score is greater than a threshold value α. This determination and Figure 6 The remainder of the steps are in Figure 5 The threshold and model adjustment module 550 is executed. If the anomaly score is not greater than the threshold α, the memory bank 340A is updated at box 612, which includes adding the features of the current (good) image to the model kernel set if certain criteria are met. The criteria include determining the distance from each feature point of the current feature set to the nearest feature point in the kernel set, and if the distance is greater than the minimum point-to-point distance of the existing feature points in the kernel set, replacing the existing feature points with the feature points of the current feature set. This will be discussed further below and in Figure 8 Shown in.

[0061] From decision diamond 610, if the anomaly score is greater than threshold α, indicating an anomaly, then at box 618, a human inspects the part or image corresponding to the current feature data. The human inspector classifies the part image as good or bad (anomaly). If the human inspector classifies the part image as good, then at decision diamond 620, the image feature data is determined to be a false anomaly.

[0062] In the case of a false anomaly, the process moves to block 622 where the kernel set (of good part feature data) is adjusted in memory bank 340A. This includes evaluating the current feature data known from manual inspection to belong to a good part image to determine whether to add the current feature data to the kernel set. This evaluation will refer to Figure 7 and 8 After the model adjustment at block 622, at block 624, the threshold α is adjusted to an incrementally higher value. Figure 7 Threshold adjustment is also described in detail in the discussion of .

[0063] If the human inspector determines that the part / image is actually an anomaly, this means that it is not a false anomaly, and from decision diamond 620, the process moves to box 626 where the anomaly library 560 is updated to include the current feature data. The relevance of the anomaly library 560 in adaptively updating the threshold α is discussed below in conjunction with Figure 7 discuss.

[0064] After the model and threshold adjustments at blocks 622 and 624 or the anomaly library update at block 626, the process returns to block 606 where new feature data (for the next part image in input 510) is provided. At decision diamond 614, when there are no more parts to analyze, the process ends at endpoint 616.

[0065] It is mentioned above that a set of adjustment data images are provided to the online testing section 504, and after processing each adjustment data image, the entire set of test data images is processed to determine the system anomaly detection performance. Figure 6 The steps are performed only for the adjustment data image and not for the test data image. Therefore, Figure 5 The system 500 is preferably configured with a "switch" that indicates whether the system is operating in adjustment mode or test mode. When the system is in adjustment mode, execution Figure 6 (and Figure 7 ) threshold and model adjustment steps. When the system is in test mode, module 550 simply classifies each part image based on a comparison of its anomaly score with the current value of the threshold α.

[0066] Figure 7 700 is a flowchart of a method for threshold and model adjustment during k-NN based anomaly detection according to an embodiment of the present disclosure, including calculations for updating a k-NN model kernel set and calculating a new threshold. Figure 7 and the accompanying discussion provides information included in Figure 6 Details of the threshold and model tuning calculation steps in the Methods.

[0067] When Figure 6 When a false anomaly is detected at the decision diamond 620 of , the process starts at the start ellipse 702. At box 704, data from the storage body 340A (the model core set M containing good component feature data) and the anomaly library 560 (containing abnormal / bad component feature data) are provided. Also at box 704, a minimum distance vector d is provided, which contains the minimum distance between each feature vector in the model core set M. In addition, an incremental step size l is provided at box 704. r and a discount factor γ, where these parameters are used in the threshold update calculation discussed below.

[0068] At box 706, feature data for false anomaly parts is provided. This is a part / image that has an anomaly score that exceeds the threshold at decision diamond 610 and is determined by a human inspector to be a good part, not an anomaly. The calculations being described here are Figure 5 The threshold and model adjustment module 550 is performed. In one embodiment, the feature data includes a set of 784 (28×28 grid) feature vectors, each feature vector having 384 dimensions. As will be appreciated by those skilled in the art, the exact composition of the feature data depends on the design of the feature extractor module 520.

[0069] After the offline learning phase (see Figure 5 ), the memory bank 340A includes a model kernel set M containing feature vectors of only good parts. The model kernel set M may initially contain 784 feature vectors for each of dozens (or hundreds) of part images, in which case the number of feature vectors may be hundreds of thousands. To reduce the inference time in the online testing phase, the model kernel set M may be downsampled to include an order of magnitude of 10 5 Instead of having multiple eigenvectors of the order of 10 6 The downsampling may be performed in any suitable manner.

[0070] What happens next is that a determination is made whether to include one or more feature vectors from the current (false anomaly) component in the model kernel set M. The idea behind this is that the current false anomaly component has feature data that makes its anomaly score above a threshold, yet it is a good component as determined by manual inspection. Therefore, there may be features of the current false anomaly component image that can be included in the model kernel set M in order to make the kernel set more robust to different image characteristics (recall that Figure 2 In a preferred embodiment, when the feature vector of the current false abnormal component is included in the model kernel set M, the feature vector is also removed from the model kernel set M so as to maintain a constant kernel set size.

[0071] At box 708, for each of the feature vectors (e.g., all 784) of the current false anomaly component, the distance from the feature vector to the nearest feature vector in the model kernel set M is calculated. At decision diamond 710, it is then determined whether the distance calculated at box 708 is greater than the minimum distance (provided in d) between the feature vectors in the model kernel set M. If the answer is no, the process proceeds to decision diamond 714, where it loops back to box 708 to calculate the distance for the next feature vector.

[0072] If at decision diamond 710, the distance calculated at box 708 is greater than the minimum distance between the feature vectors in the model core set M, then at box 712, one of the two feature vectors involved in the minimum distance is removed from the model core set M and replaced with the feature vector currently being calculated. The model core set M and the minimum distance vector d are then updated in the memory bank 340A.

[0073] Figure 8 is a diagram of a k-NN model kernel set M and a false anomaly data point 820 according to an embodiment of the present disclosure, which visually depicts the Figure 7 The calculation of the kernel set M is updated in the method of . The kernel set M shown at 810A includes feature vectors ("points") 812, 814, and 816, etc. As described above, the distance 830 between the false anomaly feature vector (point) 820 and the nearest feature vector in the kernel set M (which is feature vector 816) is calculated. Distance 832 is the minimum distance between any pair of feature vectors in the kernel set M. All distances are calculated in feature vector space. Because distance 830 is greater than distance 832, Figure 7 The logical command of eigenvector points 812 / 814 is removed from the kernel set M and replaced with eigenvector 820.

[0074] exist Figure 8 At the bottom of FIG. 8 , an updated kernel set 810B is shown, in which feature vector point 814 has been eliminated from kernel set M, and feature vector 820 has been included in kernel set M. Again, the purpose of this model kernel set adjustment is to increase the diversity of feature vectors (of known good parts) included in kernel set M, which together with the later threshold adjustment reduces the number of false anomaly classifications.

[0075] In the manner described above, all feature vectors of the current false anomaly component are evaluated, and some feature vectors may be included in the model kernel set M as replacements for closely spaced kernel set feature vectors. Figure 7 The method continues until at decision diamond 714 the feature vector of the current false anomaly component no longer exists.

[0076] The process then moves to block 716 to begin the steps involved in threshold adjustment based on the evaluation of the current false anomaly component. Figure 7 The steps of blocks 716-720 in Figure 6 At block 716, a modified anomaly score s′ is calculated in the k-NN module 540 based on the feature vector of the current false anomaly component and the modified model kernel set M from the storage bank 340A. At block 718, an upper limit score s is calculated in the k-NN module 540 based on all anomaly data in the anomaly library 560 compared with the modified model kernel set M from the storage bank 340A.u .

[0077] At block 720, a new value of threshold α is calculated that is incrementally higher than the previously existing value of threshold α. The new value of threshold α is calculated at block 720 as follows: α=min (max(s′,α)+γ·l r ,s u ) (1) where the first term is determined by selecting the higher of the modified anomaly score s′ and the old value of the threshold a, and is added to the increment step size l r The second term is calculated by multiplying by the discount factor γ. The sum of the terms and the upper limit score s u The lower of α is selected as the new value of threshold a. The logic embodied in formula (1) is that the current false anomaly part is actually a good part, so if the revised anomaly score s′ is greater than the old threshold, then the revised anomaly score s′ is acceptable as a starting point for the new threshold. Regardless of whether s′ or α is selected as the starting point, the term γ·l r The increment is added to the calculation of the new threshold, and the new threshold is capped by the fraction s. u Limit.

[0078] After the threshold adjustment calculation at block 720, Figure 7 The method ends at terminal 722. This corresponds to Figure 6 The method completes block 624 and returns to process new data (data from another image of input 510). Whenever a false exception is encountered, execute Figure 6 The steps in elements 618-624 and Figure 7 The entire threshold and model adjustment process.

[0079] Front Figure 6-7 The discussion describes the multiple evaluations and calculations performed during the threshold and model tuning process. Figure 5 It should be understood that the various boxes and modules are embodied in the algorithm running on the processor (or multiple processors communicating with each other), so that the necessary data can be used for each of the various computing modules required for feature extraction, k-NN scoring, threshold and model adjustment, storage 340A and anomaly library 580 update. Figure 5 It is described intuitively using arrows.

[0080] Fig. 9 is a series of graphs of component anomaly scores from feature data according to an embodiment of the present disclosure, which depicts the use of Figure 6 and Figure 7 The effects of threshold and model tuning in k-NN based anomaly detection are investigated, where both false anomaly and missed detection performance are improved.

[0081] A collection of feature vector points is shown in the initial scene at 900 (on the left), where the individual points are placed according to their anomaly scores plotted on the vertical axis. The feature vector points shown at 900 describe the results of sending multiple test data images through the system 500 after only offline learning from part 302A has been performed and no adjustments have occurred. Set 910 contains feature data points of items (parts or input images) that have been determined to be good (e.g., by manual inspection). Set 920 contains feature data points of items (parts or input images) that have been determined to be bad (abnormal). Threshold line 902 has been set to a low value to prevent any missed anomaly detections. At a low threshold and without model adjustments, a large number of false anomalies are encountered, as shown in box 904.

[0082] After a certain number of adjusted data images have been processed in system 500, some adjustments to the thresholds and models occur, as shown in the scenario at 930. In scenario 930, set 940 contains feature data points for good items from set 910, where the data points have generally moved in the direction of lower anomaly scores due to the model adjustments. Set 950 contains the same feature data points for bad items as in set 920. Also in scenario 930, threshold line 932 has moved incrementally upward as a result of the threshold adjustments. With a slightly higher threshold and some model adjustments, a smaller number of false anomalies are encountered, as shown in box 934. Still, no anomaly detections are missed in scenario 930 because the threshold adjustment includes calculations as described above to limit the upward movement of the threshold in a way that prevents missed detections.

[0083] After an additional number of adjusted data images are processed in system 500, the adjustment of the threshold and model has reached a final or near-final condition, as shown in the scene at 960. In scene 960, set 970 contains the same characteristic data points of good items from sets 910 and 940, where the data points should be further moved in the direction of lower anomaly scores due to model adjustment. Set 980 contains the same characteristic data points of bad items as in sets 920 and 950. Also in scene 960, threshold line 962 has been moved further upward as a result of the threshold adjustment. With the higher threshold and additional model adjustment, no false anomalies are encountered in scene 960. In addition, for the reasons described above, no anomaly detections are still missed in scene 960. It can be appreciated that as the threshold and model adjustment work, the amount of manual inspection required will drop dramatically, and eventually, all or nearly all of the parts classified as anomalies will actually be true anomalies.

[0084] The above threshold and model adjustment techniques have been shown experimentally to provide separation between sets of good and bad data points in an anomaly detection system while setting the thresholds to optimal values. These techniques provide an ideal combination of anomaly detection results (zero missed anomaly detections) and zero or near-zero false anomaly classifications. This ideal combination of results is not possible with previously existing anomaly detection methods.

[0085] Another currently disclosed technique for overcoming the limitations of existing anomaly detection methods and systems is feature weighting. Feature weighting involves assigning weights to filters used to calculate distances between feature vectors when calculating anomaly scores, and optimizing the weights to achieve the best separation between good data and bad data (e.g., component images). Details of feature weighting techniques are discussed below.

[0086] Fig.10 is a schematic block diagram illustration of a convolutional neural network (CNN) based feature extractor system as known in the art. The feature extractor module has been repeatedly discussed in the previous part of this disclosure, including the system for using Figure 5 Thresholds and model tuning for anomaly detection. Fig.10 It serves as a basis for discussing some details of the feature extractor, which are important in the feature weighting techniques described below.

[0087] As shown in the previous figures and discussed above, the input image 1010 is provided to the feature extractor module 1020. As will be appreciated by those skilled in the art, a convolutional neural network (CNN) is a common architecture for a feature extractor module, and in fact, a pre-trained feature extractor CNN is available where the network structure is fixed, including the number of layers and the size of each layer output, and the network parameters are trained. The feature extractor module 1020 is an example of such a pre-trained feature extractor CNN. Other examples of feature extractor architectures with different numbers of layers and feature vectors may also be used in the feature weighted anomaly detection techniques disclosed herein.

[0088] exist Fig.10 In the example of , an input image 1010 is provided to a CNN 1020 at a certain pixel resolution, which includes three layers (L1, L2, L3). Each layer provides input to the output of the next layer, and / or is provided as a usable output from the CNN 1020. In this example, based on the predefined structure and training of the CNN 1020, layer L2 contains 128 filters, and each filter produces a 28×28 feature map. Layer L3 contains 256 filters, and each filter produces a 14×14 feature map. Interpolation converts the L3 output to a 28×28 feature map that matches the size of the output from L2. These feature maps and the number of filters are visually depicted at 1030.

[0089] The final output of CNN 1020 to be used in k-NN anomaly detection is shown at 1040. The output depicted at 1040 is a 28×28 array of feature vectors 1042, each feature vector having dimensions of {384×1}. Therefore, the total number of feature vectors is 784 (28×28). The dimensions of each feature vector (384) represent 128 filters from the L2 output plus 256 filters from the L3 output. The output depicted at 1040 (784 feature vectors, each with 384 filter components) is the output used in the k-NN based anomaly detection calculation.

[0090] Return to reference Figure 3 As with the previous discussion of feature extraction and anomaly detection, the anomaly score is calculated by calculating the distance in vector space between each feature vector of the image and its nearest neighbor in a kernel set of feature vectors of good images. A smaller average distance of the feature vectors of the image results in a smaller anomaly score (more likely to be a good part), while a larger average distance results in a larger anomaly score (i.e., if the score is above a threshold, it is a bad / abnormal part).

[0091] When calculating the distance between the feature vector of the component / image currently being evaluated and the feature vector of the good components in the model kernel set, all vector components are traditionally given equal weights and the unweighted distance between one vector and another vector is calculated (where each vector has a dimension of, for example, {384x1}). According to the technology of the present disclosure, weights are assigned to each filter component of the vector being evaluated, and weighted distances are used. An initial weight value of 1.0 is assigned to each of the 384 filter components, and an iterative optimization process is employed in which anomaly scores are recalculated using weighted k-NN calculations, and gradient ascent calculations adjust the filter weights to maximize the gap between good and bad component anomaly scores. This technology will be discussed in detail with reference to the remaining figures.

[0092] Fig.11 1100 is a graph of anomaly scores for multiple component images including good and bad components, illustrating the selection of a set of points used in performing a gradient ascent calculation for feature weighting, according to an embodiment of the present disclosure. The data points on graph 1100 are arranged vertically according to the anomaly score of the data point (component image), and horizontally simply according to the image number.

[0093] In order to optimize the filter weights for anomaly detection, it is necessary to process multiple component images at each iteration step. Specifically, set 1110 contains data points of items (component images) that have been determined to be bad (such as by manual inspection). Set 1120 contains data points of items (component images) that have been determined to be good. A threshold line 1102 can be drawn at an anomaly score value (approximately 1.75) to distinguish between good and bad components. Using the threshold line 1102, it can be seen that there are no missed anomaly detections and no false anomaly classifications, but there is a very small vertical interval or gap between the good data point set 1120 and the bad data point set 1110. In other words, if it is not known in advance whether the component image is good or bad, it is difficult to establish a threshold that will reliably classify the component as good or bad based on the anomaly score. In this example, the threshold line has a value of approximately 1.75. In other examples using different images and / or different feature extractors and kernel sets, the good / bad threshold may have a lower or higher value. The actual value of the threshold is not important for this discussion; what is important is to separate the anomaly scores for good and bad parts so that a threshold can be established that reliably distinguishes them.

[0094] The purpose of the disclosed technology is to increase the interval or gap between good part anomaly scores and bad part anomaly scores using an optimization-based feature weighting calculation. Through the weighting values ​​of the filter components thus optimized, the processing of additional images of the same subject (e.g., screw heads) achieves the benefits of increasing the gap and the resulting improvement in anomaly detection performance.

[0095] As mentioned above, it is necessary to process multiple part images at each iteration step. Fig.11 In the actual exemplary implementation shown in the following figures, 56 component images are included in the test data set, as can be noticed by the scale on the horizontal (image number) axis. This set of 56 images is processed in each iteration of the weighted optimization algorithm discussed below. Fig.11 Another concept shown involves a gradient ascent optimization calculation. Specifically, the gradient ascent calculation is implemented using the good and bad test data points closest to the threshold line 1102; this provides a leveraging effect for the gradient ascent calculation to increase the gap between the anomaly scores of the good parts and the bad parts. Fig.11 In , ellipse 1112 contains a number of bad data points to be included in set P, and ellipse 1122 contains a number of good data points to be included in set P. Set P is Fig.11 , including three good data points and three bad data points; however, the number used in various implementations may be more or less than three, and may actually be one of each (one good and one bad). The selection and use of the various points in set P are discussed further below.

[0096] Fig.12 is a block diagram illustration of a k-NN based anomaly detection system 1200 including feature filter weighting during online test data processing according to an embodiment of the present disclosure. The system 1200 has Figure 3 and 5 The system 1200 has a similar architecture to that of the system 5, including an offline learning portion and an online testing portion. The system 1200 includes feature weighting calculations coupled with k-NN anomaly score determination, as described below.

[0097] Offline learning part 302 and Figure 3 The offline learning portion 302 of the system 1200 is the same, where the training data 310 is provided to the feature extractor module 320, and the features 330 are included in the memory bank 340. In the system 1200, the training data 310 includes images of only good parts, and not bad / abnormal parts. This is because the goal is to create a memory bank 340 with a kernel set of known good feature data, and use the model kernel set in the entire feature weighting calculation.

[0098] In the online testing portion 1204, the input 1210 includes multiple images of parts from known classifications (each part image is labeled as good or abnormal). In this example, there are 56 images in the input 1210. Each image from the input 1210 is provided to the feature extractor module 1220, which is previously Fig.10 12. The type of CNN-based feature extractor module described. Features 1230 are provided for each image from the feature extractor module 1220. Again, in this example, the features 1230 include 784 feature vectors for each component image, each feature vector containing 384 filter components. The features 1230 for each of the 56 component images, along with a label (good or bad) for each image, are provided to a weighted k-NN module 1240, which determines an anomaly score for each individual component image.

[0099] The weighted k-NN module 1240 compares the features 1230 of each component with the set of model kernels pre-trained in the offline learning portion 302 from the memory bank 340. The anomaly score of each component is determined by the weighted k-NN module 1240 based on the weighted distance (in the feature vector space) between the features of the component and the features of the nearest neighbors in the kernel set of good components in the memory bank 340. Components with features that closely match the kernel set (i.e., very small distances) will have low anomalies. Components with features that are not closely matched to the kernel set (i.e., large distances) will have high anomaly scores. In the weighted k-NN module 1240, the distance is calculated using the weighted values ​​assigned to each of the 384 filter components. The weighted values ​​are contained in a weight vector w of size {384×1}, and initially all values ​​in w are set to 1.0.

[0100] A filter weight optimization module 1250 is included in the system 1200. The filter weight optimization module 1250 calculates the optimal value of the weight vector w in an iterative loop with the k-NN module 1240. The functions performed by the filter weight optimization module 1250 include defining a set P based on the anomaly scores (e.g., 56 scores) of all component images, performing a gradient ascent calculation to calculate a new value of the weight vector w, and updating the weight library 1260. The set of model kernels (of good component feature data) is unchanged and is provided to the weighted k-NN module 1240 from the storage body 340. The updated weight vector w is also provided from the weight library 1260 to the weighted k-NN module 1240, which calculates new anomaly scores for all component images based on the new filter weights. The iterative weight optimization loop of modules 1240 and 1250 continues for a specified number of iterations, or until it is determined that the weight values ​​have been optimized. The details of these calculations will be described below in conjunction with Fig.13 discuss.

[0101] At block 1270, a final part classification is provided for each part image. The final part classification at block 1270 is not performed during the iterative weighted optimization loop of modules 1240 and 1250. However, after the filter weight values ​​are optimized as described above, the online testing portion 1204 can be used to process the part images and perform anomaly detection on unlabeled images (i.e., it is not known whether each image is a good part or a bad part), including the final part classification at block 1270. Using the optimal weight values, and thresholds assigned with values ​​that are in the gap between good and bad parts, the system 1200 will reliably classify part images without missing anomaly detections and without false anomaly classifications.

[0102] The final classification is typically -1 (for a good part) or 1 (for a bad or abnormal part), although other conventions may be defined. The final classification value is based on a weighted k-NN anomaly score compared to a threshold, where the weight values ​​used by the weighted k-NN module 1240 to determine the score are optimized in the above-mentioned iterative calculation, and the threshold used to perform the final classification is established based on the weighted optimized score distribution.

[0103] Fig.13 1300 is a flowchart of a method for feature weighting in a k-NN based anomaly detection system according to an embodiment of the present disclosure. Fig.13 The method is Fig.12 The method is performed in the system 1200 and specifically in the online testing portion 1204, wherein the iteration steps particularly occur in the weighted k-NN module 1240 and the filter weight optimization module 1250.

[0104] When the online test portion 1204 is used in training mode, the process begins at the start ellipse 1302. At box 1304, feature data of labeled or classified component images is provided, including images of known good components and images of known bad components. In the example discussed above and continued here, 56 component images are provided for training the anomaly detection system by feature weighting. Of course, more or less than 56 component images can be used in the test data set. If a larger number of input test images is required than the number of available component images, data augmentation can be achieved by flipping each available component image horizontally and vertically. For the purpose of this computational discussion, bad component images (i.e., anomalies or defects) are defined as included in the defect data set D, and good component images are defined as included in the good data set G.

[0105] At block 1306, a model kernel set M of good part feature data from memory bank 340 is provided, along with certain input parameters. The input parameters required for the calculation include a score function f (based on the feature vector data of the part image, the model kernel set M, and the weight vector w) that uses a weighted k-NN calculation to determine anomaly scores for part images, a label y defined as having a value of 1 for defective part images and a value of -1 for good part images, a function g defined by g=yf (i.e., for defective parts, the value of g is the value of the anomaly score from f, and for good parts, the value of g is the negative of the value of the anomaly score from f), and a learning rate factor α.

[0106] At box 1308, the weight vector w is initialized to a value of 1 (i.e., all 384 filter weights are set to 1). At box 1310, anomaly scores are calculated for all part images using a weighted k-NN calculation. In the current example, all 56 part images are analyzed using the score function f to determine anomaly scores, some of the 56 part images are in the defective data set D, and some of the 56 part images are in the good data set G. The first time box 1310 is executed, the filter weights in vector w all have a value of 1; in subsequent executions of box 1310, the filter weights in vector w all have different values. After executing box 1310, all parts in the test data set have anomaly scores that are Fig.11 The situation depicted in and discussed previously.

[0107] At block 1312, a set of points P is determined. As discussed previously and Fig.11 As shown, the set P includes one or more good data points with the highest anomaly scores and one or more bad data points with the lowest anomaly scores. The specific data points included in the set P may change from one iteration to the next because each data point (part image) receives a new anomaly score after each update of the weight vector w. The following is an example of a calculation that can be used in block 1312 to determine the points in the set P: P = {argmin x∈D f(x,M,w)argmin x∈G f(x,M,w)} (2) Wherein equation (2) is as follows; find the point x with the minimum value of the score function f among the elements of the defective data set D; and find the point x with the maximum value of the score function f among the elements of the good data set G. In equation (2), "points" are respective recognized component images (e.g., #1-56) defined by their extracted feature vectors, and the score function f is evaluated based on the feature vectors of x, the model kernel set M, and the weight vector w.

[0108] At block 1314, a gradient ascent calculation is performed to update the individual filter weight values ​​in the weight vector w. As is well known in the art, gradient ascent is an iterative technique that can be used to evaluate the effect of a set of input variables on a function value and to follow the gradient to maximize the function. In this case, the gradient ascent calculation performed at block 1314 is defined as: where equation (3) is obtained by adding a learning rate factor α multiplied by the gradient of the function g evaluated at the set of points P The weight vector w is updated by the term of P. Recall that the value of g is the anomaly score of the defective part from f, and the value of g is the negative of the anomaly score of the good part from f. Therefore, the value of function g is the largest when the anomaly score of the defective part in P is high and the anomaly score of the good part in P is low. In each iteration, the gradient The local value of , and subsequent iterations will use this gradient value to calculate the next iteration of the weight vector w according to equation (3). As a result, the weight vector w is updated in the direction of the positive gradient, and eventually the optimal weight vector w that maximizes g is found.

[0109] At decision diamond 1316, it is determined whether to continue iteration. Iteration continues until the gradient converges to a predetermined convergence criterion or reaches a predetermined maximum number of iterations. At box 1306, the convergence criterion and the maximum number of iterations may be provided in the input parameters. When iteration continues at decision diamond 1316, the process loops back to box 1310, where the anomaly scores for all components are recalculated using the updated weight vector w. This is followed by determination of a new set P and a new gradient ascent calculation, and so on.

[0110] When the iteration ends at decision diamond 1316, the optimal weight vector w has been found (or can be found by analyzing the data of all iterations) for separating and distinguishing good parts from bad parts in the test data set. This means Fig.12 The system 1200 is now trained to perform anomaly detection and produce the desired results. Fig.13 At block 1318 of , the weighted k-NN module 1240 is used with the final (optimal) weight vector w to perform anomaly detection in a "production" mode, in which the input component images have not yet been classified as good versus bad. As long as the input component images in production mode are of the same subject as the test images used for training (e.g. Figure 2 The system 1200 will accurately perform anomaly detection on the unclassified component image. The process ends at the end point 1320.

[0111] Fig.13 The foregoing discussion of describes the various evaluations and calculations performed in the process of feature filter weighting. Fig.12 It should be understood that the various boxes and modules are embodied in the algorithm running on the processor (or multiple processors communicating with each other), so that the necessary data is available for each computing module required for feature extraction, k-NN scoring, weight vector optimization, etc. Fig.12 It is described intuitively using arrows.

[0112] Fig.14 is a pair of graphs of anomaly scores of a plurality of component images including good components and bad components according to an embodiment of the present disclosure, which shows that Fig.13 The feature weighting method produces an improvement over the gap ratio. The graph 1400 at the left shows the initial anomaly scores for the 56 test part images described above. As explained previously, the test part images are grouped into a set 1410 of bad (abnormal or defective) parts and a set 1420 of good parts. The anomaly scores are from actual experiments under initial conditions where all filter weights are set to a value of 1, which corresponds to Fig.13 The first iteration of flowchart 1300. A threshold line 1402 can be drawn that separates a set 1410 of bad parts from a set 1420 of good parts, but there is a very small separation between the sets.

[0113] Figure 1430 at the right shows that when using Fig.13 The anomaly score of the test part image when the feature weighting is performed using the best weight vector w found by the method. A threshold line 1432 can be drawn that separates the set of bad parts 1410 from the set of good parts 1420, and it can be seen that there is a much larger gap between the sets. The gap 1440 is defined as the difference in anomaly score between the lowest scoring bad part and the highest scoring good part. The range 1450 is defined as the difference between the highest and lowest total scores. The gap ratio is then defined as the gap 1440 divided by the range 1450.

[0114] In the first iteration (initially, all feature filter weights are set to 1), the gap ratio in the experiment is 3.99%. In the iteration using the optimal weight vector w, the gap ratio is 26.98%. This significant increase in the gap ratio, visually demonstrated by the difference in the gaps between Figures 1400 and 1430, clearly demonstrates the effectiveness of the disclosed feature weighting technique for k-NN based anomaly detection. It should also be noted that increasing the function g in The gradient at has a similar mathematical meaning as increasing the gap ratio, such as Fig.14 shown.

[0115] Fig.15 According to an embodiment of the present disclosure, Fig.13 1500 of the gap ratio between good and bad parts plotted against the number of iterations obtained using the feature weighting method. Trace 1510 plots the gap ratio for each iteration. It can be seen that the gap ratio increases significantly at the beginning, peaks after about 70 iterations, then decreases and meanders thereafter. This is the essence of the gradient ascent method, which will continue to search in the feasible state space to see if a better optimum can be found.

[0116] A line 1520 is drawn at the iteration number where the peak gap ratio is achieved. The peak gap ratio corresponds to Fig.14 Figure 1430. As mentioned above, it is possible to allow Fig.13The method is iterated to continue for a sufficiently large number of iterations, and then the maximum gap ratio (such as Fig.15 As shown in ), or the iteration can be terminated when a convergence criterion is met. The convergence criterion can be defined in terms of a set improvement over successive iterations in order to avoid premature termination at a local maximum, e.g. Fig.15 The termination seen at approximately iteration number 10.

[0117] Fig.16 According to an embodiment of the present disclosure, Fig.13 The feature weighting method generates Fig.15 Graph 1600 of filter weights at the optimal number of iterations. Fig.16 The respective filter weight values ​​for each of the 384 filter components in the weight vector w are plotted. Recall that all filter weights are initially set to a value of 1. After gradient ascent optimization of the weight vector w, it can be seen that the majority of the respective filter weights remain fairly close to 1 in value, but some are significantly above 1 and some are significantly below 1. This distribution of filter weights demonstrates the efficacy of the disclosed technique. That is, the weighted k-NN anomaly scoring calculation using feature vectors extracted from component images of a particular subject, combined with the gradient ascent filter weight optimization calculation, provides a system that is trained to provide accurate anomaly detection results. The filter weighting is an effective means of separating good component images from bad component images of a subject, and Fig.16 It is shown that it is not possible to efficiently select individual filter weight values ​​in any manual approach. Selecting filter weights one at a time is also ineffective due to the interdependence of hundreds of filter weights when computing feature vector distances.

[0118] Testing of anomaly detection systems trained using feature filter weights as described above produced desired anomaly detection results characteristics in terms of recall (no missed anomaly detections) and in terms of precision (no false anomaly classifications).

[0119] Throughout the foregoing discussion, various computers are described and implied. It should be understood that the software applications and modules of these computers are executed on one or more computing devices having a processor and a memory module. In particular, this includes a computer having a processor configured to execute Figure 5 and 12 The computer may communicate with an imaging system (e.g., a camera) that provides input images and other input / output devices required to implement a fully automated system.

[0120] The foregoing discussion discloses and describes only exemplary embodiments of the present disclosure. Those skilled in the art will readily recognize from such discussion and from the accompanying drawings and claims that various changes, modifications and variations may be made therein without departing from the spirit and scope of the present disclosure as defined in the appended claims.

Claims

1. A method for adaptive anomaly detection, the method comprising: Providing an input data set including a plurality of good component images and a plurality of abnormal component images; performing feature extraction on the images in the input data set to generate feature vector data; determining an anomaly score for the image using a k-nearest neighbor (k-NN) algorithm running on a computer having a processor and a memory, including comparing the feature vector data to a model kernel set of feature vectors comprising good part images; performing a human inspection of the image when the anomaly score is greater than a threshold; When the inspection determines that the image is truly abnormal, using the feature vector data of the image to update the abnormality library; When the inspection determines that the image is a false anomaly, adjusting the model kernel set includes replacing a feature vector in the model kernel set with a feature vector in the feature vector data when a predefined relative distance criterion is met; calculating a new value of the threshold after adjusting the set of model kernels; and Perform feature extraction on the next image in the input dataset. 2 . The method of claim 1 , further comprising, after adjusting the set of model kernels and calculating a new value for the threshold, performing anomaly detection on a test data set comprising a plurality of good part images and a plurality of abnormal part images.

3. The method of claim 2, wherein performing anomaly detection on a test data set comprises using the k-NN algorithm to calculate an anomaly score for each of the individual images in the test data set, and comparing the anomaly score to a known good or anomaly classification for each of the individual images to determine a performance score for the current adjustment of the model kernel set and the threshold.

4. The method according to claim 1, wherein: A good part image is an image of a part determined to be of acceptable quality, and a bad part image is an image of a part determined to be of unacceptable quality.

5. The method according to claim 1, wherein: The set of model kernels is initially created by including feature vectors from images in a training data set including a plurality of good part images.

6. The method of claim 1 , wherein determining the anomaly score of the image using a k-NN algorithm comprises calculating a distance from each feature vector in the feature vector data to each feature vector in the model kernel set, identifying a minimum distance to a nearest neighbor kernel set feature vector, and determining the anomaly score based on the minimum distance.

7. The method according to claim 6, wherein: The feature extractor extracts a plurality of feature vectors for each of the plurality of images, each feature vector including a plurality of component values, and the distance is calculated in a vector space based on a difference between corresponding component values.

8. A method according to claim 1, wherein adjusting the model kernel set includes replacing a feature vector in the model kernel set with a feature vector in the feature vector data when the distance from a feature vector in the feature vector data to the nearest feature vector in the model kernel set is greater than the distance between the nearest neighbor pairs of feature vectors in the model kernel set.

9. The method according to claim 8, wherein: One of the feature vectors of the nearest neighbor pair is removed from the model kernel set, and the feature vector in the feature vector data is added to the model kernel set.

10. The method of claim 1, wherein calculating the new value of the threshold comprises recalculating an anomaly score of the image after adjusting the model kernel set, calculating an upper limit of the threshold using the adjusted model kernel set and an anomaly library, and calculating the new value of the threshold as the larger of the old value of the threshold and the recalculated anomaly score plus an incremental term and being capped to be no greater than the upper limit.

11. The method of claim 1 further comprising, after processing all images in the input data set, using the k-NN algorithm to calculate anomaly scores for images of unknown well or anomaly classified production parts using a final set of model kernels, accepting parts with anomaly scores below a final threshold, and rejecting parts with anomaly scores above the final threshold.

12. A method for adaptive anomaly detection, the method comprising: creating a set of model kernels comprising feature vectors from images in a training data set comprising a plurality of good part images, wherein a good part image is an image of a part having acceptable quality; providing an input data set including a plurality of good part images and a plurality of abnormal part images, wherein the abnormal part images are images of parts having unacceptable quality; performing feature extraction on the images in the input data set to generate feature vector data; determining an anomaly score for the image using a k-nearest neighbor (k-NN) algorithm running on a computer having a processor and a memory, including comparing the feature vector data to the set of model kernels; performing a human inspection of the image when the anomaly score is greater than a threshold; When the inspection determines that the image is truly abnormal, using the feature vector data of the image to update the abnormality library; When the inspection determines that the image is a false anomaly, adjusting the model kernel set includes replacing a feature vector in the model kernel set with a feature vector in the feature vector data when a predefined relative distance criterion is met; calculating a new value of the threshold after adjusting the set of model kernels; performing anomaly detection on a test data set including a plurality of good part images and a plurality of abnormal part images to determine a currently adjusted performance score of the set of model kernels and the threshold; performing feature extraction on a next image in the input data set; as well as After processing all images in the input data set, anomaly scores are calculated for images of unknown good or anomaly classified production parts using the k-NN algorithm with a final set of model kernels, parts with anomaly scores below a final threshold are accepted, and parts with anomaly scores above the final threshold are rejected.

13. A method according to claim 12, wherein adjusting the model kernel set includes, when the distance from a feature vector in the feature vector data to the nearest feature vector in the model kernel set is greater than the distance between the nearest neighbor pair of feature vectors in the model kernel set, removing one of the feature vectors of the nearest neighbor pair from the model kernel set and adding the feature vector in the feature vector data to the model kernel set.

14. The method according to claim 12, wherein calculating the new value of the threshold comprises recalculating the anomaly score of the image after adjusting the model kernel set, calculating the upper limit of the threshold using the adjusted model kernel set and anomaly library, and calculating the new value of the threshold as the larger of the old value of the threshold and the recalculated anomaly score plus an incremental term and being capped to be no greater than the upper limit.

15. An adaptive anomaly detection system, the system comprising: A computer having a processor and a memory, wherein the memory is configured with; a feature extractor module that extracts feature vectors from images in a training data set comprising a plurality of good part images, wherein the feature vectors from the training data set are used to create a set of model kernels; k-Nearest Neighbor (k-NN) algorithm; as well as Adjust the module, An input data set including a plurality of good component images and a plurality of abnormal component images is provided to the feature extractor module, The feature extractor module performs feature extraction on the images in the input data set to generate feature vector data, The k-NN algorithm determines an anomaly score for the image, comprising comparing the feature vector data to the set of model kernels, When the anomaly score is greater than a threshold and human inspection determines that the image is truly abnormal, the adjustment module updates the anomaly library using the feature vector data of the image, When the anomaly score is less than the threshold and the human inspection determines that the image is a false anomaly, the adjustment module adjusts the model kernel set, including replacing a feature vector in the model kernel set with a feature vector in the feature vector data when a predefined relative distance criterion is met, The adjustment module calculates a new value of the threshold after adjusting the model kernel set, and The feature extractor module performs feature extraction on the next image in the input dataset.

16. A system according to claim 15, wherein adjusting the model kernel set includes, when the distance from a feature vector in the feature vector data to the nearest feature vector in the model kernel set is greater than the distance between the nearest neighbor pair of feature vectors in the model kernel set, removing one of the feature vectors of the nearest neighbor pair from the model kernel set and adding the feature vector in the feature vector data to the model kernel set.

17. The system of claim 16, wherein: The feature vector data includes a plurality of feature vectors for each of the plurality of images, each feature vector including a plurality of component values, and the distance is calculated in a vector space based on differences between corresponding component values.

18. The system of claim 15, wherein calculating the new value of the threshold comprises recalculating an anomaly score of the image after adjusting the model kernel set, calculating an upper limit of the threshold using the adjusted model kernel set and an anomaly library, and calculating the new value of the threshold as the larger of an old value of the threshold and the recalculated anomaly score plus an incremental term and being capped to be no greater than the upper limit.

19. The system of claim 15, wherein the k-NN algorithm determines an anomaly score for the image by calculating a distance from each feature vector in the feature vector data to each feature vector in the model kernel set, identifying a minimum distance to a nearest neighbor kernel set feature vector, and determining the anomaly score based on the minimum distance.

20. The system of claim 15, wherein: After processing all images in the input data set, the k-NN algorithm uses a final set of model kernels to calculate anomaly scores for images of unknown good or anomaly classified production parts, where parts with anomaly scores below a final threshold are accepted and parts with anomaly scores above the final threshold are rejected.

Images