Anomaly detecting device and anomaly detecting method

Abstract

This anomaly detecting device is a trained model trained in advance using a plurality of items of image data, and comprises: a first computing unit that computes an anomaly score from an anomaly map obtained as a result of processing of an input image; a second computing unit that computes a vector in at least a partial region of the input image and computes an anomaly score based on the computed vector; and an output unit that outputs information relating to an anomaly in an object included in the input image on the basis of the anomaly score computed by the first computing unit and the anomaly score computed by the second computing unit.

Description

Anomaly detection device and anomaly detection method

[0001] This application claims priority to Japanese Patent Application No. 2024-023261, filed on February 19, 2024, the contents of which are incorporated herein by reference.

[0002] Conventionally, anomalies in objects contained in images have been detected using a model trained based on a large number of images (see, for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-64356

[0004] When the above-described object anomaly detection method is used, it is possible to infer the presence or absence of anomalies that can be determined from the images used for learning, but it may not be possible to detect anomalies depending on the object for anomaly detection. For example, when attempting to detect anomalies in a product in which multiple items are packaged, it may be possible to detect anomalies in defects of individual items contained in the package, but it is not easy to detect anomalies when the number of items contained in the package is abnormal.

[0005] The present invention has been made in view of the above circumstances, and has as its object to provide an abnormality detection device and an abnormality detection method that are capable of detecting an abnormality from an input image with high accuracy.

[0006] [1] In order to solve the above-mentioned problems, one aspect of the present invention is an anomaly detection device that includes a trained model that has been trained in advance using a plurality of image data, a first calculation unit that calculates an anomaly score from an anomaly map obtained as a result of processing an input image, a second calculation unit that calculates a vector in at least a partial area of ​​the input image and calculates an anomaly score based on the calculated vector, and an output unit that outputs information about an abnormality of an object included in the input image based on the anomaly score calculated by the first calculation unit and the anomaly score calculated by the second calculation unit.

[0007] [2] Furthermore, one aspect of the present invention is an anomaly detection device as described in [1] above, wherein the calculation process by the first calculation unit and the calculation process by the second calculation unit are performed in parallel, and when the anomaly score by at least one of them is equal to or greater than a predetermined value, the output unit outputs information indicating that an anomaly has been detected in the input image and information indicating which calculation unit detected the anomaly, in association with each other.

[0008] [3] Furthermore, one aspect of the present invention is that in the anomaly detection device described in [1] above, either the calculation processing by the first calculation unit or the calculation processing by the second calculation unit is performed first, and if the anomaly score by the calculation processing performed first is equal to or greater than a predetermined value, the subsequent calculation processing is not performed.

[0009] [4] Furthermore, one aspect of the present invention is an anomaly detection device according to any one of [1] to [3] above, wherein the first calculation unit learns parameters of a Patch Description Network (PDN) by knowledge distillation using a Student-Teacher framework based on an input image, and the second calculation unit calculates a Mahalanobis distance from an output image using a Student model constructed by the first calculation unit, and calculates an anomaly score based on the calculated Mahalanobis distance.

[0010] [5] Furthermore, one aspect of the present invention is an abnormality detection device according to any one of [1] to [4] above, wherein the first calculation unit and the second calculation unit perform calculations on the same image multiple times, and the number of calculations by the first calculation unit and the number of calculations by the second calculation unit are different from each other.

[0011] [6] Furthermore, one aspect of the present invention is an anomaly detection device according to any one of [1] to [5] above, wherein the output unit outputs information about an abnormality of an object contained in an input image based on a weighted addition or weighted average of the anomaly score calculated by the first calculation unit and the anomaly score calculated by the second calculation unit.

[0012] [7] Furthermore, one aspect of the present invention is an anomaly detection device according to any one of [1] to [5] above, wherein the output unit outputs information indicating that an object included in the input image has an abnormality when the anomaly score calculated by the first calculation unit is equal to or greater than a predetermined first threshold, or when the anomaly score calculated by the second calculation unit is equal to or greater than a predetermined second threshold.

[0013] [8] Another aspect of the present invention is an anomaly detection device that includes: a first calculation unit that calculates an anomaly score from an anomaly map obtained as a result of processing an input image, the first calculation unit being a trained model that has been trained in advance using a plurality of image data; a second calculation unit that generates a segmentation map for each element of the input image and calculates an anomaly score based on at least one of the size, color, and number of each region; and an output unit that outputs information about anomalies in objects included in the input image based on the anomaly scores calculated by the first calculation unit and the anomaly scores calculated by the second calculation unit.

[0014] [9] Another aspect of the present invention is an anomaly detection method that includes a trained model that has been trained in advance using a plurality of image data, a first calculation step of calculating an anomaly score from an anomaly map obtained as a result of processing an input image, a second calculation step of dividing the input image into a plurality of regions, calculating a Mahalanobis distance for each divided region, and calculating an anomaly score based on the calculated Mahalanobis distance, and an output step of outputting information about an abnormality of an object included in the input image based on the anomaly score calculated by the first calculation step and the anomaly score calculated by the second calculation step.

[0015] According to the present invention, it is possible to provide an abnormality detection device and an abnormality detection method that are capable of detecting an abnormality from an input image with high accuracy.

[0016] FIG. 1 is a diagram illustrating an overview of a product inspection system according to an embodiment of the present invention; FIG. 2 is a diagram illustrating an example of a structural abnormality detected by the anomaly detection device according to the embodiment of the present invention; FIG. 3 is a diagram illustrating an example of a logical abnormality detected by the anomaly detection device according to the embodiment of the present invention; FIG. 4 is a diagram illustrating an example of a process performed by the anomaly detection device according to the embodiment of the present invention; and FIG. 5 is a block diagram illustrating an example of the internal configuration of the anomaly detection device according to the embodiment of the present invention.

[0017] [Embodiments] Preferred embodiments of an anomaly detection device and an anomaly detection method according to the present invention will be described in detail below with reference to the accompanying drawings. Note that the aspects of the present invention are not limited to these embodiments and include various modifications and improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art or that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the present invention. Furthermore, in the drawings, the scale and number of components may differ from the scale and number of the actual structures to make each configuration easier to understand.

[0018] First, the premise of this embodiment will be described. The anomaly detection device and anomaly detection method according to this embodiment detect anomalies in an object captured in an image. Objects that are the target of anomaly detection include a variety of objects, such as products manufactured in a factory, people captured on a surveillance camera, and moving objects such as vehicles and trains. In the embodiment described below, an example will be described in which the anomaly detection device and anomaly detection method according to this embodiment are used to detect anomalies in products manufactured in a factory. However, the scope of application of this embodiment is not limited to this example.

[0019] FIG. 1 is a diagram showing an overview of a product inspection system according to this embodiment. A product inspection system 1 according to this embodiment will be described with reference to this diagram. The product inspection system 1 is installed in a product manufacturing factory, detects whether or not there is a defect in the appearance of manufactured products, and removes any products detected to be defective from the manufacturing line. The product inspection system 1 includes a product conveyor belt 11, an imaging unit 13, a gripping device 14, and an anomaly detection device 15.

[0020] The product conveying belt 11 conveys manufactured products 18. The product conveying belt 11 may be a belt conveyor or the like. The products 18 are placed on the product conveying belt 11 and conveyed within the manufacturing factory. The products 18 may be finished products manufactured in the manufacturing factory, or may be parts in the middle of manufacturing. Furthermore, the products 18 are not limited to industrial products, but may also be materials, food, medicines, etc.

[0021] The imaging unit 13 is provided at a position where it can capture an image of the product 18 being transported on the product conveyor belt 11. The position where it can capture an image of the product 18 being transported on the product conveyor belt 11 may be, for example, directly above the product conveyor belt 11. The imaging unit 13 captures an image of the appearance of the product 18. The image captured by the imaging unit 13 is provided to the anomaly detection device 15.

[0022] The anomaly detection device 15 determines whether an object shown in an image captured by the imaging unit 13 is normal or abnormal, based on the image. The anomaly detection device 15 may include an anomaly detection algorithm based on machine learning, for example. The anomaly detection algorithm may include a pre-trained model. The anomaly detection device 15 transfers the results of the anomaly detection to a predetermined device, for example, the gripping device 14. Alternatively, the results of the anomaly detection by the anomaly detection device 15 may be stored in a storage device (not shown) or output to an alarm unit (not shown) to issue an alarm depending on the presence or absence of an anomaly.

[0023] The gripping device 14 grips the products 18 determined to be abnormal based on the results of the abnormality detection performed by the abnormality detection device 15, and removes them from the product conveying belt 11. The gripping device 14 may also remove the products 18 determined to be abnormal from the product conveying belt 11 by a method other than gripping. An example of such a method is to use different conveying paths on the product conveying belt 11 for normal products 18 and for abnormal products 18.

[0024] Specifically, in the illustrated example, products 18, namely, product 18-1, product 18-2, product 18-3, and product 18-4, are being conveyed on the product conveying belt 11. Of these products 18, product 18-2 has been determined to have an abnormal appearance. Therefore, the gripping device 14 grips product 18-2 and removes it from the product conveying belt 11.

[0025] Here, product anomalies detected by the anomaly detection device 15 include structural anomalies and logical anomalies. Examples of structural anomalies and logical anomalies will be described with reference to FIGS. 2 and 3 . In the description with reference to the figures, an example is given in which the product targeted for anomaly detection is a product in which a plurality of bolts, washers (flat washers), and nuts are individually packaged. However, this embodiment is not limited to an example in which a plurality of parts are individually packaged, and structural anomalies and logical anomalies may occur even in a single part.

[0026] FIG. 2 is a diagram illustrating an example of a structural anomaly detected by the anomaly detection device according to this embodiment. First, an example of a structural anomaly will be described with reference to the same figure. A structural anomaly is, for example, an anomaly (or defect) present on the exterior of a product, such as a dent or a scratch. A structural anomaly can be said to be an anomaly regarding the structure of a product. A structural anomaly can be detected by using a trained model that is trained in advance using image data and determines the presence or absence of an anomaly from an anomaly map based on an input image. For example, if the product to be detected for anomaly is a screw, structural anomalies such as dirt or scratches on the screw can be detected using an anomaly map corresponding to the input image, and therefore can be said to be a structural anomaly. An example of a trained model for detecting anomalies using an anomaly map is EfficientAD.

[0027] 2(A) shows an example of a normal product. As shown in the figure, the normal product includes two different types of bolts b1 and b2. The normal product also includes the same type of washers w1-1 and w1-2. The normal product also includes the same type of nuts n1-1 and n1-2.

[0028] FIG. 2(B) is a diagram showing a first example of a structural abnormality. Product p12 includes all of the bolts, washers, and nuts included in product p11, but also includes a foreign object f1. One example of the foreign object f1 that may be mixed into product p12 is a metal piece or the like that is generated during the manufacturing process of the bolts, washers, and nuts. Because product p12 contains foreign object f1, it is a product that should be rejected from the production line. Therefore, product p12 can be said to have an abnormality.

[0029] FIG. 2(C) is a diagram showing a second example of a structural abnormality. Product p13 includes all of the bolts, washers, and nuts included in product p11, but bolt b2 has a flaw. Because bolt b2 included in product p13 has a flaw, it is referred to as bolt b2A. Product p13 contains a defective part (bolt b2A), and therefore should be rejected from the production line. Therefore, product p13 can be said to have an abnormality.

[0030] 2B and 2C can be detected as an anomaly (appears on the anomaly map) by using a trained model that determines the presence or absence of an anomaly based on an input image using an anomaly map. Therefore, such structural anomalies can be easily detected by using an anomaly map corresponding to the image.

[0031] FIG. 3 is a diagram illustrating an example of a logical anomaly detected by the anomaly detection device according to this embodiment. Next, an example of a logical anomaly will be described with reference to the same figure. A logical anomaly refers to an anomaly that is not structurally abnormal but is logically abnormal. Examples of logical anomalies include missing parts and inaccurate part quantities. Specifically, in the example shown in FIG. 2 , if there are no anomalies (no defects such as scratches or dents) in the individual products included in product p11, but there is an error in the number or type of packaged parts, this corresponds to a logical anomaly. When using an algorithm based on an anomaly map for an image, logical anomalies are difficult to detect because they do not appear in the map.

[0032] 3A shows a normal product p21, which is the same as product p11.

[0033] FIG. 3B shows a first example of a logical anomaly. Product p22 contains two different types of bolts b1 and b2, the same types of nuts n1-1 and n1-2, and a washer w1-1, all of which are contained in product p21. However, product p22 contains a nut n1-3 instead of washer w1-2. In other words, product p22 differs from a normal product in that it contains a nut instead of a washer. Such a product should be rejected from the production line. Therefore, product p22 can be said to have an anomaly.

[0034] FIG. 3C shows a second example of a logical anomaly. Product p23 contains the same types of nuts n1-1 and n1-2 and the same types of washers w1-1 and w1-2 as product p21, but instead of two different types of bolts b1 and b2, it contains two bolts b2-1 and b2-2 of the same type. In other words, product p23 differs from a normal product in that it contains two bolts of the same type instead of two different types of bolts. Such a product should be rejected from the production line. Therefore, product p23 can be said to have an anomaly.

[0035] In the past, it was not easy to detect abnormalities such as those shown in Figures 3(B) and 3(C) when a trained model was used to determine the presence or absence of an abnormality from an anomaly map based on an input image, because the abnormalities were unlikely to appear on the anomaly map.

[0036] In practical applications, products with logical anomalies may lead to substantial functional problems. Logical anomalies are typically as serious as cosmetic problems (i.e., structural anomalies) such as dents or scratches. The anomaly detection device and anomaly detection method according to the present embodiment can accurately detect both the structural anomalies and logical anomalies described above.

[0037] Examples of structural anomalies and logical anomalies have been described with reference to Figures 2 and 3. A structural anomaly is an anomaly that can be plotted on a heat map, while a logical anomaly is an anomaly that cannot be plotted on a heat map. For example, some logical anomalies can be considered structural. Examples of logical anomalies that can be considered structural include (1) different bin contents and (2) the presence or absence of labels, which are used as logical anomalies in the MVTec LOCO AD Dataset. However, in reality, these are position-dependent structural anomalies, and the increased or decreased contents or the original label position can be plotted on a heat map. Therefore, it may be more appropriate to classify structural anomalies or logical anomalies into anomalies that can be plotted on a heat map and anomalies that cannot be plotted on a heat map.

[0038] FIG. 4 is a diagram showing an example of processing performed by the anomaly detection device according to this embodiment. An overview of an anomaly detection algorithm 20 relating to the anomaly detection device and anomaly detection method according to this embodiment will be described with reference to this diagram. The anomaly detection algorithm 20 may employ, at least in part, a feature-based method. The feature-based method is a type of out-of-distribution detection method. According to the feature-based method, a pre-trained feature extractor is used to project an image into a feature space, and anomalies are evaluated using distance-based or density-based metrics.

[0039] First, a Patch Description Network (PDN) is constructed using four convolutional layers to efficiently extract features from 33x33 pixel patches. To enhance feature generation, the PDN is trained based on features from a pre-trained deep network using images from a large image database (e.g., ImageNet). This configuration generates distinct feature vectors that depend only on each 33x33 pixel patch, improving anomaly localization.

[0040] Next, a model for detecting structural anomalies is constructed using a Student-Teacher framework, with the extracted PDN as the teacher. The parameters of the PDN can be said to be learned by knowledge distillation using the Student-Teacher framework. The objective function is preferably based on mean squared error.

[0041] To build a model for detecting structural anomalies, two improvements proposed in EfficientAD are preferably applied. One is a hard loss that focuses on image patches where the Student model 222 is least similar to the Teacher model 221, calculated by squaring the difference between their outputs, and only uses the most lossy elements for backpropagation. The other is an additional loss penalty that is used to prevent the Student model 222 from imitating the Teacher model 221 with out-of-distribution images, incorporating random images from the Teacher model 221's pre-training dataset into the training process.

[0042] Furthermore, a model for detecting logical anomalies is constructed using the Auto Encoder model 223. The Auto Encoder is specifically trained to predict the output of the Teacher model 221, but it is not good at fine-grained pattern reconstruction, and using it directly for anomaly detection may lead to false positives. To address this issue, the output of the Student model 222 is doubled and trained to predict both the output of the Teacher model 221 and the output of the Auto Encoder model 223, effectively learning the systematic reconstruction error of the autoencoder in normal images.

[0043] That is, the Teacher model 221 outputs a Teacher output image 231, the Student model 222 outputs a Student output image (former) 232 and a Student output image (latter) 233, and the Auto Encoder model 223 outputs an Auto Encoder output image 234. The output of the Student model 222 has twice the number of channels compared to the outputs of the other models, so by dividing the output into a front half and a rear half in the channel direction, four outputs of the same shape can be obtained.

[0044] A Teacher output image 231, which is the output of the Teacher model 221, and a Student output image (former) 232, which is the output of the Student model 222, are averaged along the channel direction to calculate the difference, thereby obtaining a local map 241. Similarly, a Student output image (latter) 233, which is the output of the Student model 222, and an Auto Encoder output image 234, which is the output of the Auto Encoder model 223, are averaged along the channel direction to calculate the difference, thereby obtaining a global map 242.

[0045] The obtained local map 241 and global map 242 are merged using the method proposed in EfficientAD to obtain a drawable anomaly map 25. Furthermore, a drawable anomaly score 26 is calculated from the obtained drawable anomaly map 25. The drawable anomaly score 26 can be, for example, the maximum value of the drawable anomaly map 25.

[0046] The process of calculating the drawable anomaly score 26 from the input image 21 using such a procedure may be referred to as a first calculation process, and the configuration that performs the first calculation process may be referred to as a first calculation unit. The first calculation unit can be said to be a trained model that has been trained in advance using multiple image data, and that calculates the anomaly score from an anomaly map obtained as a result of processing the input image. Note that the first calculation unit may also use other CNN-based methods. Examples of other CNN-based methods include a method of detecting anomalies based on differences between multiple images and a method of detecting anomalies that can be drawn on a heat map.

[0047] Next, the Mahalanobis distance calculation unit 27 converts the features of the input image 21 into a vector and performs out-of-distribution detection (OOD detection). The processing performed by the Mahalanobis distance calculation unit 27 can also be described as processing of dividing the input image into multiple regions and calculating the Mahalanobis distance for each divided region. More specifically, the Mahalanobis distance calculation unit 27 may calculate the Mahalanobis distance based on an output image (e.g., Student output image (former) 232) using the model constructed by the first calculation unit. Furthermore, an undrawable abnormality score 28 is calculated from the obtained Mahalanobis distance. This processing may be referred to as a second calculation step, and the configuration that performs this second calculation step (i.e., the Mahalanobis distance calculation unit 27) may be referred to as a second calculation unit.

[0048] Note that instead of calculating the Mahalanobis distance by the Mahalanobis distance calculation unit 27, another calculation method such as out-of-distribution detection may be performed. Specifically, the second calculation unit may generate a segmentation map for each element of the input image and calculate an anomaly score based on at least one of the size, color, and number of each region. That is, the processing performed by the Mahalanobis distance calculation unit 27 can be replaced by other vector calculation processing. In this case, it can also be said that the Mahalanobis distance calculation unit 27 calculates a vector based on feature amounts in at least a partial region of the input image and calculates an anomaly score based on the calculated vector.

[0049] A final anomaly score 29 is calculated from the obtained drawable anomaly score 26 and undrawable anomaly score 28. The anomaly score 29 is compared with a predetermined threshold value to determine whether an object included in the input image 21 is abnormal. The determination result is output by a predetermined output unit.

[0050] Examples of the predetermined method may be weighted addition, weighted averaging, etc. Furthermore, the abnormality score 29 may be a score based on thresholds that are predetermined according to the drawable abnormality score 26 and the undrawable abnormality score 28. That is, if the drawable abnormality score 26 is equal to or greater than a predetermined first threshold, or if the undrawable abnormality score 28 is equal to or greater than a predetermined second threshold, the score may be designed to indicate that the object included in the input image 21 has an abnormality.

[0051] The arithmetic processing by the first arithmetic unit and the arithmetic processing by the second arithmetic unit may be performed in parallel. In this case, if the anomaly score by at least one of them is equal to or greater than a predetermined value, the anomaly score 29 may be designed to indicate that the object included in the input image 21 has an abnormality. Furthermore, the anomaly detection algorithm 20 may output information indicating which arithmetic unit detected the abnormality in association with the anomaly score 29.

[0052] It should be noted that either the calculation process by the first calculation unit or the calculation process by the second calculation unit may be performed first, and if an abnormality is detected in the calculation performed first, the subsequent calculation process may not be performed. The process performed first is preferably the calculation process with a lighter processing load.

[0053] The first calculation unit and the second calculation unit may perform calculations multiple times on the same input image 21. Furthermore, the number of calculations by the first calculation unit and the number of calculations by the second calculation unit may be different from each other.

[0054] FIG. 5 is a block diagram showing an example of the internal configuration of the anomaly detection device 15 according to this embodiment. At least some of the functions of the anomaly detection device 15 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions contained in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. Note that RAM is an abbreviation for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. Furthermore, all or part of the functional units provided in the anomaly detection device 15 may be realized using hardware (e.g., circuitry) such as an ASIC, PLD, or FPGA. Furthermore, all or part of the functional units may be realized by a combination of software and hardware.

[0055] [Summary of the Embodiment] According to the embodiment described above, the anomaly detection device according to this embodiment includes a first calculation unit, a second calculation unit, and an output unit. The first calculation unit is a trained model trained in advance using multiple image data, and calculates an anomaly score from an anomaly map obtained as a result of processing an input image. The second calculation unit divides the input image into multiple regions, calculates a Mahalanobis distance for each divided region, and calculates an anomaly score based on the calculated Mahalanobis distance. The output unit outputs information about an anomaly of an object included in the input image based on the anomaly score calculated by the first calculation unit and the anomaly score calculated by the second calculation unit. That is, according to this embodiment, the first calculation unit detects structural anomalies, and the second calculation unit detects logical anomalies. By adopting this configuration, this embodiment enables accurate detection of anomalies from an input image.

[0056] Furthermore, according to the embodiment described above, the arithmetic processing by the first arithmetic unit and the arithmetic processing by the second arithmetic unit are performed in parallel. When the anomaly score by at least one of the first arithmetic unit and the second arithmetic unit is equal to or greater than a predetermined value, the output unit outputs information indicating that an anomaly has been detected in the input image. Furthermore, the output unit outputs information indicating which of the first arithmetic unit and the second arithmetic unit detected the anomaly in association with the information. That is, according to this embodiment, two arithmetic operations are performed in parallel, and if an anomaly is detected, information indicating whether the anomaly is a structural anomaly or a logical anomaly can be output. Therefore, according to this embodiment, high-speed processing is possible, and the type of anomaly (structural anomaly or logical anomaly) can be output.

[0057] Furthermore, according to the embodiment described above, either the calculation process by the first calculation unit or the calculation process by the second calculation unit is performed first. If the anomaly score from the first calculation process is equal to or greater than a predetermined value, the subsequent calculation process is not performed. That is, according to this embodiment, whichever calculation process requires the lighter processing is performed first, and if an abnormality is detected, the subsequent calculation process is not performed. Therefore, according to this embodiment, it is possible to reduce the processing load and ensure real-time performance.

[0058] Note that all or part of the functions of each unit of the anomaly detection device 15 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, and reading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0059] Furthermore, "computer-readable recording media" refers to portable media such as optical magnetic disks, ROMs, and CD-ROMs, as well as storage units such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over a network such as the Internet, and devices that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client in such cases. Furthermore, the program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0060] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the spirit of the present invention.

[0061] According to the present invention, an abnormality can be detected from an input image with high accuracy.

[0062] 1...Product inspection system, 11...Product conveying belt, 13...Imaging unit, 14...Gripping device, 15...Anomaly detection device, 18...Product, 20...Anomaly detection algorithm, 21...Input image, 221...Teacher model, 222...Student model, 223...Auto Encoder model, 231...Teacher output image, 232...Student output image (former), 233...Student output image (latter), 234...Auto Encoder output image, 241...Local map, 242...Global map, 25...Drawable anomaly map, 26...Drawable anomaly score, 27...Mahalanobis distance calculation unit, 28...Undrawable anomaly score, 29...Anomaly score

Claims

1. An anomaly detection device comprising: a trained model that has been trained in advance using multiple image data, a first calculation unit that calculates an anomaly score from an anomaly map obtained as a result of processing an input image; a second calculation unit that calculates a vector in at least a partial area of ​​the input image and calculates an anomaly score based on the calculated vector; and an output unit that outputs information about anomalies in objects included in the input image based on the anomaly scores calculated by the first calculation unit and the anomaly scores calculated by the second calculation unit.

2. The anomaly detection device according to claim 1, wherein the calculation process by the first calculation unit and the calculation process by the second calculation unit are performed in parallel, and when the anomaly score by at least one of them is equal to or greater than a predetermined value, the output unit outputs information indicating that an anomaly has been detected in the input image in association with information indicating which calculation unit detected the anomaly.

3. The anomaly detection device according to claim 1, wherein either the calculation process by the first calculation unit or the calculation process by the second calculation unit is performed first, and if the anomaly score by the first calculation process is equal to or greater than a predetermined value, the subsequent calculation process is not performed.

4. The anomaly detection device according to any one of claims 1 to 3, wherein the first calculation unit learns parameters of a PDN (Patch Description Network) by knowledge distillation using a Student-Teacher framework based on an input image, and the second calculation unit calculates a Mahalanobis distance from an output image using a Student model constructed by the first calculation unit, and calculates an anomaly score based on the calculated Mahalanobis distance.

5. An abnormality detection device according to any one of claims 1 to 3, wherein the first calculation unit and the second calculation unit perform calculations on the same image multiple times, and the number of calculations by the first calculation unit and the number of calculations by the second calculation unit are different from each other.

6. An anomaly detection device according to any one of claims 1 to 3, wherein the output unit outputs information about an abnormality in an object contained in an input image based on a weighted addition or weighted average of the anomaly score calculated by the first calculation unit and the anomaly score calculated by the second calculation unit.

7. An anomaly detection device as described in any one of claims 1 to 3, wherein the output unit outputs information indicating that an object included in the input image has an abnormality when the anomaly score calculated by the first calculation unit is equal to or greater than a predetermined first threshold, or when the anomaly score calculated by the second calculation unit is equal to or greater than a predetermined second threshold.

8. An anomaly detection device comprising: a first calculation unit that is a trained model that has been trained in advance using multiple image data and that calculates an anomaly score from an anomaly map obtained as a result of processing an input image; a second calculation unit that generates a segmentation map for each element of the input image and calculates an anomaly score based on at least one of the size, color, and number of each region; and an output unit that outputs information about anomalies in objects included in the input image based on the anomaly scores calculated by the first calculation unit and the anomaly scores calculated by the second calculation unit.

9. An anomaly detection method comprising: a trained model that has been trained in advance using multiple image data, a first calculation step of calculating an anomaly score from an anomaly map obtained as a result of processing an input image; a second calculation step of dividing the input image into multiple regions, calculating a Mahalanobis distance for each divided region, and calculating an anomaly score based on the calculated Mahalanobis distance; and an output step of outputting information about anomalies in objects included in the input image based on the anomaly scores calculated by the first calculation step and the anomaly scores calculated by the second calculation step.

Images