Anomaly detection method, anomaly detection device, and program

Abstract

An abnormality detection method for a computer to detect abnormality includes: acquiring (S12) first feature amount data to be output via N convolutional layers (N is an integer of 1 or more) of a convolutional neural network when an image is input (S11) to the convolutional neural network constituted as an encoder; acquiring (S13) second feature amount data to be output via M convolutional layers (M is an integer of 1 or more and M ≠ N is satisfied) of the convolutional neural network, the second feature amount data having a size different from that of the first feature amount data; and detecting (S14) abnormality to the image using feature amounts respectively indicated in the first feature amount data and the second feature amount data having mutually different sizes.

Description

Technical Field

[0001] The present invention relates to an abnormality detection method, an abnormality detection device, and a program.

Background Art

[0002] There is a method of detecting an abnormality appearing in an image by a convolutional neural network (CNN (Convolutional Neural Network)) extracting semantic information from the image and generating a downsampled feature map (see Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Non-Patent Document 2

Non-Patent Document 3

[0004] However, when performing highly accurate anomaly detection, there is a problem in that the processing required for anomaly detection takes a relatively long time.

[0005] Therefore, the present invention provides an anomaly detection method that performs anomaly detection with high accuracy and in a short time. [Means for solving the problem]

[0006] An anomaly detection method according to one aspect of the present invention is an anomaly detection method in which a computer detects anomalies, wherein when an image is input to a convolutional neural network configured as an encoder, first feature data is obtained from N convolutional layers (N is an integer of 1 or more) of the convolutional neural network, and second feature data, which is of a different size from the first feature data, is obtained from M convolutional layers (M is an integer of 1 or more, and M ≠ N) of the convolutional neural network, and an anomaly is detected in the image using the features shown in the first feature data and the second feature data, which are of different sizes.

[0007] These comprehensive or specific embodiments may be implemented as a system, device, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, device, integrated circuit, computer program, and recording medium. [Effects of the Invention]

[0008] According to the anomaly detection method of the present invention, anomalies can be detected with high accuracy and in a short amount of time. [Brief explanation of the drawing]

[0009] [Figure 1] This is an explanatory diagram illustrating a common method for detecting anomalies. [Figure 2] This is a schematic block diagram showing the functional configuration of the anomaly detection device in the embodiment. [Figure 3] This is an explanatory diagram showing the configuration of the encoder in the embodiment. [Figure 4] This is a first flowchart illustrating the processing of the anomaly detection device in the embodiment. [Figure 5] This is a second flowchart illustrating the processing of the anomaly detection device in the embodiment. [Figure 6] This is the first explanatory diagram showing the processing results of the anomaly detection device in the embodiment. [Figure 7] This is a second explanatory diagram showing the processing results of the anomaly detection device in the embodiment. [Figure 8] This is a third explanatory diagram showing the processing results of the anomaly detection device in the embodiment. [Figure 9] This is the first explanatory diagram showing the performance values ​​of the anomaly detection device in the embodiment. [Figure 10] This is a second explanatory diagram showing the performance values ​​of the anomaly detection device in the embodiment. [Figure 11] This is a third explanatory diagram showing the performance values ​​of the anomaly detection device in the embodiment. [Figure 12] This is a fourth explanatory diagram showing the performance values ​​of the anomaly detection device in the embodiment.

Best Mode for Carrying Out the Invention

[0010] (Findings on which the present invention is based) The present inventor has found that the following problems occur with respect to the anomaly detection technology described in the "Background Art" section.

[0011] Anomaly detection (also referred to as AD) involving identification of an anomaly location (so-called localization) has many practical applications such as industrial inspection, road traffic monitoring, and medical diagnosis.

[0012] However, general supervised anomaly detection is difficult to perform in practical applications. First, labeled data is required, but obtaining such labeled data is costly. Second, anomalies usually occur rarely, and the probability of being observed by sensors is relatively low. Third, broad expertise is required to perform consistent labeling of anomalies.

[0013] Given such limitations of supervised AD, a more attractive approach is to collect only anomaly-free images without labels as the training dataset D ,

[0015] , (see the upper part of FIG. 1 described later).

[0014] In the above method, an image deviating from an image without anomalies is classified as an image with anomalies. AD based on a dataset with a relatively low proportion of anomalies is generally regarded as unsupervised AD. Therefore, such an AD task can be reformulated as a task of out-of-distribution (OOD) detection for the purpose of AD.

[0015] OOD for low-dimensional industrial sensors (e.g., power line sensors or acoustic sensors) can be achieved using common k-nearest-neighbor or more advanced clustering methods. However, OOD for high-resolution images is not so straightforward.

[0016] In recent years, CNNs have become useful as a method for extracting semantic information from images and generating downsampled feature maps. While feature extraction using CNNs is less complex than feature extraction using other methods, it is difficult to perform post-processing of feature maps in unsupervised AD methods in real time.

[0017] To overcome the drawbacks of this complexity, the present inventor proposes an AD method based on a conditional normalization flow.

[0018] The proposed AD method, like CNNs, does not depend on the spatial dimension of the feature map, making it possible to achieve high accuracy metrics, low computational complexity, and low memory usage.

[0019] Here, we will illustrate the main idea behind the present invention with the example of an OOD detector.

[0020] Figure 1 shows the training image dataset D. train An example of AD detection, more specifically OOD detection, for identifying abnormal locations, trained using this method, is shown.

[0021] Figure 1 shows D train and D test This is an image showing the cross-section of a sliced ​​cable. train This is a dataset containing images that do not contain any anomalies. test This is a dataset containing images without abnormalities and images with abnormalities. test For images containing abnormalities, a ground truth mask indicating the location of the abnormality within the image is further displayed.

[0022] The OOD detector uses a probability density function to determine the distribution of feature quantities in an image that does not contain anomalies.

[0023]

number

[0024]

number

[0025]

number

[0026]

number

[0027]

number

[0028] This invention provides an anomaly detection method that performs anomaly detection with high accuracy and in a short time.

[0029] An anomaly detection method according to one aspect of the present invention is an anomaly detection method in which a computer detects an anomaly, wherein when an image is input to a convolutional neural network configured as an encoder, first feature data is obtained from N convolutional layers (N is an integer of 1 or more) of the convolutional neural network, second feature data of a different size from the first feature data is obtained from M convolutional layers (M is an integer of 1 or more, and M ≠ N) of the convolutional neural network, and an anomaly is detected in the image using the features shown in the first feature data and the second feature data, which are of different sizes.

[0030] According to the above embodiment, the anomaly detection method performs anomaly detection in an image with high accuracy and in a short time by using first and second feature data of different sizes. Of the first and second feature data, the feature data with the larger size outputs information about anomalies in more detailed regions of the image (i.e., information in which anomalies have been detected with relatively high accuracy), and the processing time required for outputting this information is relatively long. On the other hand, of the first and second feature data, the feature data with the smaller size outputs information about anomalies in broader regions of the image, and the processing time required for outputting this information is relatively short. Therefore, the above anomaly detection method can reduce the processing time required for outputting information compared to outputting anomaly information using two of the larger feature data. Reducing the processing time required for outputting this information contributes to real-time anomaly detection. Also, since at least one of the larger feature data is used, it is possible to output information that incorporates information about anomalies detected with relatively high accuracy. In this way, the above anomaly detection method can perform anomaly detection with high accuracy and in a short time.

[0031] For example, in the anomaly detection, the first output data output from the first decoder is obtained by inputting the first feature data into the first decoder, the second output data output from the second decoder is obtained by inputting the second feature data into the second decoder, and the first output data and the second output data are aggregated to generate and output an output image that indicates the location where the anomaly is appearing in the image.

[0032] According to the above embodiment, the anomaly detection method outputs an output image that indicates the location where the anomaly appears on an image of the same size as the input image (also called the input image). Therefore, by comparing the input image and the output image, it is easy to determine the location of the anomaly in the input image. In this way, the anomaly detection method can perform anomaly detection with high accuracy and in a short time, and the detected anomaly can be more easily understood.

[0033] For example, the first decoder and the second decoder are trained to output first output data and second output data, respectively, that satisfy predetermined conditions, in response to input first feature data and second feature data acquired from a normal image in which no abnormalities are observed. The predetermined conditions may be that an output image in which no abnormalities are shown is generated by aggregating the first output data and the second output data.

[0034] According to the above embodiment, the anomaly detection method is trained so that when a normal image is input, the first and second decoders output an output image that does not show an anomaly. Therefore, it can easily output an appropriate output image in response to a normal image input. Thus, the anomaly detection method can easily, accurately, and quickly detect anomalies.

[0035] An anomaly detection device according to one aspect of the present invention is an anomaly detection device in which a computer detects anomalies, comprising a processor and a memory connected to the processor, wherein the processor uses the memory to acquire first feature data output through N convolutional layers (N is an integer of 1 or more) of a convolutional neural network configured as an encoder when an image is input to the convolutional neural network, and acquires second feature data of a different size from the first feature data, which is output through M convolutional layers (M is an integer of 1 or more, and M ≠ N) of the convolutional neural network, and uses the features shown in the first feature data and the second feature data of different sizes to detect anomalies in the image.

[0036] According to the above embodiment, the same effect as the above anomaly detection method is achieved.

[0037] A program according to one aspect of the present invention is a program that causes a computer to execute the above-described anomaly detection method.

[0038] According to the above embodiment, the program has the same effect as the anomaly detection method described above.

[0039] These comprehensive or specific embodiments may be implemented as a system, device, integrated circuit, computer program, or recording medium such as a computer-readable CD-ROM, or as any combination of a system, device, integrated circuit, computer program, or recording medium.

[0040] The embodiments will be described in detail below with reference to the drawings.

[0041] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components.

[0042] (Embodiment) In this embodiment, we will describe an anomaly detection method that performs anomaly detection with high accuracy and in a short time.

[0043] Figure 2 is a schematic block diagram showing the functional configuration of the anomaly detection device 10 in this embodiment. The anomaly detection device 10 can be implemented by a computer, or in other words, by a processor (CPU (Central Processing Unit) etc.) executing a predetermined program using memory.

[0044] The anomaly detection device 10 detects anomalies in the input image when an image is input to the encoder 20. Specifically, the detection of anomalies in the input image is performed by detecting the location of the anomaly in the input image using inference with an inference model. The image input to the encoder 20 is also called the input image. The configuration of the encoder 20 will be explained with reference to Figure 3.

[0045] Figure 3 is an explanatory diagram showing the configuration of the encoder 20 in this embodiment.

[0046] As shown in Figure 3, the encoder 20 is composed of multiple convolutional layers (also called CLs). Here, let N be the number of CLs in the encoder 20. N is an integer greater than or equal to 1.

[0047] The first of several CLs (CL#1) acquires the input image, performs a convolution operation on the input image, and provides the resulting feature vector to the next CL (CL#2). Subsequent CLs (CL#2 and later) acquire the feature vector provided by the previous CL, perform a convolution operation on the acquired feature vector, and provide the result to the next CL.

[0048] The encoder 20 outputs the first feature data and the second feature data. The first feature data and the second feature data are of different sizes.

[0049] The first feature data is output by CL#N. In other words, the first feature data is data output through N (where N is an integer greater than or equal to 1) convolutional layers of the convolutional neural network provided by the encoder 20. The first feature data is, for example, data having a feature corresponding to each of the four blocks (i.e., blocks #0 to #3) that have two blocks in the vertical direction and two blocks in the horizontal direction. Each feature included in the first feature data has, for example, a value indicating whether or not there is an anomaly at the position corresponding to that feature in the input image, but is not limited to this.

[0050] The second feature data is output by CL#M. In other words, the second feature data is data output through M convolutional layers (where M is an integer greater than or equal to 1 and M ≠ N) of the convolutional neural network provided by the encoder 20. The second feature data is, for example, data with features corresponding to each of the 9 blocks (i.e., blocks #0 to #8) that have 3 blocks vertically and 3 blocks horizontally. Each of the features included in the second feature data has, for example, a value indicating whether or not there is an anomaly at the position corresponding to that feature in the input image, but is not limited to this.

[0051] The first feature data can be described as data indicating anomalies contained within relatively large regions of the input image. In contrast, the second feature data can be described as data indicating anomalies contained within relatively small regions of the input image.

[0052] As shown in Figure 2, the anomaly detection device 10 comprises a first feature acquisition unit 11, a second feature acquisition unit 12, a first output acquisition unit 13, a second output acquisition unit 14, and an aggregation unit 15.

[0053] The first feature acquisition unit 11 acquires the first feature data output by the encoder 20 when an image is input to the encoder 20.

[0054] The second feature acquisition unit 12 acquires the second feature data output by the encoder 20 when an image is input to the encoder 20. The second feature data is different in size from the first feature data.

[0055] The first output acquisition unit 13 inputs the first feature data acquired by the first feature acquisition unit 11 to the first decoder 21 by providing it to the first decoder 21. The first output acquisition unit 13 also acquires the first output data output from the first decoder 21 by inputting the first feature data to the first decoder 21. The first output data is image data relating to an image having the same image size as the input image and indicating the location of the region where an anomaly is indicated by the first feature data. When the first feature data is input to the first decoder 21, it uses the input first feature data to calculate image data of an image having the same image size as the input image and indicating the location of the region where an anomaly is indicated, and outputs it as the first output data.

[0056] The second output acquisition unit 14 inputs the second feature data acquired by the first feature acquisition unit 11 to the second decoder 22 by providing it to the second decoder 22. The second output acquisition unit 14 also acquires the second output data output from the second decoder 22 by inputting the second feature data to the second decoder 22. The second output data is image data relating to an image having the same image size as the input image and indicating the location of the region where an anomaly is indicated by the second feature data. When the second feature data is input to the second decoder 22, it uses the input second feature data to calculate image data of an image having the same image size as the input image and indicating the location of the region where an anomaly is indicated, and outputs it as the second output data.

[0057] The aggregation unit 15 uses the first output data acquired by the first output acquisition unit 13 and the second output data acquired by the second output acquisition unit 14 to detect anomalies in the input image.

[0058] More specifically, the aggregation unit 15 aggregates the first output data acquired by the first output acquisition unit 13 and the second output data acquired by the second output acquisition unit 14 to generate and output an output image that indicates the location where an anomaly is present in the input image. Aggregation is a process that obtains a single output data that combines the characteristics of the multiple output data by calculating a representative value (for example, sum, average, or maximum) of the multiple output data, and can be done by well-known techniques.

[0059] Here, the first decoder 21 and the second decoder 22 may each include a neural network model constructed by machine learning using an input image in which no abnormalities are observed (also called a normal image). That is, the first decoder 21 may include a neural network model trained to output first output data that satisfies predetermined conditions in response to first feature data obtained from a normal image. Similarly, the second decoder 22 may include a neural network model trained to output second output data that satisfies predetermined conditions in response to second feature data obtained from a normal image. The predetermined conditions are that an output image in which no abnormalities are shown is generated by aggregating the first and second output data by the aggregation unit 15.

[0060] The processing of the anomaly detection device 10 configured as described above will now be explained.

[0061] Figure 4 is a first flowchart showing the processing of the anomaly detection device 10 in this embodiment.

[0062] As shown in Figure 4, in step S11, an image is input to the encoder 20. The image input to the encoder 20 may be performed by the anomaly detection device 10 or by another device.

[0063] In step S12, the first feature acquisition unit 11 acquires the first feature data output by the encoder 20 via the N convolutional layers provided by the encoder 20. The first feature data is the first feature data output by the encoder 20 as a result of the image being input to the encoder 20 in step S11.

[0064] In step S13, the second feature acquisition unit 12 acquires the second feature data output by the encoder 20 via the M convolutional layers provided by the encoder 20. The second feature data is the second feature data output by the encoder 20 as a result of the image being input to the encoder 20 in step S11.

[0065] In step S14, the anomaly detection device 10 detects anomalies in the input image using the features shown in the first feature data and the second feature data, which are of different sizes.

[0066] Figure 5 is a second flowchart showing the processing of the anomaly detection device 10 in this embodiment. The flowchart shown in Figure 5 shows the detailed processing included in step S14 of Figure 4.

[0067] In step S14a, the first output acquisition unit 13 acquires the first output data output from the first decoder 21 by inputting the first feature data into the first decoder 21, which includes a neural network.

[0068] In step S14b, the second output acquisition unit 14 acquires the second output data output from the second decoder 22 by inputting the second feature data into the second decoder 22, which includes a neural network.

[0069] In step S14c, the aggregation unit 15 aggregates the first output data acquired in step S14a and the second output data acquired in step S14b to generate and output an output image that indicates the location where an anomaly is present in the input image.

[0070] The processes shown in Figures 4 and 5 enable the anomaly detection device 10 to perform anomaly detection with high accuracy and in a short amount of time.

[0071] In the following sections, the results of the processing by the anomaly detection device 10 will be explained with specific examples.

[0072] Figure 6 is a first explanatory diagram showing the processing results of the anomaly detection device 10 in this embodiment.

[0073] Figure 6 shows three images (a), (b), and (c) of a cross-section of a cable containing three internal wires, along with the Grand Truth mask (in Figure 6), the Score Map (in Figure 6), and the Predicted Mask (in Figure 6). The images shown in Figure 6 were obtained using the MVTec dataset (see Non-Patent Literature 1) as the target for anomaly detection.

[0074] A ground truth mask image is an image in which the location of anomalies is added to the input image. In a ground truth mask image, the location of anomalies is indicated by a white border surrounding the pixel in which the anomaly is detected.

[0075] The score map image shows the degree of abnormality (also called the score) detected by the anomaly detection device 10 for each pixel in the input image. In the score map image, pixels closer to black indicate a higher degree of abnormality; conversely, pixels closer to white indicate a lower degree of abnormality. The score map image is an example of an output image.

[0076] The detection mask image is an image that shows the location of anomalies detected by the anomaly detection device 10 in the input image. The location of anomalies detected by the anomaly detection device 10 is indicated, for example, by a white border surrounding pixels where the score map exceeds a threshold. The detection mask image is an example of an output image. A higher degree of agreement between the detection mask and the ground truth mask indicates a higher accuracy of detection by the anomaly detection device 10.

[0077] Hereafter, the upward direction on the paper in Figure 6 will be referred to as the "upward direction on the paper." The same applies to the downward, leftward, and rightward directions. Also, the direction perpendicular to the paper in Figure 6 and extending towards the viewer will be referred to as the "forward direction on the paper."

[0078] Figure 6(a) is an image showing three wires. In the image shown in Figure 6(a), one of the three wires, the one located towards the top of the page, is protruding towards the front of the page, and a portion of its copper wire is bent downwards. The ground truth mask image in Figure 6(a) shows that the pixel showing the wire located towards the top of the page is abnormal.

[0079] In the score map image in Figure 6(a), the pixels where the wires located on the upper side of the paper are shown are indicated by a color relatively close to black, indicating that they have a relatively high score.

[0080] In the detection mask image in Figure 6(a), it is shown that pixels containing the wire located on the upward side of the paper, a bent portion of the copper wire, and pixels in the vicinity of the aforementioned pixels are detected as abnormal.

[0081] Figure 6(b) is an image showing three wires. In the image shown in Figure 6(b), one of the three wires, located towards the lower right of the page, is discolored black. The ground truth mask image in Figure 6(b) shows that the pixel where the wire located towards the lower right of the page is located is abnormal.

[0082] In the score map image in Figure 6(b), the pixels where the wires located in the lower right corner of the page are shown are indicated by a color relatively close to black, indicating a relatively high score.

[0083] In the detection mask image in Figure 6(b), it is shown that pixels showing the wire located in the lower right direction of the page are detected as abnormal.

[0084] Figure 6(c) is an image showing three wires. The image in Figure 6(c) shows that there are no abnormalities in the cross-sections of the three wires. In the ground truth mask image in Figure 6(c), the white border lines indicating abnormalities are not displayed.

[0085] The score map image in Figure 6(c) shows that the anomaly score is relatively low across the entire image.

[0086] Figure 6(c) shows that no areas were detected as abnormal in the detection mask image.

[0087] Figure 7 is a second explanatory diagram showing the processing results of the anomaly detection device 10 in this embodiment.

[0088] Figure 7 shows the ground truth mask, score map, and detection mask images for three images (a), (b), and (c) that depict a mesh-like wire pattern, similar to Figure 6. The images shown in Figure 7 were created using the MVTec dataset (see Non-Patent Document 1) as the target for anomaly detection.

[0089] Figure 7(a) shows an image of a mesh-like wire in which foreign matter is mixed in a portion of the mesh-like wire (specifically, in the center in the vertical direction of the paper, and extending from the center to the right in the horizontal direction of the paper). The ground truth mask image in Figure 7(a) shows that the pixels where the foreign matter is visible are abnormal.

[0090] In the score map image in Figure 7(a), the pixels at the location where the foreign object is visible are shown in a color relatively close to black, indicating a relatively high score.

[0091] Figure 7(a) shows that in the detection mask image, some of the pixels containing the foreign object are detected as abnormal.

[0092] Figure 7(b) shows an image of a mesh-like wire structure in which foreign matter is mixed in with a portion of the mesh-like wire structure (specifically, in a linear position that runs vertically through the paper). The ground truth mask image in Figure 7(b) shows that the pixels where the foreign matter is visible are abnormal.

[0093] In the score map image in Figure 7(b), the pixels at the location where the foreign object is visible are shown in a color relatively close to black, indicating a relatively high score.

[0094] Figure 7(b) shows that in the detection mask image, some of the pixels containing the foreign object are detected as abnormal.

[0095] Figure 7(c) shows an image of a mesh-like wire pattern that does not contain any abnormalities. In the ground truth mask image of Figure 7(c), the white border lines indicating abnormalities are not displayed.

[0096] The score map image in Figure 7(c) shows that the anomaly score is relatively low overall in the image.

[0097] Figure 7(c) shows that no areas were detected as abnormal in the detection mask image.

[0098] Figure 8 is a third explanatory diagram showing the processing results of the anomaly detection device 10 in this embodiment.

[0099] Figure 8 shows the ground truth mask, score map, and detection mask images for three images (a), (b), and (c) that depict acorns, similar to Figure 6. The images shown in Figure 8 were obtained using the MVTec dataset (see Non-Patent Document 1) as the target for anomaly detection.

[0100] Figure 8(a) shows an image of an acorn with three scratches on a part of it (specifically, the area from the hilum to the stigma). The ground truth mask image in Figure 8(a) shows that the pixels where the scratches are visible are abnormal.

[0101] In the score map image in Figure 8(a), the pixels at the location where the aforementioned scratches are visible are shown in a color relatively close to black, indicating that they have a relatively high score.

[0102] The detection mask image in Figure 8(a) shows that one region containing two of the three scratches mentioned above has been detected as an anomaly.

[0103] Figure 8(b) shows an image of an acorn with a single scratch on a part of it (specifically, the part between the hilum and the stigma). The ground truth mask image in Figure 8(b) shows that the pixel where the scratch is located is abnormal.

[0104] In the score map image in Figure 8(b), the pixels at the location where the aforementioned scratches are visible are shown in a color relatively close to black, indicating that they have a relatively high score.

[0105] The detection mask image in Figure 8(b) shows that the area containing the aforementioned damage is detected as an anomaly.

[0106] Figure 8(c) shows an image of an acorn that is undamaged. In the ground truth mask image in Figure 7(c), the white border indicating an abnormality is not displayed.

[0107] The score map image in Figure 8(c) shows that the anomaly score is relatively low overall in the image.

[0108] Figure 8(c) shows that no areas were detected as abnormal in the detection mask image.

[0109] Figure 9 is a first explanatory diagram showing the performance values ​​of the anomaly detection device 10 in this embodiment.

[0110] Figure 9 shows the detection rate and localization rate of anomalies detected by the anomaly detection device 10, along with the detection rate and localization rate in the comparative example.

[0111] The detection rate indicates the proportion of anomalies that were correctly detected, and more specifically, it is the ratio of the number of anomalies detected by the anomaly detection device 10 for a given image to the total number of anomalies contained in the image. The detection rate is shown for cases where AUROC is used as the evaluation index.

[0112] The localization rate indicates the percentage of anomalies whose location was correctly identified. More specifically, it is the ratio of the number of anomalies in an image whose location was correctly detected by the anomaly detection device 10 to the total number of anomalies in the image. The localization rate is shown for cases using AUROC and AUPRO as evaluation indicators.

[0113] Comparative Example 1-1 is SVDD (Non-Patent Document 2), Comparative Example 1-2 is SPADE (Non-Patent Document 3), Comparative Example 1-3 is CutPaste (Non-Patent Document 4), and Comparative Example 1-4 is PaDiM (Non-Patent Document 5).

[0114] The detection rate of the anomaly detection device 10 was 98.26%, the highest compared to Comparative Examples 1-1 to 1-4. Furthermore, the location identification rate of the anomaly detection device 10 was 98.62% for AUROC and 94.60% for AUPRO, both of which were the highest compared to Comparative Examples 1-1 to 1-4.

[0115] Figure 10 is a second explanatory diagram showing the performance values ​​of the anomaly detection device 10 in this embodiment.

[0116] Figure 10 shows the detection rate and localization rate of anomalies by the anomaly detection device 10, along with the detection rate and localization rate in a comparative example. The detection rate and localization rate are shown when AUROC is used as the evaluation index. The performance values ​​shown in Figure 10 are based on the STC (Non-Patent Literature 6) dataset as the target of anomaly detection.

[0117] Comparative Example 2-1 is CAVGA (Non-Patent Document 7), Comparative Example 2-2 is SPADE (Non-Patent Document 3), and Comparative Example 2-3 is PaDiM (Non-Patent Document 5).

[0118] The detection rate of the anomaly detection device 10 was 72.63%, which is higher than that of Comparative Example 2-2. Furthermore, the location identification rate of the anomaly detection device 10 was 94.48%, which is the highest compared to Comparative Examples 2-1 to 2-3.

[0119] Figure 11 is a third explanatory diagram showing the performance values ​​of the anomaly detection device 10 in this embodiment.

[0120] Figure 11 shows the inference speed and model size for detection by the anomaly detection device 10, along with the case of a comparative example.

[0121] The inference speed is expressed as the number of images processed by the anomaly detection device 10 per unit time. The unit time is, for example, 1 second. The inference speeds for input images with a size of 256 × 256 pixels and 512 × 512 pixels are shown to the left and right of the slash ( / ).

[0122] Furthermore, the data sizes for the inference models are shown to the left and right of the slash ( / ), respectively, for cases using the STC dataset and cases using the MVTec dataset.

[0123] Comparative Example 3-1 is PaDiM (Non-Patent Document 5).

[0124] The inference speed of the anomaly detection device 10 is 34 fps (frames per second) when the input image size is 256 × 256 pixels, and 12 fps when the input image size is 512 × 512 pixels. In both cases, it is higher than that of Comparative Example 3-1.

[0125] Furthermore, the data size of the inference model for the anomaly detection device 10 is 96MB whether using the STC dataset or the MVTec dataset, which is smaller than that of Comparative Example 3-1 in both cases.

[0126] Figure 12 is a fourth explanatory diagram showing the performance values ​​of the anomaly detection device 10 in this embodiment. The performance values ​​shown in Figure 12 are obtained using a different encoder architecture than those shown in Figure 11. The manner in which the performance values ​​shown in Figure 12 are described is the same as in Figure 11.

[0127] Comparative Example 4-1 is SPADE (Non-Patent Document 3), and Comparative Example 4-2 is PaDiM (Non-Patent Document 5).

[0128] The inference speed of the anomaly detection device 10 is 27 fps when the input image size is 256 × 256 pixels, and 9 fps when the input image size is 512 × 512 pixels, which is higher than comparative examples 4-1 to 4-2 in both cases.

[0129] Furthermore, the data size of the inference model for the anomaly detection device 10 is 947MB whether using the STC dataset or the MVTec dataset, which is smaller in both cases compared to Comparative Examples 4-1 to 4-2.

[0130] As described above, the anomaly detection method according to the above embodiment enables high-precision and rapid anomaly detection in an image by using first and second feature data of different sizes. Of the first and second feature data, the larger feature data outputs information about anomalies in more detailed regions of the image (i.e., information in which anomalies have been detected with relatively high accuracy), and the processing time required for outputting this information is relatively long. On the other hand, of the first and second feature data, the smaller feature data outputs information about anomalies in broader regions of the image, and the processing time required for outputting this information is relatively short. Therefore, the above anomaly detection method can reduce the processing time required for outputting information compared to outputting anomaly information using two of the larger feature data. Reducing the processing time required for outputting this information contributes to real-time anomaly detection. Also, since at least one of the larger feature data is used, it is possible to output information that incorporates information about anomalies detected with relatively high accuracy. In this way, the above anomaly detection method can perform anomaly detection with high accuracy and in a short time.

[0131] Furthermore, the above anomaly detection method outputs an output image that indicates the location of the anomaly on the same image size as the input image (also called the input image). Therefore, by comparing the input image and the output image, it is easy to determine the location of the anomaly in the input image. In this way, the above anomaly detection method can perform anomaly detection with high accuracy and in a short time, and the detected anomaly can be more easily understood.

[0132] Furthermore, the above anomaly detection method is trained so that the first and second decoders output an output image that does not show an anomaly when a normal image is input. Therefore, it can easily output an appropriate output image for a normal image input. Thus, the above anomaly detection method can easily, accurately, and quickly detect anomalies.

[0133] In the above embodiment, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that implements the anomaly detection device of the above embodiment is the following program.

[0134] In other words, this program is a program that causes a computer to execute an anomaly detection method in which the computer detects anomalies, and when an image is input to a convolutional neural network configured as an encoder, it acquires first feature data output through N convolutional layers (N is an integer of 1 or more) of the convolutional neural network, acquires second feature data of a different size from the first feature data output through M convolutional layers (M is an integer of 1 or more, and M ≠ N) of the convolutional neural network, and uses the features shown in the first feature data and the second feature data of different sizes to detect anomalies in the image.

[0135] Although the present invention has been described above based on embodiments of anomaly detection methods and the like according to one or more embodiments, the present invention is not limited to these embodiments. Without departing from the spirit of the present invention, various modifications that a person skilled in the art can conceive of may be applied to these embodiments, and forms constructed by combining components from different embodiments may also be included within the scope of one or more embodiments. [Industrial applicability]

[0136] This invention can be used in an anomaly detection device that detects abnormalities in images. [Explanation of symbols]

[0137] 10 Anomaly detection device 11. First Feature Acquisition Unit 12 Second Feature Acquisition Unit 13 First output acquisition section 14 Second output acquisition section 15. Aggregation Section 20 encoders 21 First Decoder 22 Second Decoder

Claims

1. An anomaly detection method in which a computer detects anomalies, When an image is input to a convolutional neural network configured as an encoder, first feature data is obtained by obtaining the output of N convolutional layers (where N is an integer greater than or equal to 1) of the convolutional neural network, and second feature data, which is different in size from the first feature data, is obtained by obtaining the output of M convolutional layers (where M is an integer greater than or equal to 1, and M ≠ N) of the convolutional neural network. Anomalies in the image are detected using the features shown in the first feature data and the second feature data, which are of different sizes. At least a portion of the M convolutional layers are common to at least a portion of the N convolutional layers. Anomaly detection method.

2. In detecting the aforementioned anomaly, By inputting the first feature data into a first decoder including a neural network, the first output data output from the first decoder is obtained. By inputting the second feature data into a second decoder including a neural network, the second output data output from the second decoder is obtained. By aggregating the first output data and the second output data, an output image indicating the location where an anomaly is observed in the image is generated and output. The anomaly detection method according to claim 1.

3. The first decoder and the second decoder are trained to output first and second output data, respectively, that satisfy predetermined conditions, in response to input first and second feature data acquired from normal images in which no abnormalities are observed. The aforementioned predetermined condition is that, by aggregating the first output data and the second output data, an output image in which no abnormalities are shown is generated. The anomaly detection method according to claim 2.

4. An anomaly detection device in which a computer detects anomalies, The system comprises a processor and memory connected to the processor, The processor uses the memory to: When an image is input to a convolutional neural network configured as an encoder, first feature data is obtained by obtaining the output of N convolutional layers (where N is an integer greater than or equal to 1) of the convolutional neural network, and second feature data, which is different in size from the first feature data, is obtained by obtaining the output of M convolutional layers (where M is an integer greater than or equal to 1, and M ≠ N) of the convolutional neural network. Anomalies in the image are detected using the features shown in the first feature data and the second feature data, which are of different sizes. At least a portion of the M convolutional layers are common to at least a portion of the N convolutional layers. Anomaly detection device.

5. A program that causes a computer to execute the anomaly detection method described in any one of claims 1 to 3.

Images