Anomaly detection device, method, and program

Abstract

This enables highly accurate anomaly detection. [Solution] The anomaly detection device according to this embodiment includes a first acquisition unit, an estimation unit, a specification unit, a suppression unit, and a determination unit. The first acquisition unit acquires target data that is subject to anomaly detection. The estimation unit estimates the degree of anomaly of the target data using a pre-trained model that has been trained in advance to perform anomaly detection on the input data. The specification unit refers to a false detection object dictionary which contains information about false detection objects, and determines whether or not the false detection object exists in the target data. If the false detection object exists in the target data, the suppression unit suppresses the degree of anomaly of the region corresponding to the false detection object and generates a corrected degree of anomaly for the target data. The determination unit determines whether the target data is abnormal or normal based on the corrected degree of anomaly.

Description

[Technical Field]

[0001] Embodiments of the present invention relate to an anomaly detection device, method, and program. [Background technology]

[0002] In recent years, there has been a growing need for anomaly detection using surveillance camera footage. For example, unsupervised anomaly detection methods that use only normal images to train the model have the advantage of not requiring abnormal images or anomaly annotations during training, and can detect unknown anomalies. On the other hand, if an object that was not visible during model training is present in the target image, for example, in a model that detects anomalies in road conditions, if a bus that was not visible in the training images appears in the image during inference due to a route change, the presence of the bus may be falsely detected as an anomaly. One solution to this is to retrain the anomaly detection model. However, retraining requires a large number of images that include normal objects that cause false detections, and collecting such images can be difficult. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Patent No. 6809250 [Patent Document 2] Patent No. 7015001 [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] This disclosure was made to solve the above-mentioned problems and aims to provide an anomaly detection device, method, and program that achieve high-precision anomaly detection. [Means for solving the problem]

[0005] The anomaly detection device according to this embodiment includes a first acquisition unit, an estimation unit, a specification unit, a suppression unit, and a determination unit. The first acquisition unit acquires target data that is subject to anomaly detection. The estimation unit estimates the degree of anomaly of the target data using a pre-trained model that has been trained in advance to perform anomaly detection on the input data. The specification unit refers to a false detection object dictionary containing information about false detection objects, which are falsely detected objects, and determines whether or not the false detection object exists in the target data. If the false detection object exists in the target data, the suppression unit suppresses the degree of anomaly of the region corresponding to the false detection object and generates a corrected anomaly degree for the target data. The determination unit determines whether the target data is abnormal or normal based on the corrected anomaly degree. [Brief explanation of the drawing]

[0006] [Figure 1] A block diagram showing an anomaly detection device according to the first embodiment. [Figure 2] A flowchart showing an example of operation of the anomaly detection device according to the first embodiment. [Figure 3] A diagram illustrating an example of identifying a falsely detected object according to the first embodiment and the information stored in the falsely detected object dictionary. [Figure 4] A block diagram showing an anomaly detection device according to the second embodiment. [Figure 5] A flowchart showing an example of operation of the anomaly detection device according to the second embodiment. [Figure 6] This figure shows an example of generating a similarity map of a specific part according to the second embodiment. [Figure 7] This figure shows an example of generating a correction feature map for the suppression unit according to the second embodiment. [Figure 8] A diagram illustrating an example of identifying a falsely detected object according to the second embodiment and a filter stored in the falsely detected object dictionary. [Figure 9] A diagram illustrating an example of identifying a falsely detected object according to the second embodiment and another example of a filter stored in the falsely detected object dictionary. [Figure 10] A flowchart showing an example of operation of the anomaly detection device according to the third embodiment. [Figure 11]A diagram showing an example of the hardware configuration of the abnormality detection device.

Embodiment for Implementing the Invention

[0007] Hereinafter, the abnormality detection device, method, and program according to this embodiment will be described in detail with reference to the drawings. In the following embodiments, parts with the same reference numerals are assumed to perform the same operations, and duplicate descriptions will be omitted as appropriate.

[0008] The abnormality detection device according to this embodiment will be described with reference to the block diagram of FIG. 1. The abnormality detection device 10 according to the first embodiment includes a target acquisition unit 101, an estimation unit 102, a specification unit 103, a suppression unit 104, a determination unit 105, an output unit 106, a user instruction acquisition unit 107, a dictionary generation unit 108, and a storage unit 109.

[0009] The target acquisition unit 101 acquires target data that is the target of abnormality detection. In this embodiment, as an example of the target data, a target image that is the target of abnormality detection is assumed. Note that it is not limited to acquiring still images, and a plurality of images may be acquired as one batch. Also, a moving image may be acquired, and for a moving image, processing may be executed frame by frame or by grouping several frames as one batch. Further, the target data is not limited to images, and may be one-dimensional time-series data such as a voice signal, or data represented as a vector or a multi-dimensional tensor.

[0010] The estimation unit 102 uses a learned model that has been pre-trained to perform abnormality detection on the input data, inputs the target image to the learned model, and estimates the abnormality detection result.

[0011] The pre-trained model assumes the architecture of a deep neural network including a convolutional neural network (CNN) such as ResNet. However, not limited to CNN, an architecture without convolutional layers such as Vision Transformer (ViT) may also be used. The pre-trained model is assumed to be trained by unsupervised learning using only normal data for training data. For example, based on a plurality of normal images acquired by the target acquisition unit 101, an unsupervised anomaly detection model may be trained. Note that the pre-trained model may use general pre-trained model methods for anomaly detection such as PaDiM (a Patch Distribution Modeling Framework for Anomaly Detection and Localization), FastFlow. Alternatively, the model may be trained with multi-dimensional tensor data during training, and it may be a multi-modal AI (Artificial Intelligence) capable of anomaly detection for any type of data. The pre-trained model may be stored in the storage unit 109, or stored in an external device and referred to when the estimation unit 102 is used.

[0012] Although not shown in the figure, the anomaly detection device 10 may have a training unit, and the training unit may generate a pre-trained model for performing anomaly detection by performing unsupervised learning of the model based on the normal images acquired by the acquisition unit. The anomaly detection result is, for example, a two-dimensional anomaly score map based on the degree of anomaly for each pixel, which is the degree of occurrence of an anomaly in the target image. The anomaly score map is a map showing the high and low values of the degree of anomaly for each pixel in a two-dimensional plane corresponding to the size of the target image.

[0013] The identification unit 103 refers to the information of falsely detected objects in the falsely detected object dictionary stored in the storage unit 109 to determine whether or not a falsely detected object exists in the target image, and if a falsely detected object exists, it identifies the region of the falsely detected object. The falsely detected object dictionary is a dictionary that stores information about falsely detected objects, which are objects that have been falsely detected as abnormal. The information about falsely detected objects includes the size of the unit region containing the falsely detected object (region size on the image, e.g., coordinate information), pixel values, etc.

[0014] The suppression unit 104 suppresses the abnormality level of the region corresponding to the falsely detected object if a falsely detected object exists in the target image, and generates a corrected abnormality level for the target data. The corrected abnormality level is, for example, a corrected abnormality score map obtained by using the target image in which the region of the falsely detected object has been identified and the abnormality score map to reduce the abnormality level for the region of the falsely detected object.

[0015] The determination unit 105 uses the corrected abnormality score map obtained from the suppression unit 104 to determine whether the target image is abnormal or normal.

[0016] The output unit 106 outputs a determination result to a display device such as a display, indicating whether the target image is abnormal or normal. Alternatively, at least one of the target image and the abnormal score map may be output along with the determination result, and the determination result and at least one of the target image and the abnormal score map may be displayed on the display device.

[0017] The user instruction acquisition unit 107 acquires user input from the user regarding the target image and the abnormal score map as user instructions.

[0018] The dictionary generation unit 108 extracts partial data from the target image to identify new falsely detected objects, in accordance with user instructions.

[0019] The storage unit 109 stores the target image, the anomaly score map of the target image, the trained model, the false detection object dictionary, and so on.

[0020] Next, an example of the operation of the anomaly detection device 10 according to the first embodiment will be explained with reference to the flowchart in Figure 2.

[0021] In step SA1, the target acquisition unit 101 acquires the target image. In step SA2, the estimation unit 102 uses a pre-trained model to perform anomaly detection, inputs the target image into the pre-trained model, and estimates an anomaly score map related to the target image.

[0022] In step SA3, the identification unit 103 refers to the false detection object dictionary and determines whether or not a false detection object exists in the target image. For example, if the template matching process finds that there is a region in the target image where the similarity to the pixel value of the false detection object is greater than or equal to a threshold, it determines that a false detection object exists and proceeds to step SA4. On the other hand, if the similarity to the pixel value of the false detection object is less than the threshold, it determines that no false detection object exists and proceeds to step SA6. It is assumed that the determination will be made for all false detection objects included in the false detection object dictionary, but false detection objects that are clearly unlikely to be included in the target image may be excluded. For example, since it is unlikely that cars, people, etc., will be included as false detection objects in inspection images of small products such as semiconductor chips, the identification unit 103 does not need to process false detection objects such as cars and people in the template matching process. Furthermore, any method that can determine whether or not a falsely detected object exists in the target image is acceptable, including not only template matching but also feature point matching.

[0023] In step SA4, the identification unit 103 obtains coordinate information for the region corresponding to the falsely detected object in the target image. In step SA5, the suppression unit 104 generates a corrected anomaly score map, which is a corrected anomaly score map, by reducing the anomaly score of the region corresponding to the falsely detected object in the target image. For example, the anomaly score of the region corresponding to the coordinate information obtained in step SA4 may be set to zero. Alternatively, the anomaly score of several surrounding pixels may be set to zero, based on the pixel of the coordinate indicated by the coordinate information obtained in step SA4. Furthermore, a two-dimensional map may be generated based on the matching degree score from the matching result of step SA3, and the anomaly score may be increased or decreased according to the score of each pixel. In other words, the higher the matching degree, the higher the score, and the higher the score, the lower the anomaly score should be set. Conversely, the lower the matching degree, the lower the score, and the lower the score, the less correction should be applied to reduce the anomaly score. Alternatively, if the score is below a threshold, the anomaly score may not be corrected.

[0024] In step SA6, the determination unit 105 determines whether or not an abnormality exists in the target image, that is, whether it is abnormal or normal, based on the corrected abnormality score map. Specifically, it calculates the abnormality score of the image from the corrected abnormality score map. The abnormality score can be defined by calculating a statistical quantity such as the maximum or average value of the corrected abnormality score map. If the abnormality score is above a threshold, the determination unit 105 determines that the target image is abnormal, and if the abnormality score is below the threshold, it determines that the target image is normal. The threshold may be set in advance by the user, or it may be determined from the trend of abnormality scores in normal data during the training of a trained model. In addition, if there are images that have been previously determined to be abnormal, the abnormality score of those images (or the average value of the abnormality scores if there are multiple abnormal data) may be used as the standard.

[0025] In step SA7, the output unit 106 outputs the determination result of the determination unit 105 to the outside. In addition, at least one of the target image, abnormal score map, and corrected abnormal score map may be output along with the determination result.

[0026] In step SA8, the determination unit 105 determines whether a user instruction has been given to identify a falsely detected object in the target image for which anomaly detection was performed. In other words, if a user instruction to identify a falsely detected object has been given, it indicates that a new false detection has occurred in the target image. If a user instruction has been given, the process proceeds to step SA9; if no user instruction has been given, the process ends.

[0027] In step SA9, the user instruction acquisition unit 107 acquires coordinate information for a partial image of a falsely detected object and the pixel values ​​of each region specified by the coordinate information, based on the user instruction. In step SA10, the dictionary generation unit 108 newly saves the coordinate information and pixel values ​​acquired in step SA9 as information about the falsely detected object in the falsely detected object dictionary. The saved falsely detected object information is used in the operation of the anomaly detection device 10 for subsequent target images.

[0028] Next, an example of identifying a falsely detected object and information stored in the falsely detected object dictionary, which are acquired by the user instruction acquisition unit 107 according to the first embodiment, will be explained with reference to Figure 3.

[0029] Figure 3 shows the target image 31 and the anomaly score map 32 for the target image. It is assumed that the coordinates of the target image 31 and the anomaly score map 32 are mapped so that a region specified in one image can specify a corresponding region in the other image. Assume that in the target image 31, the car is an anomaly object 33-1 and the person is a normal object 34-1, and in the anomaly score map 32, the region of the car 33-2 and the region of the person 34-2 are found to have a high degree of anomaly. The result that the region of the car 33-2 has a high degree of anomaly is correct, but the result that the region of the person 34-2 has a high degree of anomaly is incorrect. In this way, the user is informed of regions that are normal objects but were mistakenly detected as anomalies while looking at the target image 31 and the anomaly score map 32.

[0030] Specifically, the user checks the target image 31 and the anomaly score map 32 displayed on the screen, and is instructed to specify the area that is abnormal in the anomaly score map 32 by enclosing it with a stylus pen, finger, mouse, keyboard input, etc. Figure 3 shows an example in which the area 34-2 of the anomaly score map 32 corresponding to the normal object 34-1 in the target image 31 is displayed with a dashed line.

[0031] The user instruction acquisition unit 107 extracts a partial image 36 from the target image 31 that corresponds to the area of ​​the specified falsely detected object (in this case, the area of ​​the person 34-2). The falsely detected object is assumed to be specified by a rectangle as shown in Figure 3, for example, but any method of specification that identifies the falsely detected object is acceptable, such as circling it. Alternatively, by teaching the background area of ​​the extracted falsely detected object as the partial image 36, only the target object, i.e., the area of ​​the person in this case, may be extracted as is.

[0032] The dictionary generation unit 108 acquires coordinate information and pixel values ​​related to the partial image 36 or the extracted partial image. The dictionary generation unit 108 stores the coordinate information and pixel values ​​in the false detection object dictionary as information about the falsely detected object.

[0033] Furthermore, it is not limited to specifying the region of the falsely detected object in the anomaly score map 32; the region of the falsely detected object may also be specified in the target image 31, where the outline of the object is clearly visible. In this case, the coordinate information of the target image 31 and the anomaly score map 32 should be associated so that the region of the anomaly score map 32 corresponding to the region commanded in the target image 31 is specified. Furthermore, the system is not limited to displaying the target image 31 and the anomaly score map 32 side by side; the target image 31 and the anomaly score map 32 may be displayed as a superimposed image, and the region of the falsely detected object may be specified for the superimposed image. The superimposed image makes it easier to recognize the falsely detected object.

[0034] According to the first embodiment described above, the anomaly detection device determines whether or not a falsely detected object exists in the target image based on the falsely detected object dictionary, and if a falsely detected object exists, it generates a corrected anomaly score map that reduces the degree of anomaly in the corresponding area. By determining the degree of anomaly in the target image based on the corrected anomaly score map, the degree of anomaly in the area corresponding to the falsely detected object is corrected to be reduced, thereby suppressing false detection of objects that exist in a normal state as being abnormal, and enabling highly accurate anomaly detection.

[0035] (Second Embodiment) In the second embodiment, the localization of falsely detected objects and the suppression of false detections are performed on features extracted from the target image (e.g., a feature map), which is different from the embodiment described above.

[0036] The anomaly detection device 10 according to the second embodiment will be described with reference to the block diagram in Figure 4. The anomaly detection device 10 includes an object acquisition unit 101, an estimation unit 102, an identification unit 103, a suppression unit 104, a determination unit 105, an output unit 106, a user instruction acquisition unit 107, and a storage unit 109, as well as an extraction unit 201 and a calculation unit 202. The object acquisition unit 101, the identification unit 103, the determination unit 105, the output unit 106, and the user instruction acquisition unit 107 operate in substantially the same manner as in the first embodiment, so a detailed explanation is omitted.

[0037] The extraction unit 201 uses a pre-trained model, such as a convolutional neural network (CNN) or ResNet, which has been trained in advance for a predetermined task, to perform feature extraction on the target image and extract one or more feature maps. Note that the machine learning model used is not limited to convolutional processing; any model capable of extracting features from the input data may be used. Furthermore, the task is not limited to general classification tasks; metric learning may also be used. Using metric learning allows for the extraction of features that are more similar to those of the same object and are more robust to changes in color, posture, etc.

[0038] The calculation unit 202 calculates a representative feature map, which is a feature map that represents the normal data, from the multiple feature maps of normal data obtained when training the model using the training data. The representative feature map can be calculated by calculating the average value for each pixel of all the training data, which are the normal data. However, when generating the representative feature map, it is not limited to calculating the average value; representative statistics that represent the normal data can be calculated using other statistics such as the mode, median, or sum.

[0039] If a falsely detected object exists in the target image, the suppression unit 104 corrects the feature quantities of the region corresponding to the falsely detected object so that the degree of abnormality is suppressed, and generates corrected feature quantities for the target image. The estimation unit 102 estimates the corrected anomaly score for the corrected features using a pre-trained model that has been trained in advance to perform anomaly detection based on the input features. The storage unit 109 stores the trained model, the false positive object dictionary, and the representative feature map. The false positive object dictionary stores filters, which are feature maps related to false positive objects. The filters will be described later with reference to Figure 8.

[0040] Next, an example of the operation of the anomaly detection device 10 according to the second embodiment will be described with reference to the flowchart in Figure 5. Here, it is assumed that a representative feature map is generated during the training process of an already learned model, and that the representative feature map is stored in the storage unit 109.

[0041] In step SB1, the target acquisition unit 101 acquires the target image. In step SB2, the extraction unit 201 extracts features from the target image and generates a feature map. Since the feature map extraction process using the trained model can be performed using general convolutional operations, a detailed explanation is omitted.

[0042] In step SB3, the identification unit 103 generates a similarity map using the feature map obtained in step SA2 and the filters stored in the false detection object dictionary. An example of similarity map generation will be described later with reference to Figure 6.

[0043] In step SB4, the suppression unit 104 generates a corrected feature map that suppresses the influence of features of falsely detected objects, using the representative feature map and the similarity map. The generation of the corrected feature map will be described later with reference to Figure 7.

[0044] In step SB5, the estimation unit 102 outputs an anomaly score map of the target image by inputting a correction feature map into a pre-trained model whose task is anomaly detection. A pre-trained model trained by unsupervised learning for anomaly detection has a high probability of not responding to the features of the normal data used for training. On the other hand, in the correction feature map, the average feature amount of the normal data is added in regions similar to the features of the falsely detected object. Therefore, by using the correction feature map as input to the pre-trained model, an anomaly score map in which false detections of objects specified by the user are suppressed can be obtained.

[0045] In step SB6, the determination unit 105 determines whether or not an abnormality exists in the target image, that is, whether it is abnormal or normal, based on the abnormality score map. The determination method is the same as that used in step SA6 in Figure 2. In step SB7, the output unit 106 outputs the determination result of the determination unit 105 to the outside, similar to step SA7 in Figure 2.

[0046] In step SB8, the determination unit 105 determines, similar to step A8 in Figure 2, whether or not a user instruction has been given by the user to identify a falsely detected object in the target image for which anomaly detection was performed. If a user instruction has been given, the process proceeds to step SB9; otherwise, the process ends. In step SB9, the user instruction acquisition unit 107 acquires coordinate information relating to the feature map of the falsely detected object and the pixel values ​​of each region specified by the coordinate information, based on the user instruction.

[0047] In step SB11, the storage unit 109 stores the coordinate information and pixel value information related to the falsely detected object acquired in step SB10 as a filter into the falsely detected object dictionary.

[0048] Next, an example of generating a similarity map in the specific unit 103 will be explained with reference to Figure 6. As shown in Figure 6, a similarity map 63 is obtained by performing a two-dimensional convolution on the feature map 61 obtained by feature extraction by the extraction unit 201 and the filter 62 stored in the false detection object dictionary. Alternatively, the identification unit 103 may normalize the values ​​between 0 and 1 using a function such as a sigmoid function after the two-dimensional convolution to generate the similarity map 63.

[0049] During the 2D convolution process, padding may be applied to the similarity map 63 so that the output similarity map 63 is the same size as the feature map 61 obtained by the extraction unit 201. If there are multiple filters 62, the resulting similarity map 63 will be multi-channel, so a single-channel similarity map 63 can be generated by processing such as summing each pixel. In addition, to prevent the accumulation of small values ​​when summing, a threshold may be set, and the similarity of pixels below the threshold may be set to zero before calculating the sum. The identification unit 103 may generate the similarity map 63 by calculating the difference while shifting the filter 62, similar to template matching, instead of using a 2D convolution process.

[0050] For the sake of explanation, an example of a 5x5 pixel feature map 61 and a 3x3 filter 62 is shown, but the size (resolution) of the feature map 61 and filter 62 is not limited to these and can be any size.

[0051] Next, an example of generating a correction feature map of the suppression unit 104 according to the second embodiment will be described with reference to the conceptual diagram in Figure 7. The suppression unit 104 calculates a corrected feature map 72 by performing a weighted average processing 75 using the feature map 61 before correction, the representative feature map 71, and the similarity map 63. For example, it can be calculated using the weighted average shown in equation (1).

[0052]

number

[0053] F' is the corrected feature map 72, S is the similarity map 63 obtained by the identification unit 103, F is the feature map 61 before correction, F ave This is the representative feature map 71.

[0054] Equation (1) means that in the similarity map 63, representative features based on normal data are added to locations (pixels) where the value is large, i.e., where there is a high probability that the object is a false detection. Therefore, the value in the corresponding pixel approaches that of the normal data, making it possible to generate a corrected feature map 72 that suppresses the possibility of false detection.

[0055] Next, an example of identifying a falsely detected object and a filter stored in the falsely detected object dictionary, both acquired by the user instruction acquisition unit 107 according to the first embodiment, will be described with reference to Figure 8. Figure 8 shows the coordinate information of the target image 31 and the feature map 81 before correction, assuming that in the target image 31, the car is an abnormal object 33-1 and the person is a normal object 34-1. Similar to Figure 3, an example is shown where the region of the person 34-2 is enclosed by a dashed line. Various methods can be used by the user to specify objects and cut out regions, as shown in Figure 3.

[0056] The user instruction acquisition unit 107 uses the coordinates of the region taught as region 34-2 to extract a portion of the feature map 81 and generate partial features. Here, a partial feature map 82 is generated. Specifically, it can be calculated using, for example, equation (2) below.

[0057] [Number]

[0058] x 1, x 2, y 1, y2 are the x - coordinate and y - coordinate respectively that identify the area when cutting out the feature map 81. x1 input , x2 input , y1 input , y2 input are the rectangular coordinates instructed by the user. w input , h input are the width and height of the target image 31 respectively. w feat <00000!5> , h feat (l) are the width and height of the partial feature map 82 to be cut out from the feature map 81 in the l - th layer (l is an integer greater than or equal to 2) of the convolutional neural network of the learned model. round() is a function that rounds a decimal to an integer. Note that the layer l of the convolutional neural network for cutting out may be specified by the user as a hyperparameter, or a predetermined layer may be specified in advance.

[0059] The partial feature map 82 cut out from the feature map 81 is saved in the false detection object dictionary as a filter and is used in subsequent anomaly detection processing. <00002!5> Next, a specific example of a false detection object according to the second embodiment and another example of a filter saved in the false detection object dictionary will be described with reference to FIG. 9. In FIG. 9, an anomaly score map 91 of the target image 31 is shown. It is assumed that the region 92 with a high degree of anomaly in the anomaly score map is a region related to a false detection object.

[0061] The user instruction acquisition unit 107 acquires instruction from the user that region 92 is a falsely detected object, pixel by pixel. Specifically, the pixels of region 92 can be specified by clicking with a stylus pen or mouse. Region 92 can be specified on the anomaly score map 91 as described above, or a falsely detected object can be specified in the target image, and the corresponding region 92 can be specified in the anomaly score map 91 to which the coordinates are linked.

[0062] When pixels in region 92 are specified, the dictionary generation unit 108 generates a binary image 93 of the same size as the abnormal score map 91. Specifically, the image should be a binary image in which pixels in the specified region 94 corresponding to region 92 are set to 1 (255 in 255 gradations = white), and pixels in the other regions are set to 0 (0 in 255 gradations = black).

[0063] The dictionary generation unit 108 resizes the binary image 93 so that it is the same size as the feature map 95 of the target image 31, and obtains a resized binary image 96. For example, the binary image 93 can be adjusted to the width and height of the feature map 95, and interpolation processing such as nearest neighbor interpolation or bilinear interpolation can be performed to match the size of the feature map 95. The dictionary generation unit 108 generates a false detection object feature map 97 by calculating the element-wise product of the resized binary image 96 and the feature map 95. That is, since the part of the binary image 96 corresponding to region 94 is 1 and the other parts are 0, pixels corresponding to region 94 are extracted from the feature map 95. The dictionary generation unit 108 generates a filter 98 by cutting out the false detection object feature map 97 with a bounding rectangle or the like. The generated filter 98 is added to the false detection object dictionary.

[0064] Furthermore, if it is known that the target image is normal, the regions that are detected as abnormal in the abnormality score map 91 (regions with a high degree of abnormality) can be determined to be false detections. In this case, a binary image 93 may be generated by binarizing the abnormality score map 91 using thresholding. Specifically, when the dictionary generation unit 108 obtains information from the user that the target image is normal, it can generate a binary image 93 by thresholding the abnormality score map 91 so that pixels corresponding to region 92 are displayed in white and other pixels are displayed in black. This allows the system to automatically generate a filter for falsely detected objects simply by inputting information that the target image is normal, without the user having to specify falsely detected objects on a pixel-by-pixel basis.

[0065] According to the second embodiment described above, the anomaly detection device generates a feature map using a trained model that performs feature extraction from a target image, and generates a similarity map using the feature map and a filter related to falsely detected objects. A corrected feature map is generated using the similarity map, a representative feature map based on normal data, and the feature map, and an anomaly score map is generated from the corrected feature map using a trained model that performs anomaly detection, and it is determined whether or not an anomaly exists in the target image. This allows for the generation of a corrected feature map in which features related to falsely detected objects are suppressed in the feature map of the target image, enabling highly accurate anomaly detection.

[0066] (Third embodiment) In the third embodiment, the region of the falsely detected object is masked on the target image, thereby suppressing false detections that may occur in subsequent processing on the target image.

[0067] The block diagram of the anomaly detection device 10 according to the third embodiment is the same as that of Figure 1 or Figure 4, so its explanation is omitted.

[0068] An example of the operation of the anomaly detection device 10 according to the third embodiment will be explained with reference to the flowchart in Figure 10.

[0069] In step SC1, the target acquisition unit 101 acquires the target image. In step SC2, the identification unit 103 refers to the false detection object dictionary to determine whether or not a false detection object exists in the target image. For example, if the template matching process finds that there is a region in the target image where the similarity to the pixel value of the false detection object is greater than or equal to a threshold, it is determined that a false detection object exists and the process proceeds to step SC3. On the other hand, if the similarity to the pixel value of the false detection object is less than the threshold, it is determined that no false detection object exists and the process proceeds to step SC4.

[0070] In step SC3, the suppression unit 104 generates corrected data for the target image by correcting the area corresponding to the falsely detected object in the target image so that it is not falsely detected. For example, mask processing can be performed. Specifically, the suppression unit 104 can correct the area corresponding to the falsely detected object in the target image to a single color pixel value such as black or white and fill it in. Alternatively, the suppression unit 104 may correct the area corresponding to the falsely detected object in the target image by pasting a part of the normal image over it.

[0071] In step SC4, the estimation unit 102 uses a trained model to perform anomaly detection, inputs the masked target image into the trained model, and estimates an anomaly score map for the masked target image.

[0072] In step SC5, the determination unit 105 determines whether or not an abnormality exists in the target image, that is, whether it is abnormal or normal, based on the abnormality score map. The subsequent processing from step SA7 to step SA10 is the same as in the first embodiment.

[0073] According to the third embodiment described above, the anomaly detection device determines whether or not a falsely detected object exists in the target image based on the falsely detected object dictionary, and if a falsely detected object exists, it performs a masking process on the corresponding region so that the region is not judged as abnormal. An anomaly score map is generated for the target image after the masking process, and the degree of abnormality of the target image is determined based on the anomaly score map. As a result, since the area corresponding to the falsely detected object on the target image is masked in such a way that the degree of abnormality is reduced, it is possible to suppress false detection of objects that exist in a normal state as abnormal, and to perform highly accurate anomaly detection. Next, an example of the hardware configuration of the anomaly detection device 10 according to the above embodiment is shown in the block diagram of Figure 11.

[0074] The anomaly detection device 10 includes a CPU (Central Processing Unit) 1101, RAM (Random Access Memory) 1102, ROM (Read Only Memory) 1103, storage 1104, display device 1105, input device 1106, and communication device 1107, all of which are connected by a bus.

[0075] The CPU 1101 is a processor that performs arithmetic and control processing according to a program. The CPU 1101 uses a predetermined area of ​​the RAM 1102 as a working area and, in cooperation with programs stored in the ROM 1103 and storage 1104, performs the processing of each part of the abnormality detection device 10 described above.

[0076] RAM1102 is a type of memory such as SDRAM (Synchronous Dynamic Random Access Memory). RAM1102 functions as a workspace for CPU1101. ROM1103 is a memory that stores programs and various information in a non-rewritable format.

[0077] The storage device 1104 is a device that writes and reads data to and from magnetic recording media such as HDDs (Hard Disk Drives), semiconductor storage media such as flash memory, or magnetically recordable storage media such as HDDs, or optically recordable storage media. The storage device 1104 writes and reads data to and from the storage media in response to control from the CPU 1101.

[0078] The display device 1105 is a display device such as an LCD (Liquid Crystal Display). The display device 1105 displays various information based on display signals from the CPU 1101.

[0079] The input device 1106 is an input device such as a mouse or keyboard. The input device 1106 receives information input by the user as an instruction signal and outputs the instruction signal to the CPU 1101.

[0080] The communication device 1107 communicates with external devices via a network in response to control from the CPU 1101.

[0081] The instructions shown in the processing procedure described in the above-described embodiment can be executed based on a software program. A general-purpose computer system can store this program in advance and, by reading this program, can obtain effects similar to those of the control operation of the anomaly detection device described above. The instructions described in the above-described embodiment are recorded as a program that can be executed by a computer on a magnetic disk (flexible disk, hard disk, etc.), optical disk (CD-ROM, CD-R, CD-RW, DVD-ROM, DVD±R, DVD±RW, Blu-ray® Disc, etc.), semiconductor memory, or similar recording medium. Any storage format is acceptable as long as it is a recording medium that can be read by a computer or embedded system. The computer can read the program from this recording medium and, based on this program, have the CPU execute the instructions described in the program, thereby achieving operation similar to the control of the anomaly detection device in the above-described embodiment. Of course, when the computer acquires or reads the program, it may do so via a network. Furthermore, an operating system (OS) running on a computer, a database management software, a network, or other middleware (MW) operating on a computer, based on instructions from a program installed on a computer or embedded system from a recording medium, may execute some of the processes necessary to realize this embodiment. Furthermore, the recording medium in this embodiment is not limited to a medium independent of the computer or embedded system, but also includes a recording medium that stores or temporarily stores a program that has been downloaded via a LAN, the Internet, or the like. Furthermore, the recording medium is not limited to one; even when the processing in this embodiment is performed from multiple media, these are also included as recording media in this embodiment, and the configuration of the media may be any configuration.

[0082] In this embodiment, the computer or embedded system is used to execute each process in this embodiment based on a program stored on a recording medium, and may be configured as any of the following: a single device such as a personal computer or microcontroller, or a system in which multiple devices are connected via a network. Furthermore, the term "computer" in this embodiment is not limited to personal computers, but also includes arithmetic processing units, microcontrollers, and the like included in information processing equipment, and refers collectively to any equipment or device capable of realizing the functions of this embodiment through a program.

[0083] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0084] 10...Anomaly detection device, 31...Target image, 32...Anomaly score map, 33, 33-2, 34, 34-2, 94...Region, 33-1...Anomaly object, 34-1...Normal object, 36...Partial image, 61...Feature map, 62...Filter, 63...Similarity map, 71...Representative feature map, 72...Corrected feature map, 75...Weighted average processing, 81...Feature map, 82...Partial feature map, 91...Anomaly score map, 93, 96...Binary image, 95...Feature map, 97 ...False detection object feature map, 98...filter, 101...target acquisition unit, 101...acquisition unit, 102...estimation unit, 103...specification unit, 104...suppression unit, 105...determination unit, 106...output unit, 107...user instruction acquisition unit, 108...dictionary generation unit, 109...storage unit, 201...extraction unit, 202...calculation unit, 1101...CPU, 1102...RAM, 1103...ROM, 1104...storage, 1105...display device, 1106...input device, 1107...communication device.

Claims

1. A first acquisition unit acquires target data that is subject to anomaly detection, An estimation unit that estimates the degree of anomaly in the target data using a pre-trained model that has been trained in advance to perform anomaly detection on the input data, An identification unit that refers to a falsely detected object dictionary containing information about falsely detected objects and determines whether or not the falsely detected object exists in the target data, If the target data contains the falsely detected object, the suppression unit suppresses the degree of abnormality of the region corresponding to the falsely detected object and generates a corrected degree of abnormality for the target data. A determination unit that determines whether the target data is abnormal or normal based on the correction abnormality degree, An anomaly detection device equipped with the following:

2. The anomaly detection device according to claim 1, wherein the suppression unit generates the corrected anomaly degree by reducing the value of the anomaly degree of the region corresponding to the falsely detected object among the anomaly degree of the target data.

3. A second acquisition unit that acquires user instructions to identify new falsely detected objects, A generation unit that extracts partial data from the target data to identify the new falsely detected object in response to the user instruction, An anomaly detection device according to claim 1, further comprising:

4. The anomaly detection device according to claim 3, wherein the generation unit adds the partial data to the false detection object dictionary.

5. A first acquisition unit acquires target data that is subject to anomaly detection, An extraction unit that extracts feature quantities from the target data using a first trained model that performs feature extraction processing, An identification unit that refers to a falsely detected object dictionary containing information about falsely detected objects and determines whether or not the falsely detected object exists in the target data, If the target data contains the falsely detected object, the suppression unit corrects the feature quantities of the region corresponding to the falsely detected object so as to suppress the degree of abnormality, and generates corrected feature quantities for the target data. An estimation unit that estimates the degree of anomaly for the corrected features using a second pre-trained model that has been pre-trained to perform anomaly detection based on the input features, A determination unit that determines whether the target data is abnormal or normal based on the degree of abnormality, An anomaly detection device equipped with the following:

6. The extraction unit extracts feature quantities from multiple normal data, The anomaly detection device according to claim 5, further comprising a generation unit that generates a representative feature representing the feature quantity that is representative of the plurality of normal data using the feature quantities of the plurality of normal data.

7. The anomaly detection device according to claim 6, wherein the suppression unit generates the corrected feature quantity by correcting the region in which the feature quantity of the target data and the feature quantity of the falsely detected object are similar using the representative feature.

8. A second acquisition unit that acquires user instructions to identify new falsely detected objects, A generation unit that extracts a partial feature quantity from the feature quantity to identify the new falsely detected object in response to the user instruction, The abnormality detection device according to claim 5, further comprising:

9. A second acquisition unit that acquires user instructions to identify new falsely detected objects, A generation unit generates binary data by binarizing the first region corresponding to the new falsely detected object and the region other than the first region of the correction anomaly degree in response to the user instruction, and extracts a partial feature quantity that identifies the new falsely detected object based on the product of the binary data and the feature quantity. The abnormality detection device according to claim 5, further comprising:

10. A second acquisition unit that acquires a user instruction indicating that the target data is normal, When the user instruction is received, the generation unit generates binary data by binarizing the correction anomaly degree using thresholding, and extracts a partial feature quantity that identifies the new falsely detected object based on the product of the binary data and the feature quantity. The abnormality detection device according to claim 5, further comprising:

11. A first acquisition unit acquires target data that is subject to anomaly detection, An identification unit that refers to a falsely detected object dictionary containing information about falsely detected objects and determines whether or not the falsely detected object exists in the target data, If the target data contains the falsely detected object, the suppression unit generates corrected target data by correcting the target data so that the region corresponding to the falsely detected object is not falsely detected. An estimation unit that estimates the degree of anomaly in the data to be corrected using a pre-trained model that has been trained in advance to perform anomaly detection on the input data, A determination unit that determines whether the target data is abnormal or normal based on the degree of abnormality, An anomaly detection device equipped with the following:

12. The first acquisition means acquires the target data that is subject to anomaly detection, The estimation means estimates the degree of anomaly in the target data using a pre-trained model that has been trained in advance to perform anomaly detection on the input data. The identification means refers to a false detection object dictionary containing information about false detection objects, and determines whether or not the false detection object exists in the target data. If the suppression means contains the falsely detected object in the target data, it suppresses the degree of abnormality of the region corresponding to the falsely detected object and generates a corrected degree of abnormality for the target data. An anomaly detection method in which a determination means determines whether the target data is abnormal or normal based on the correction anomaly degree.

13. Computers, A first acquisition means for acquiring target data that is subject to anomaly detection, An estimation means for estimating the degree of anomaly in the target data using a pre-trained model that has been trained in advance to perform anomaly detection on the input data, A means for identifying whether or not a falsely detected object exists in the target data by referring to a falsely detected object dictionary which contains information about falsely detected objects, If the target data contains the falsely detected object, the suppression means suppresses the degree of abnormality of the region corresponding to the falsely detected object and generates a corrected degree of abnormality for the target data. An anomaly detection program that functions as a determination means for determining whether the target data is abnormal or normal based on the correction anomaly degree.

Images