Anomaly detection system and anomaly detection method

Abstract

To provide an abnormality detection system that suppresses occurrence of excessive detection and improves abnormality detection accuracy in a method for detecting abnormality using an inspection target image and a restored image, and an abnormality detection method.SOLUTION: An abnormality detection system includes: an acquisition section that acquires an inspection target image of an inspection target; a restored image generation section that inputs the inspection target image to a learning model learned using an image of the inspection target, which is normal, as teacher data to generate a restored image; and a detection section that determines whether or not the inspection target in the inspection target image is abnormal using the restored image and a normal restored image that is a restored image generated by inputting the image of the inspection target, which is normal, to the learning model.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an anomaly detection system and an anomaly detection method. [Background technology]

[0002] Conventionally, there is a method for detecting anomalies in an inspection object by image analysis, in which images of non-defective products without anomalies and images of defective products containing anomalies are trained to generate a model, and the model is used to detect anomalies in the inspection object. However, depending on the inspection object, the frequency of occurrence of defective products may be low, and images of defective products may not be collected sufficiently.

[0003] Therefore, for example, Patent Document 1 discloses a technology in which a neural network trained using images of non-defective products as learning images generates a restored image (pseudo image) from an image of the inspection object, and detects the presence of an abnormality in the inspection object based on the difference between the image of the inspection object and the restored image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-160997 A Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the inventors discovered a problem with this method of detecting anomalies using the difference between the image to be inspected and the restored image: it is prone to so-called overdetection, in which an object to be inspected that is actually a good product is detected as abnormal.

[0006] For example, when inspecting images of inspection targets that have variation in shape, even among good products, such as weld marks formed by welding, or when inspecting images of inspection targets where there is deviation or variation in the position or orientation of the inspection target within the inspection target image due to problems with the imaging environment of the inspection target, these variations can cause inspection targets that are actually good to be detected as abnormal, resulting in overdetection.

[0007] The present invention has been made in consideration of the above circumstances, and provides an anomaly detection system and an anomaly detection method that suppress the occurrence of overdetection and improve the accuracy of anomaly detection. [Means for solving the problem]

[0008] An anomaly detection system according to one aspect of the present invention includes: an acquisition unit for acquiring an inspection object image of an inspection object; a restored image generating unit that inputs the image of the test object to a learning model that has been trained using images of the test object that are normal as training data, and generates a restored image; a detection unit that uses the restored image and a normal restored image that is a restored image generated by inputting an image of the inspection target that is normal to the learning model to determine whether or not the inspection target in the inspection target image is abnormal; It is equipped with the following.

[0009] With this configuration, in the case of an inspection object in which overdetection occurred due to variations in brightness, etc., as in the conventional method of detecting abnormalities using the difference between the image of the inspection object and the restored image, the restored image generated from the image of the inspection object and the normally restored image generated from the image of the inspection object, which is normal, are similar images, thereby suppressing the occurrence of overdetection and improving the accuracy of abnormality detection.

[0010] In the anomaly detection system of the above aspect, the detection unit may determine whether the inspection object in the inspection object image is abnormal or not by using the similarity between the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection object that is normal into the learning model.

[0011] With this configuration, it is possible to determine with higher accuracy whether or not an abnormality has occurred, thereby suppressing the occurrence of overdetection and improving the accuracy of detecting an abnormality.

[0012] The anomaly detection system of the above aspect may further include a preliminary detection unit that uses a difference between the inspection target image and the restored image to determine whether or not an inspection target in the inspection target image is abnormal, and the detection unit may re-determine whether or not an inspection target determined to be abnormal by the preliminary detection unit is abnormal.

[0013] With this configuration, the preliminary detection unit performs a judgment using differences that require less calculation processing, and the detection unit re-evaluates only the inspection objects that are judged to be abnormal, thereby shortening the time required for abnormality processing.

[0014] In the anomaly detection system of the above aspect, the learning model may be a learning model using a generative adversarial network.

[0015] With this configuration, it is possible to improve the reproducibility of the restored image generated by the restored image generating unit, and to improve the accuracy of detecting anomalies.

[0016] In the anomaly detection system according to the above aspect, the inspection object may be a weld mark formed by welding on a member.

[0017] With this configuration, it is possible to suppress the occurrence of overdetection and improve the accuracy of abnormality detection, particularly in the detection of abnormalities in welding marks, which are prone to overdetection due to variations in shape, etc.

[0018] In the anomaly detection system according to the above aspect, the inspection object image may be an image captured on a production line.

[0019] With this configuration, it is possible to reduce the occurrence of overdetection and improve the accuracy of anomaly detection, particularly in anomaly detection using images captured on a production line where overdetection is likely to occur due to misalignment or variation in the position or orientation of the inspection object.

[0020] An anomaly detection method according to one aspect of the present invention includes: An acquisition step of acquiring an inspection object image of an inspection object; a restored image generating step of inputting the inspection target image into a learning model trained using a normal inspection target image as training data, and generating a restored image; a detection step of determining whether or not the inspection target in the inspection target image is abnormal using the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection target that is normal to the learning model; It is equipped with the following. Effect of the Invention

[0021] The present invention can provide an anomaly detection system and an anomaly detection method that suppresses the occurrence of overdetection and improves the accuracy of anomaly detection. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 is a block diagram showing an outline of a hardware configuration of an anomaly detection system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a functional configuration of an anomaly detection system according to a first embodiment. [Diagram 3] FIG. 4 is a flow diagram showing an anomaly detection process of the anomaly detection system according to the first embodiment. [Figure 4] 1A to 1C are explanatory diagrams showing a captured image, a restored image, and a difference image of a normal inspection target without variation in brightness, etc. [Diagram 5] 1A to 1C are explanatory diagrams showing a schematic diagram of a captured image, a restored image, and a difference image of a normal inspection target, although there is variation in brightness, etc. [Figure 6]1A to 1C are explanatory diagrams showing a captured image, a restored image, and a difference image of an inspection object that includes a defective portion and is abnormal. [Figure 7] FIG. 2 is an explanatory diagram illustrating a restored image of an object to be inspected. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. However, the present invention is not limited to the following embodiments. In addition, the following description and drawings are appropriately simplified for clarity of explanation.

[0024] <First embodiment> First, a description will be given of an anomaly detection system according to a first embodiment of the present invention. Fig. 1 is a block diagram showing an outline of a hardware configuration of the anomaly detection system according to the present embodiment.

[0025] As shown in Fig. 1, the anomaly detection system 100 has computer resources that a general information processing device such as a personal computer has. Specifically, the system includes a central processing unit (CPU) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a hard disk drive (HDD) 104, a communication interface (I / F) 105, and an input / output interface (I / F) 106. These components are connected to each other so as to be able to communicate with each other via a data bus 107. Note that, although an example in which each component of the anomaly detection system 100 is realized by a general information processing device will be described here, some or all of these functions may be realized in an external device such as an edge or a cloud.

[0026] The CPU 101 is a microprocessor that performs overall control of the anomaly detection system 100. Specifically, the CPU 101 reads out various control programs executed in this embodiment stored in the ROM 102 and the HDD 104, and executes these programs deployed on the RAM 103. Here, instead of or in addition to the HDD 104, an SSD (Solid State Drive) may be provided as a storage device.

[0027] The communication interface 105 communicates between the anomaly detection system 100 and an external device. In this embodiment, an image of an inspection target is received from an external device such as an imaging device that captures an image of the inspection target. Here, the communication is realized by various communication technologies, regardless of whether it is wireless or wired.

[0028] The input / output interface 106 performs input / output between the anomaly detection system 100 and the outside. For example, an output device such as a display that displays information about anomaly detection, and an input device for use by an operator may be provided.

[0029] Next, a functional configuration of the anomaly detection system according to this embodiment will be described with reference to Fig. 2, which is a block diagram showing the functional configuration of the anomaly detection system according to this embodiment.

[0030] 2, the anomaly detection system 100 includes, as its functional components, an acquisition unit 201, a restored image generation unit 202, a preliminary detection unit 203, and a detection unit 204. These units function when a CPU 101 executes various control programs stored in a ROM 102 or the like. Some or all of the functions of the anomaly detection system 100 may be realized by a hardware circuit.

[0031] The acquisition unit 201 acquires a captured image from an external device. Specifically, when detecting an abnormality in an inspection target, an inspection target image to be inspected and a normal image of the inspection target that is normal are acquired. Here, the normal image is an image of an inspection target that has been confirmed to have no abnormality. The acquired inspection target image is output to the restored image generation unit 202 and the preliminary detection unit 203, and the acquired normal image is output to the restored image generation unit 202. Note that, although an example in which the captured image is acquired from an imaging device that is an external device will be described here, the configuration may be realized in which the imaging device is included in the anomaly detection system.

[0032] The restored image generating unit 202 generates a restored image by restoring the inspection target image and the normal image output from the acquiring unit 201 to a learning model that has been trained in advance using the normal image as teacher data. A learning method of this learning model will be described in detail later. The restored image by restoring the inspection target image is output to a preliminary detection unit, and the normally restored image by restoring the normal image is output to a detection unit.

[0033] The preliminary detection unit 203 judges whether the inspection object is abnormal by using the difference between the inspection object image output from the acquisition unit 201 and the restored image output from the restored image generation unit 202. Specifically, first, a difference image is generated by subtracting pixel values ​​of the restored image from pixel values ​​of the inspection object image. Next, it is judged whether the sum of pixel values ​​included in the generated difference image is greater than a predetermined threshold value, and if it is greater, it is judged to be abnormal. Then, the restored image of the inspection object judged to be abnormal is output to the detection unit. Here, a difference image is generated by subtracting pixels of the restored image at corresponding positions for each pixel of the inspection object image, but the difference image may be generated using the difference for each group including a plurality of pixels.

[0034] The detection unit 204 re-determines whether the inspection target is abnormal or not by using the restored image of the inspection target determined to be abnormal by the preliminary detection unit 203 and outputted, and the normal restored image which is a restored image obtained by restoring the normal image outputted from the restored image generation unit 202. Specifically, the similarity between these images is calculated first. For example, the similarity can be calculated by various similarity calculation methods such as normalized correlation method (NCC) and shape pattern matching which calculates similarity based on edge features. It is determined whether the calculated similarity is lower than a predetermined threshold value, and if it is lower, it is determined to be abnormal. Here, the similarity between the restored image of the inspection target and the normal restored image which is a restored image obtained by restoring a normal image is calculated, and whether the inspection target is abnormal or not is determined. However, similar to the method by the preliminary detection unit, a difference image may be generated from these images, and whether the inspection target is abnormal or not may be determined from the pixel value included in the difference image. In this way, the detection unit re-determines whether the inspection target is abnormal or not, and information on the abnormality detection is outputted to the input / output interface 106.

[0035] Comparing the anomaly determination methods of the preliminary detection unit 203 and the detection unit 204 of this embodiment, the method of determining anomaly by the preliminary detection unit 203 can determine anomaly simply by taking the difference between the images to be compared, and therefore, although the accuracy of the determination is lower than the method of calculating the similarity by the detection unit 204, less calculation processing is required and the processing time is shorter. Therefore, in this embodiment, the preliminary detection unit 203 determines in advance whether or not the inspection object is abnormal, and the detection unit 204 re-determines only those inspection objects determined to be abnormal, thereby making it possible to reduce the overall time required for inspection while suppressing overdetection.

[0036] Further, although an example in which the functional configuration of the anomaly detection system 100 includes the preliminary detection unit 203 has been described here, the functional configuration may have a configuration that does not include the preliminary detection unit 203. In this case, the restored image of the inspection object generated by the restored image generation unit 202 and the normally restored image obtained by restoring the normal image are directly output to the detection unit 204. Then, the detection unit 204 determines whether or not the inspection object is abnormal using the restored image of the inspection object and the normally restored image obtained by restoring the normal image.

[0037] Next, a learning method of the learning model according to this embodiment will be described. In this embodiment, as a preliminary step before the anomaly detection process is performed, learning of a neural network, which is a learning model of the restored image generating unit 202, is performed.

[0038] When learning a neural network, the acquisition unit 201 acquires normal images, which are captured images of a normal inspection target, as teacher data. Here, the acquisition unit 201 may acquire images of the inspection target including an abnormality as long as the number of normal images acquired as teacher data is a small number equal to or less than a preset ratio.

[0039] Using the acquired teacher data, a generative adversarial network (GAN), which is a learning model of the restored image generating unit 202, is trained. Specifically, the restored image generating unit 202 includes a generator having a neural network that generates pseudo data from the acquired teacher data, and a discriminator having a neural network that judges whether the pseudo data is true or false, and the generator and the discriminator are trained using the teacher data and the pseudo data generated by the generator. In this way, by using the generative adversarial network, the reproducibility of the restored image can be improved. In this embodiment, a restored image is generated by inputting an image to be inspected to the learning model trained in this way.

[0040] Next, the anomaly detection processing flow of the anomaly detection system 100 according to this embodiment will be described. Fig. 3 is a flow diagram of the anomaly detection processing according to this embodiment. This flow starts when the learning of the learning model described above is completed. In addition, it is preferable that a normally restored image obtained by restoring a normal image is stored in the anomaly detection system 100 in advance.

[0041] In this description, we will use diagrams (hereinafter referred to as images) that show the images in Figures 4 to 7 as appropriate, and take as an example a weld mark formed by welding on various members such as parts as an inspection target. Specifically, Figure 4(a) is an image of a normal weld mark with no variation in brightness, etc., Figure 5(a) is an image of a normal weld mark including a portion 5A that is locally different in color due to variation in brightness, etc., and Figure 6(a) is an image of an abnormal weld mark including a defective portion 6A.

[0042] First, in S11, the acquisition unit 201 acquires an inspection object image such as those shown in FIG. 4(a), FIG. 5(a), or FIG. 6(a) captured by an external device.

[0043] Next, in S12, the restored image generating unit 202 inputs the inspection target image output from the acquiring unit 201 into a learning model that has been trained in advance, thereby generating restored images by restoring these captured images. Specifically, Fig. 4(b), Fig. 5(b), and Fig. 6(b) are restored images generated by inputting the images in Fig. 4(a), Fig. 5(a), and Fig. 6(a) into the learning model, respectively. Here, as described above, since this learning model is a learning model that has been trained using normal images as teacher data, the input captured image is restored as an image close to a normal image. Therefore, the part 5A with a locally different color and the defective part 6A confirmed in the images in Fig. 5(a) and Fig. 6(a) are not restored in the restored images in Fig. 5(b) and Fig. 6(b), respectively. On the other hand, the image in FIG. 5(a) is an image of a normal weld mark just like the image in FIG. 4(a) except for including a locally different colored area 5A, so the restored image in FIG. 5(b) is relatively close to the restored image in FIG. 4(b) obtained by restoring the image in FIG. 4(a). However, the image in FIG. 6(a) includes a defective area 6A, so the image features change and the restored image in FIG. 6(b) is a blurry image.

[0044] Next, in S13, the preliminary detection unit 203 judges whether the inspection object is abnormal or not by using the difference between the captured image and the restored image. Specifically, first, the difference images shown in Figs. 4(c), 5(c), and 6(c) are generated by the difference between the captured images shown in Figs. 4(a), 5(a), and 6(a) and the restored images shown in Figs. 4(b), 5(b), and 6(b). Here, Fig. 4(c), which is a difference image of a normal weld mark without variation in brightness, etc., has a small difference because Figs. 4(a) and 4(b) are similar images. On the other hand, the difference images shown in Figs. 5(c) and 6(c) have a large difference because the part 5A with a locally different color and the defective part 6A are not restored in the restored images of Figs. 5(b) and 6(b), as described above. Next, it is judged whether the difference is larger than a predetermined threshold value. If the difference is smaller than the predetermined threshold, as in the difference image shown in Fig. 4(c), the preliminary detection unit 203 determines that the inspection object is normal, and this flow ends. On the other hand, if the difference is larger than the predetermined threshold, as in the difference images shown in Fig. 5(c) and Fig. 6(c), the process proceeds to S14.

[0045] Here, although the captured image shown in FIG. 5(a) is an image of a normal weld mark, the inspection object is not judged to be normal in S13, i.e., in the judgment using the difference between the inspection object image and the restored image. In other words, if the inspection object image has variation in brightness or shape, the normal images used for learning the learning model cannot cover all normal images, so even normal images cannot be properly restored, and as a result, the inspection object is detected to be abnormal based on the difference, which is a so-called overdetection problem. On the other hand, in the captured image shown in FIG. 4(a), the inspection object is judged to be normal in S13, making it unnecessary for the detection unit 204 to perform re-judgment, and the processing time required for the abnormality detection process can be shortened.

[0046] Next, in S14, the detection unit 204 re-determines whether the inspection target is abnormal or not by using the restored image of the inspection target in which an abnormality was detected by the preliminary detection unit 203 and the normal restored image generated by the restored image generating unit 202 or previously stored in the abnormality detection system 100, which is a restored normal image. Specifically, the similarity between these images is calculated first. FIG. 7(a) shows a normal restored image restored from an image of a normal welding mark, FIG. 7(b) shows a restored image of the inspection target shown in FIG. 5(b), and FIG. 7(c) shows a restored image of the inspection target shown in FIG. 6(b). Here, FIG. 7(b), which is a restored image of an image of a welding mark including a portion 5A that is locally different in color due to variations in brightness, etc., is a restored image of an image of a normal welding mark except for the portion 5A that is locally different in color, as described above, and therefore shows a high similarity to FIG. 7(a), which is a normally restored image. On the other hand, FIG. 7(c), which is a restored image of an image of an abnormal welding mark including a defective portion 6A, is a blurred image, and therefore shows a low similarity to FIG. 7(a), which is a normal restored image. Next, it is determined whether the calculated similarity is lower than a predetermined threshold value. When the similarity is high, as in the restored image of the inspection target shown in FIG. 7(b), the detection unit 204 determines that the inspection target is normal, and this flow ends. On the other hand, when the similarity is low, as in the restored image of the inspection target shown in FIG. 7(c), the detection unit 204 determines that the inspection target is abnormal, and this flow ends. Although omitted in this flow, a process of outputting information on the abnormality detection to the input / output interface 106 may be added.

[0047] In addition, although an example including S13 has been described here as an anomaly detection processing flow of the anomaly detection system 100, an anomaly detection processing flow not including S13 may be used. In that case, after a restored image is generated in S12, the process proceeds directly to S14. Then, in S14, the detection unit 204 uses the restored image of the inspection object and a normal restored image obtained by restoring a normal image to determine whether or not the inspection object is abnormal. In this anomaly detection processing flow, when an anomaly detection process is performed on the inspection object in the captured images shown in FIG. 4(a), FIG. 5(a), and FIG. 6(a) in the same manner, since S13 is not included, in S14, the inspection object in the captured images shown in FIG. 4(a) and FIG. 5(a) is determined to be normal, and the inspection object in the captured image shown in FIG. 6(a) is determined to be abnormal.

[0048] As described above, the anomaly detection process flow of the anomaly detection system 100 according to this embodiment judges that there is no abnormality in the inspection target of a welding mark that is normal with no variation in brightness, etc., as shown in Fig. 4(a) and that is normal but includes a portion that locally differs in color due to variation in brightness, etc., as shown in Fig. 5(a), while judging that there is no abnormality in the inspection target of a welding mark that includes a defective portion and is abnormal, as shown in Fig. 6(a). That is, even if the above-mentioned problem of overdetection occurs due to variation in brightness or shape in the abnormality judgment (S13) by the preliminary detection unit 203, the occurrence of overdetection can be suppressed in the abnormality judgment (S14) by the detection unit 204, and the accuracy of anomaly detection can be improved. Here, the anomaly detection process of the anomaly detection system 100 according to this embodiment is not limited to a particular inspection object, but is particularly suitable for the case where the inspection object is a welding mark which is prone to variations in shape and brightness, or for the case where an image of the inspection object is captured on a production line where deviations and variations are likely to occur in the position and orientation of the inspection object in the inspection object image due to problems in the imaging environment.

[0049] <Other embodiments> In the first embodiment, a generative adversarial network, specifically, a GAN, has been described as an example of a learning model possessed by the restored image generating unit 202, but various generative adversarial networks such as CGAN, DCGAN, SRGAN, CycleGAN, and VAEGAN can be used in addition to GAN. Also, various autoencoders such as AE, VAE, and CVAE may be used in addition to GAN.

[0050] The present invention is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit and scope of the present invention. For example, in addition to the anomaly detection system, the present invention can be realized in various forms, such as an anomaly detection device, an anomaly detection method, and a computer program for implementing the method. [Explanation of symbols]

[0051] 100 Anomaly Detection System 101 CPU 102 ROM 103 RAM 104 HDD 105 Communication I / F 106 Input / Output Interface 107 Data Bus 201 Acquisition Department 202 Restored image generation unit 203 Preliminary detection section 204 Detection unit

Claims

1. an acquisition unit for acquiring an inspection object image of an inspection object; a restored image generating unit that inputs the image to be inspected into a learning model that has been trained using images of the inspection object that are normal as training data, and generates a restored image; a preliminary detection unit that uses a difference between the inspection object image and the restored image to determine whether or not an inspection object in the inspection object image is abnormal; a detection unit that, when the preliminary detection unit determines that the inspection target in the inspection target image is abnormal, determines whether or not the inspection target in the inspection target image is abnormal by using a similarity between the restored image and a normal restored image that is a restored image generated by inputting an image of the inspection target that is normal to the learning model; An anomaly detection system comprising:

2. The learning model is a learning model using an adversarial generative network. The anomaly detection system according to claim 1 .

3. The inspection object is a weld mark formed on a member by welding. The anomaly detection system according to claim 1 or 2.

4. The inspection object image is an image captured on a production line. The anomaly detection system according to any one of claims 1 to 3.

5. An acquisition step of acquiring an inspection object image of an inspection object; a restored image generating step of inputting the inspection target image into a learning model trained using a normal inspection target image as training data, and generating a restored image; a preliminary detection step of determining whether or not an object to be inspected in the image to be inspected is abnormal by using a difference between the image to be inspected and the restored image; a detection step of judging whether or not the inspection object in the inspection object image is abnormal by using a similarity between the restored image and a normal restored image, which is a restored image generated by inputting an image of the inspection object that is normal to the learning model, when the inspection object in the inspection object image is judged to be abnormal in the preliminary detection step; The anomaly detection method includes:

Images