Anomaly detection method and anomaly detection apparatus

Abstract

In the technology provided by the present application, in the abnormality detection method, the normal inspection object can be reduced to be detected as an abnormality, and the accuracy of the abnormality detection can be improved. The abnormality detection method obtains a photographed image of an inspection object, inputs the photographed image into a first learning model to generate a restored image, the first learning model is learned by using normal images of normal inspection objects, a difference image of the photographed image and the restored image is generated, the generated difference image is input into a second learning model to generate a restored difference image, the second learning model is learned by using normal difference images, and the abnormality of the inspection object is detected by using the difference image and the restored difference image.

Description

TECHNICAL FIELD

[0001] The present application relates to an abnormality detection method and an abnormality detection apparatus. BACKGROUND

[0002] An image processing apparatus is known that detects an abnormality in an image of an inspection target in a case where a difference between a simulation image generated by a neural network and the image of the inspection target is equal to or greater than a predetermined threshold value (for example, Patent Literature 1).

[0003] PRIOR ART DOCUMENTS

[0004] PATENT LITERATURE

[0005] Patent Literature 1: Japanese Patent Application Publication No. 2020-160997 SUMMARY

[0006] PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] In the abnormality detection method using the image of the inspection target, it is desirable to further improve the detection accuracy of the abnormality of the inspection target.

[0008] METHOD FOR SOLVING THE PROBLEM

[0009] The present application can be implemented as the following modes.

[0010] (1) According to one embodiment of the present application, there is provided an abnormality detection method. The abnormality detection method acquires a captured image that captures an inspection target, generates a restored image by inputting the captured image to a first learning model that is learned using normal images that capture normal inspection targets, generates a difference image of the captured image and the restored image, generates a restored difference image by inputting the generated difference image to a second learning model that is learned using normal difference images, and detects an abnormality of the inspection target using the difference image and the restored difference image.

[0011] According to the abnormality detection method of this mode, the second learning model that is learned using the difference image of the captured image and the restored image can generate the restored difference image that extracts an over-detection element included in the difference image. Therefore, at the time of detecting the abnormality, it is possible to distinguish the over-detection element included in the difference image from the abnormality, it is possible to reduce or suppress the bad situation in which the normal inspection target is detected as the abnormality, and it is possible to improve the detection accuracy of the abnormality.

[0012] (2) In the abnormality detection method of the above-described aspect, in the detection of the abnormality of the inspection target, an abnormality determination image is generated using a difference between the difference image and the restored difference image, and the abnormality determination image is used to determine the abnormality of the inspection target.

[0013] According to the abnormality detection method of this aspect, it is possible to detect an abnormality using an image in which an over-detection element included in a difference image is removed.

[0014] (3) In the abnormality detection method of the above-described aspect, the first learning model generates the restored image with a position of the inspection target in the captured image being removed.

[0015] According to the abnormality detection method of this aspect, it is possible to generate a restored difference image in which a photographing error is learned by inputting a difference between a captured image and a restored image to a second learning model.

[0016] (4) In the abnormality detection method of the above-described aspect, the first learning model is a learning model using a generative adversarial network.

[0017] According to the abnormality detection method of this aspect, it is possible to improve reproducibility of a restored image based on the first learning model, and to improve accuracy of abnormality detection.

[0018] (5) In the abnormality detection method of the above-described aspect, the second learning model is a learning model using a generative adversarial network.

[0019] According to the abnormality detection method of this aspect, it is possible to improve reproducibility of a restored difference image based on the second learning model, and to improve accuracy of abnormality detection.

[0020] (6) In the abnormality detection method of the above-described aspect, the first learning model and the second learning model use the same kind of learning model.

[0021] According to the abnormality detection method of this aspect, compared to a case in which learning models of different kinds are used, a tendency of a photographing error included in a difference image and a photographing error included in a restored difference image is likely to be consistent. Therefore, it is possible to more accurately remove a photographing error by a difference between the difference image and the restored difference image.

[0022] The present application can be implemented in various forms other than an abnormality detection method. For example, it can be implemented as an abnormality detection device, an image processing device, a manufacturing method of an abnormality detection device, a control method of an abnormality detection device, a computer program that implements the control method, a non-transitory recording medium that records the computer program, and the like. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1is a block diagram showing the internal functional structure of the abnormality detection device.

[0024] Figure 2 is a block diagram showing an example of the functional structure of the CPU in the abnormality detection device.

[0025] Figure 3 is a flowchart showing the abnormality detection processing performed by the abnormality detection device.

[0026] Figure 4 is an explanatory diagram showing a captured image of an inspection object captured by an external device.

[0027] Figure 5 is an explanatory diagram showing an example of a restored image generated by the restored image generation section.

[0028] Figure 6 is an explanatory diagram showing an example of a difference image generated by the difference image generation section.

[0029] Figure 7 is an explanatory diagram showing an example of a restored difference image generated by the restored difference image generation section.

[0030] Figure 8 is an explanatory diagram showing an example of an abnormality judgment image generated by the abnormality judgment image generation section. DETAILED DESCRIPTION

[0031] A. First Embodiment

[0032] Figure 1 is a block diagram showing the internal functional structure of the abnormality detection device 100 as the first embodiment of the present application. The abnormality detection device 100 is a personal computer connected with an external device. The external device is, for example, a video camera that captures an image of an inspection object of the abnormality detection device 100. In the present embodiment, the image taken by the abnormality detection device 100 is constituted by an RGB input image signal composed of each image signal component represented by R (red), G (green), and B (blue). The input image signal can be, for example, a YUV image signal composed of Y (luminance signal), U (first color difference signal), and V (second color difference signal), or a YCbCr image signal or a YPbPr image signal. The image can be a gray scale image having a luminance value with an arbitrary color depth of 1 bit, 8 bits, or the like, in addition to a color image. The external device can be, for example, a manufacturing device for manufacturing an inspection object, an inspection device for inspecting an inspection object, or a video camera incorporated in these devices.

[0033] The abnormality detection device 100 has a CPU 110 as a central arithmetic processing device, a storage section 130, a transmission / reception section 120, and a display section 140. These sections are communicably connected to each other via a data bus 150. The CPU 110, the storage section 130, and the transmission / reception section 120 can communicate with each other bidirectionally. Part or all of the functions of the processing of the abnormality detection device 100 can also be implemented by an edge, a cloud, or the like. For example, the abnormality detection device 100 can acquire an image captured by an external device via a network or the like, process the acquired image by an edge, a cloud, or the like, and output the processing result to the outside via the network on this basis.

[0034] The CPU 110 is a microprocessor that comprehensively controls the abnormality detection device 100. The storage section 130 is, for example, a RAM, a ROM, or a hard disk drive (HDD) as a large-capacity storage medium. In the HDD or the ROM, various programs for realizing the functions provided in the present embodiment are stored. The CPU 110 executes various programs read out from the HDD or the ROM of the storage section 130 and loaded in the RAM. As the large-capacity storage medium possessed by the storage section 130, a solid state drive (SSD) can be possessed instead of or together with the HDD.

[0035] The transmission / reception section 120 communicates with an external device. In the present embodiment, the transmission / reception section 120 receives an image of an abnormality detection object captured by an external device via wireless communication. As the wireless communication, for example, wireless communication via a wireless local area network (LAN) conforming to the IEEE 802.11a standard in a 2.4 GHz band or a 5 GHz band, wireless communication using a sub-gigahertz band of less than 1 GHz band (916.5 MHz to 927.5 MHz), or wireless communication using Bluetooth (registered trademark) can be used. The connection between the transmission / reception section 120 and the external device is not limited to wireless connection, and can be wired connection by an Ethernet (registered trademark) or the like.

[0036] The display section 140 is a display for displaying an operation screen of the abnormality detection device 100 or result information of the abnormality detection of the abnormality detection device 100. The display section 140 can also be provided on an external device different from the abnormality detection device 100, such as an inspection device. The abnormality detection device 100 can also have an input device such as a keyboard, a mouse, or a touch panel.

[0037] Figure 2is a block diagram showing an example of a functional configuration of the CPU 110 in the abnormality detection device 100. The CPU 110 functions as the acquisition section 111, the restored image generation section 113, the difference image generation section 115, the restored difference image generation section 117, the abnormality determination image generation section 118, and the recognition section 119 by executing various control programs stored in the storage section 130. Part or all of the functions of the abnormality detection device 100 can be implemented by hardware circuitry.

[0038] The functions of each section at the time of execution of the abnormality detection processing based on the abnormality detection device 100 will be described. The acquisition section 111 acquires a captured image of an inspection object from an external device in a case where an abnormality of the inspection object is detected. The acquisition section 111 acquires an image of the same size as the image acquired as a learning image, as a captured image of the inspection object. The acquisition section 111 outputs the acquired captured image of the inspection object to the restored image generation section 113 and the difference image generation section 115.

[0039] The restored image generation section 113 generates a restored image in which a captured image is restored, by a neural network learned, at the time of acquisition of the captured image of the inspection object from the acquisition section 111. The restored image generation section 113 outputs the generated restored image to the difference image generation section 115.

[0040] The difference image generation section 115 generates a difference image of the captured image of the inspection object acquired from the acquisition section 111 and the restored image acquired from the restored image generation section 113. The difference image generation section 115 generates the difference image, for example, by subtracting the pixel value of the restored image from the pixel value of the captured image. In the present embodiment, the pixel value of the restored image at a position corresponding to each pixel of the captured image is subtracted from the pixel value of the captured image, respectively, and thereby the difference image is generated. In contrast, the difference image can also be generated using a difference for each group including a plurality of pixels. The difference image generation section 115 outputs the generated difference image to the restored difference image generation section 117 and the abnormality determination image generation section 118.

[0041] The restored difference image generation section 117 generates a restored difference image in which the acquired difference image is restored, by a neural network learned, at the time of acquisition of the difference image from the difference image generation section 115. The restored difference image generation section 117 outputs the generated restored difference image to the abnormality determination image generation section 118.

[0042] The abnormality determination image generation section 118 generates an abnormality determination image using the difference between the difference image acquired from the difference image generation section 115 and the restored difference image acquired from the restored difference image generation section 117. The abnormality determination image generation section 118 generates the abnormality determination image, for example, by subtracting the pixel value of the restored difference image from the pixel value of the difference image. In the present embodiment, the abnormality determination image is generated by subtracting the pixel of the restored difference image at a position corresponding to each pixel of the difference image, respectively. The abnormality determination image can also be generated using the difference for each group including a plurality of pixels. The abnormality determination image generation section 118 outputs the generated abnormality determination image to the recognition section 119.

[0043] The recognition section 119 detects an abnormality in the detection target using the abnormality determination image acquired from the abnormality determination image generation section 118. In the present embodiment, the recognition section 119 determines that an abnormality exists in the detection target when the total value of the pixel values included in the abnormality determination image is greater than a predetermined threshold value. The recognition section 119 outputs the abnormality determination result to the display section 140.

[0044] The learning method of the learning model possessed by the abnormality detection apparatus 100 will be described using Figure 2 The abnormality detection apparatus 100 performs learning of the neural network possessed by the restored image generation section 113 and the restored difference image generation section 117 as a preliminary preparation for performing the abnormality detection processing. The neural network possessed by the restored image generation section 113 is also referred to as a "first learning model", and the neural network possessed by the restored difference image generation section 117 is also referred to as a "second learning model".

[0045] The acquisition section 111 outputs to the restored image generation section 113 and the restored difference image generation section 117 when acquiring the learning image from the external apparatus. In the present embodiment, the acquisition section 111 acquires an image in which a normal inspection target is photographed (hereinafter, also referred to as a "normal image") as the learning image. The normal inspection target refers to an inspection target that does not have an abnormality. The acquisition section 111 can also acquire an image in which an inspection target having an abnormality is photographed (hereinafter, also referred to as an "abnormal image") in the case of a small number of less than a predetermined number, for example.

[0046] In the present embodiment, the restored image generation section 113 has a generative adversarial network (GAN) as the first learning model. Specifically, the restored image generation section 113 has a conditional GAN (CGAN). The restored image generation section 113 has a generator having a neural network that generates simulated data and a discriminator having a neural network that performs judgment of authenticity of the simulated data. When a normal image as a learning image is input to the restored image generation section 113, the generator generates a simulated image. The restored image generation section 113 causes the discriminator and the generator to learn using the normal image, the simulated image generated by the generator using the normal image.

[0047] In the present embodiment, the restored difference image generation section 117 has a CGAN as the same kind of learning model as the restored image generation section 113, and has a generator and a discriminator. In the case of learning, the normal difference image as a learning image is input from the difference image generation section 115 to the restored difference image generation section 117. The normal difference image refers to a difference image of a normal image and a restored image generated using the normal image. The restored difference image generation section 117 causes the discriminator and the generator to learn using the normal difference image, the simulated image generated by the generator using the normal difference image, at the time of learning of the second learning model. The normal difference image used for learning of the restored difference image generation section 117 can be generated using, for example, a normal image different from the normal image used for learning of the restored image generation section 113. In the learning of the restored difference image generation section 117, for example, an image prepared additionally in advance can also be used.

[0048] In this case, in the difference image between the restored image generated using the normal image and the normal image, an image indicating a photographing error included in the normal image before the restoration can be included. The photographing error refers to a portion removed from the normal image at the time of the restoration based on the first learning model, and is a factor of a defect in which a normal inspection target is detected as an abnormality. The defect in which a normal inspection target is detected as an abnormality is also referred to as over-detection. As the photographing error, for example, a detection error of a portion other than the detection target included in the photographed image, such as a deviation of a position, a direction, a deviation of a brightness such as a background, a dirt of a background, and the like of the detection target in the photographed image is included. The second learning model learns using the difference image including the photographing error. Therefore, the restoration difference image generation section 117 generates a portion removed from the input photographed image by the restoration by the first learning model, that is, an image in which a portion corresponding to the photographing error is restored, as a restoration difference image. In the learning of the restoration difference image generation section 117, it is preferable to input a plurality of difference images in which the number and the kind of the photographing errors that are factors of over-detection are extracted. According to the abnormality detection device 100 of this way, it is possible to obtain a restoration difference image in which the photographing error is made more explicit. Therefore, it is possible to improve the possibility of removing the factor of over-detection, and it is possible to improve the accuracy of the abnormality detection based on the abnormality detection device 100. In the present application, "removing the factor of over-detection" refers to removing the factor of over-detection from the image.

[0049] in combination Figure 3 and suitably used Figures 4 to 8 An abnormality detection process performed by the abnormality detection device 100 will be described. Figure 3 is a flowchart indicating an abnormality detection process performed by the abnormality detection device 100. This flow is started, for example, at the time when the power of the abnormality detection device 100 is turned on. This flow can be started by inputting the photographed image of the inspection target to the acquisition section 111, or can be started by the power of the external device connected to the abnormality detection device 100 being turned on. At the start time of this flow, the learning of the first learning model of the restoration image generation section 113 using the normal image and the learning of the second learning model of the restoration difference image generation section 117 using the normal difference image are in a completed state.

[0050] In step S10, the acquisition section 111 acquires the photographed image of the inspection target photographed by the external device. Figure 4 is an explanatory diagram indicating an example of the photographed image CP of the inspection target photographed by the external device. The photographed image CP of the inspection target is input to the acquisition section 111 from the external device via the transmission and reception section 120 through wireless communication. In the photographed image CP, the image of the product PP as an example of the inspection target (hereinafter, also referred to as "product image PP") is included. Figure 4 In the photographed image CP, the image of the product PP as an example of the inspection target (hereinafter, also referred to as "product image PP") is included. Figure 4This example illustrates the state of a foreign object (FM) adhering to a product (PP) as an anomaly to be inspected. Hereinafter, the image of the foreign object FM will also be referred to as "Foreign Object Image FM". The captured image (CP) is an example of an image of an anomaly of the inspected object. Anomalies to the inspected object include, in addition to the adhesion of foreign object FM, various other anomalies such as dirt, surface irregularities, and missing parts. Figure 4 In the example, the captured image CP includes an overall image of the product PP, which is the object of inspection, but is not limited to an overall image of the product PP; it may also include capturing any position of the product PP. The acquisition unit 111 outputs the acquired captured image CP to the restoration image generation unit 113 and the differential image generation unit 115.

[0051] In step S20, the restored image generation unit 113 obtains the image from the acquisition unit 111. Figure 4 When the image CP is captured, the restored image RS is generated using the first learned model. Figure 5 This is an explanatory diagram showing an example of the restored image RS generated by the restored image generation unit 113. As described above, the first learning model of the restored image generation unit 113 has completed the learning of a normal image using the product image PP. Therefore, as... Figure 5 As shown, the restored image RS is restored to an image close to a normal image, while the foreign object image FM is not restored. Therefore, the restored product image PRS contained in the restored image RS is generated without the foreign object image FM. At this time, the restored image RS can be restored based on the removal of shooting errors such as the positional deviation of the product PP in the captured image CP and the deviation of the background other than the product PP contained in the captured image CP. The restored image generation unit 113 outputs the generated restored image RS to the differential image generation unit 115.

[0052] In step S30, the differential image generation unit 115 uses Figure 4 The captured image CP is shown. Figure 5 The difference between the restored image RS and the image DP is generated. Figure 6 This is an explanatory diagram showing an example of a differential image DP generated by the differential image generation unit 115. The differential image DP is an image formed by subtracting the pixels of the restored image RS from the pixels of the captured image CP. For example... Figure 6 As shown, the difference image DP includes the difference anomaly image PFM and the difference extracted image PDP. The difference anomaly image PFM is related to... Figure 4The image shown corresponds to the foreign object image FM. The differential extracted image PDP is the image obtained by subtracting the pixels of the restored product image PRS from the pixels of the product image PP. In other words, the differential extracted image PDP can be described as an image that extracts the shooting error removed by the restoration image generation unit 113 during the generation of the restoration image RS. The differential image generation unit 115 outputs the generated differential image DP to the restoration differential image generation unit 117 and the anomaly judgment image generation unit 118.

[0053] In step S40, when the differential image generation unit 117 obtains the differential image DP from the differential image generation unit 115, it restores the differential image DP and generates the restored differential image DRS by using the learned second learning model. Figure 7 This is an explanatory diagram showing an example of a restored differential image DRS generated by the restored differential image generation unit 117. As described above, the second learning model of the restored differential image generation unit 117 completes learning based on a normal differential image. Therefore, the restored differential image DRS generated by the restored differential image generation unit 117 corresponds to an image that restores the shooting errors contained in the differential image DP. Therefore, as... Figure 7 As shown, the restored differential image DRS is generated as an image close to the differential image DP that does not contain the differential anomaly image PFM. The shooting error extraction image PDR is an image from the differential image DP in which shooting errors, which are elements of over-detection, are extracted, i.e., an image corresponding to the differential extraction image PDP. The restored differential image generation unit 117 outputs the generated restored differential image DRS to the anomaly detection image generation unit 118.

[0054] In step S50, the anomaly detection image generation unit 118 uses the difference between the differential image DP and the restored differential image DRS to generate an anomaly detection image FDP. Figure 8 This is an explanatory diagram showing an example of an anomaly detection image FDP generated by the anomaly detection image generation unit 118. The anomaly detection image generation unit 118 uses an image generated from... Figure 6 Subtract from the pixel values ​​of the difference image DP shown Figure 7 The difference is obtained by reconstructing the difference image DRS pixel values ​​shown, and then generating... Figure 8 The anomaly detection image FDP is shown. The anomaly detection image FDP corresponds to the image extracted from the difference image DP by removing the difference extraction image PDP corresponding to the shooting error. For example... Figure 8 As shown, the restored differential image DRS includes a restored differential anomaly image FFM. The restored differential anomaly image FFM corresponds to the differential anomaly image PFM of the differential image DP. The anomaly detection image generation unit 118 outputs the generated anomaly detection image FDP to the recognition unit 119.

[0055] In step S60, the identification section 119 judges the presence or absence of an abnormality by detecting the recovered difference abnormality image FFM from the abnormality judgment image FDP. In the present embodiment, the identification section 119 detects the recovered difference abnormality image FFM as an abnormality when the total value of the pixel values included in the abnormality judgment image FDP is greater than a predetermined threshold value, and judges that an abnormality is present in the inspection object. The identification section 119 judges that the inspection object has no abnormality when the total value of the pixel values included in the abnormality judgment image FDP is less than a predetermined threshold value. As another embodiment of the abnormality detection method using the abnormality judgment image FDP, for example, a GMM (Gaussian Mixture Model) or a DAGMM (Deep Autoencoder Gaussian Mixture Model) can be used. In this case, the identification section 119 can use a latent variable that represents an intermediate value at the time of image restoration by the recovered difference image generation section 117 in addition to the difference between the difference image DP and the recovered difference image DRS. As another detection method of the recovered difference abnormality image FFM in the abnormality judgment image FDP, for example, an abnormality can also be detected by extracting a feature point or a feature quantity corresponding to the recovered difference abnormality image FFM on the abnormality judgment image FDP. The recovered difference abnormality image FFM can also be detected by visual inspection of the image data of the abnormality judgment image FDP generated by the abnormality judgment image generation section 118 by the operator, in which case the identification section 119 can be omitted. Further, for example, an abnormality can be detected by comparing the image data of the difference image DP shown in FIG. 6 and the image data of the recovered difference image DRS shown in FIG. 7 based on the visual inspection by the operator, and an abnormality can also be detected without generating the abnormality judgment image FDP. In this case, the abnormality judgment image generation section 118 can be omitted. When the judgment of the presence or absence of an abnormality of the inspection object based on the identification section 119 ends, the identification section 119 outputs the judgment result to the display section 140, and the present flow ends. Figure 6 Figure 7

[0056] ​​As described above, according to the abnormality detection device 100 of the present embodiment, the difference image DP is generated by the difference of the captured image CP and the restored image RS generated by the first learning model that learns using normal images. The abnormality of the inspection target is detected using the generated difference image DP and the restored difference image DRS generated by the second learning model that learns using normal difference images. The restored difference image DRS restores the capturing error that is an element of over-detection by learning using normal difference images. By using the restored difference image DRS and the difference image DP, it is possible to distinguish the difference abnormal image PFM of the inspection target included in the difference image DP and the capturing error that is an element of over-detection. Therefore, when detecting abnormality, it is possible to detect abnormality on the basis of the removal of the capturing error, it is possible to reduce over-detection, and it is possible to improve the accuracy of abnormality detection.

[0057] According to the abnormality detection device 100 of the present embodiment, the recognition section 119 judges the abnormality of the inspection target on the basis of the abnormality judgment image FDP generated using the difference of the difference image DP and the restored difference image DRS. Therefore, it is possible to detect abnormality using an image in which the element of over-detection included in the difference image DP is removed.

[0058] According to the abnormality detection device 100 of the present embodiment, the first learning model generates the restored image RS by removing the capturing error of the position deviation of the inspection target included in the captured image CP. Therefore, according to the difference image DP of the restored image RS generated using the first learning model and the captured image CP, it is possible to cause the second learning model to learn the capturing error included in the captured image CP.

[0059] According to the abnormality detection device 100 of the present embodiment, the first learning model is a learning model using a generative adversarial network (GAN). Therefore, it is possible to improve the reproducibility of the restored image RS based on the first learning model, and it is possible to improve the accuracy of abnormality detection.

[0060] According to the abnormality detection device 100 of the present embodiment, the second learning model is a learning model using a generative adversarial network (GAN). Therefore, it is possible to improve the reproducibility of the restored difference image DRS based on the second learning model, and it is possible to improve the accuracy of abnormality detection.

[0061] According to the abnormality detection device 100 of the present embodiment, the first learning model of the restoration image generation section 113 and the second learning model of the restoration difference image generation section 117 use CGAN as the same kind of learning model. Therefore, the tendency of the photographing error removed from the captured image CP by the restoration image RS generated by the first learning model and the photographing error restored by the restoration difference image DRS generated by the second learning model are likely to be consistent. As a result, the tendency of the photographing error included in the difference image DP and the photographing error included in the restoration difference image DRS are likely to be consistent compared to the abnormality detection device having different kinds of learning models. Therefore, it is possible to more accurately remove the photographing error by the difference between the difference image DP and the restoration difference image DRS.

[0062] B. Other Embodiments

[0063] (B1) In the above-described first embodiment, an example in which CGAN is used in the first learning model of the restoration image generation section 113 and the second learning model of the restoration difference image generation section 117 is shown. In contrast to this, in the learning model of at least one of the restoration image generation section 113 or the restoration difference image generation section 117, a learning model other than CGAN or GAN can be used, and various Auto encoders such as AE (Auto encoder), VAE (Variational Auto encoder), and CVAE (Conditional Variational Auto encoder) can be used, and various GANs such as DCGAN (Deep Convolutional GAN), SRGAN, CycleGAN, and VAEGAN can be used.

[0064] The present application is not limited to the above-described embodiments, and can be implemented in various structures without departing from the gist thereof. For example, in order to solve part or all of the above-described problems, or in order to achieve part or all of the above-described effects, the technical features of the embodiments corresponding to the technical features in each of the modes described in the specification of the application can be appropriately replaced or combined. In addition, if the technical feature is not described as a necessary content in the specification, it can be appropriately deleted.

[0065] Explanation of Reference Signs

[0066] 100 anomaly detection device; 110 CPU; 111 acquisition unit; 113 restored image generation unit; 115 difference image generation unit; 117 restored difference image generation unit; 118 anomaly determination image generation unit; 119 recognition unit; 120 transmission / reception unit; 130 storage unit; 140 display unit; 150 data bus; CP captured image; DP difference image; DRS restored difference image; FDP anomaly determination image; FFM restored difference anomaly image; FM foreign matter image; LAN wire; PDP difference extraction image; PDR captured error extraction image; PFM difference anomaly image; PP product image; PRS restored product image; RS restored image.

Claims

1.An abnormality detection method, acquiring a captured image that captures an inspection object, generating a restored image by inputting the captured image to a first learning model that is learned using a normal image that captures a normal inspection object, the restored image being an image that is restored on the basis of removal of a capturing error and that does not have an abnormality of the inspection object, generating a difference image of the captured image and the restored image, generating a restored difference image by inputting the generated difference image to a second learning model that is learned using a normal difference image that is a difference image of a normal image that captures a normal inspection object and a restored image that is generated using the normal image, the restored difference image corresponding to an image in which the capturing error included in the difference image of the captured image and the restored image is restored and not including a difference abnormality image corresponding to the abnormality, detecting the abnormality of the inspection object using the difference image and the restored difference image. 2.The abnormality detection method according to claim 1, wherein, in the detection of the abnormality of the inspection object, an abnormality judgment image is generated using a difference between the difference image and the restored difference image, and the abnormality of the inspection object is judged using the abnormality judgment image. 3.The abnormality detection method according to claim 1 or 2, wherein, the first learning model generates the restored image by removing a position of the inspection object in the captured image. 4.The abnormality detection method according to claim 1 or 2, wherein, the first learning model is a learning model using a generative adversarial network. 5.The abnormality detection method according to claim 1 or 2, wherein, the second learning model is a learning model using a generative adversarial network. 6.The abnormality detection method according to claim 1 or 2, wherein, the first learning model and the second learning model use the same kind of learning model. 7.An abnormality detection apparatus having: an acquisition unit that acquires a captured image that captures an inspection object; a restored image generation unit that generates a restored image by inputting the captured image to a first learning model that is learned using a normal image that captures a normal inspection object, the restored image being an image that is restored on the basis of removal of a capturing error and that does not have an abnormality of the inspection object; a difference image generation unit that generates a difference image of the captured image and the restored image; a restored difference image generation unit that generates a restored difference image by inputting the generated difference image to a second learning model that is learned using a normal difference image that is a difference image of a normal image that captures a normal inspection object and a restored image that is generated using the normal image, the restored difference image corresponding to an image in which the capturing error included in the difference image of the captured image and the restored image is restored and not including a difference abnormality image corresponding to the abnormality. A recognition section detects an abnormality of the inspection object using the difference image and the restored difference image.

Images