Defect Diagnosis Model Creation Device, Defect Diagnosis Model Creation Method, and Defect Diagnosis Model Creation Program

Abstract

Provided is a fault diagnosis model creation device for creating a fault diagnosis model that determines the presence / absence of a fault in a target product from an image of the target product, said fault diagnosis model creation device comprising: an image acquisition unit that acquires a base image which captures a product identical or similar to the target product, and a plurality of fault images which capture a fault occurring in the product and which include a rare fault image; a pseudo fault image generation unit that generates a pseudo fault image by inputting, into a restorer which has learned the features of the base image in advance, a synthesized image in which the fault images are placed on part of the base image; a training data acquisition unit that acquires training data which includes a pseudo fault image group constituted by a plurality of pseudo fault images; and a fault diagnosis model creation unit that creates a fault diagnosis model by training with the training data.

Description

Technical Field

[0001] The present disclosure relates to a defect diagnosis model creation device, a defect diagnosis model creation method, and a defect diagnosis model creation program for creating a defect diagnosis model.

Background Art

[0002] There is known a technique for determining the presence or absence of a defect in a target product by using a learned defect diagnosis model that outputs a determination result on the presence or absence of a defect in the target product by inputting an image of the target product. Some of these defect diagnosis models are configured by a neural network generated by learning teacher data such as images of actual target products.

[0003] As teacher data for training a defect diagnosis model (neural network), it is necessary to prepare a large amount of various images in which various defects appear. However, there is a problem that it is difficult to prepare a large amount of images corresponding to defects with a low occurrence frequency (rare defects).

[0004] Patent Document 1 discloses a restorator (learned model) that generates a defect in a specified defect area (mask) by performing machine learning using a defective product image and a mask image of a defect area. By inputting an image (non-defective product mask image) with a predetermined defect candidate area corresponding to the part where the defect occurs as a predetermined area into this restorator, a pseudo-defect image (restored image) with a defect generated in the defect candidate area is generated. The pseudo-defect image generated by the restorator is used as teacher data for deep learning of a neural network.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] Since the restorer described in Patent Document 1 uses a large number of defective images for its learning, there is a risk that the characteristics of the defects are averaged in the restorer, and only a pseudo-defective image that generates a plausible defect in the defect candidate region can be output. That is, there is a risk that the above restorer cannot generate an image of a rare defect with a low occurrence frequency. There is a risk that the defect diagnosis model (neural network) learned with the pseudo-defective image generated by the above restorer has low determination accuracy for the presence or absence of defects for rare defects. Further, even when an image obtained by photographing an actual defect is learned in the defect diagnosis model (neural network), since the proportion of rare defect images in the teacher data is small, there is a risk that the determination accuracy for the presence or absence of defects for rare defects is low. As a result, the defect diagnosis model learned with the pseudo-defective image generated by the restorer described in Patent Document 1 may have low determination accuracy for the presence or absence of defects.

[0007] In view of the above circumstances, at least one embodiment of the present invention aims to provide a defect diagnosis model creation device, a defect diagnosis model creation method, and a defect diagnosis model creation program that can create a defect diagnosis model capable of improving the determination accuracy of the presence or absence of defects.

Means for Solving the Problems

[0008] The defect diagnosis model creation device according to at least one embodiment of the present invention is a defect diagnosis model creation device for creating a defect diagnosis model for determining the presence or absence of defects in a target product from an image of the target product, an image acquisition unit that acquires a plurality of defect images including at least one base image of a product identical or similar to the target product and a plurality of defect images of defects that have occurred in the product, and including at least one rare defect image of a defect with a generation probability equal to or less than a predetermined value; a pseudo-defect image generation unit that generates a pseudo-defect image by inputting a composite image in which the defect image is pasted on a part of the base image into at least one restorer that has previously learned the characteristics of the base image; A teacher data acquisition unit that acquires teacher data including a group of pseudo-defect images composed of a plurality of the pseudo-defect images; A defect diagnosis model creation unit that creates a defect diagnosis model by learning the teacher data.

[0009] A method for creating a defect diagnosis model according to at least one embodiment of the present invention is A method for creating a defect diagnosis model for determining the presence or absence of a defect in a target product from an image of the target product, An image acquisition step of acquiring at least one base image of a product identical or similar to the target product and a plurality of defect images in which defects occurring in the product are photographed, the plurality of defect images including at least one rare defect image in which a defect with a generation probability of a predetermined value or less is photographed; A pseudo-defect image generation step of generating a pseudo-defect image by inputting a composite image in which the defect image is pasted on a part of the base image into at least one restorer that has previously learned the features of the base image; A teacher data acquisition step of acquiring teacher data including a group of pseudo-defect images composed of a plurality of the pseudo-defect images; A defect diagnosis model creation step of creating a defect diagnosis model by learning the teacher data.

[0010] A defect diagnosis model creation program according to at least one embodiment of the present invention is A defect diagnosis model creation program for creating a defect diagnosis model for determining the presence or absence of a defect in a target product from an image of the target product, An image acquisition step of acquiring at least one base image of a product identical or similar to the target product and a plurality of defect images in which defects occurring in the product are photographed, the plurality of defect images including at least one rare defect image in which a defect with a generation probability of a predetermined value or less is photographed; A pseudo-defect image generation step of generating a pseudo-defect image by inputting a composite image in which the defect image is pasted on a part of the base image into at least one restorer that has previously learned the features of the base image; A teacher data acquisition step of acquiring teacher data including a pseudo-defect image group composed of a plurality of the pseudo-defect images; A defect diagnosis model creation step of creating a defect diagnosis model by training the teacher data, which is for causing a computer to execute.

Effect of the Invention

[0011] According to at least one embodiment of the present invention, there are provided a defect diagnosis model creation device, a defect diagnosis model creation method, and a defect diagnosis model creation program capable of creating a defect diagnosis model that can improve the determination accuracy of the presence or absence of defects.

Brief Description of the Drawings

[0012]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0013] Hereinafter, some embodiments of the present invention will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of the components described as embodiments or shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative examples.

[0014] (Defect Diagnosis System) FIG. 1 is a schematic configuration diagram schematically showing the configuration of a defect diagnosis system including a defect diagnosis model creation device according to an embodiment. The defect diagnosis model 11 according to some embodiments is mounted on the defect diagnosis system 10. The defect diagnosis model creation device 1 is a device for creating a defect diagnosis model 11 that determines the presence or absence of defects in the target product from an image of the target product. As shown in FIG. 1, the defect diagnosis system 10 includes a defect diagnosis model creation device 1 and a defect diagnosis device 2 that determines the presence or absence of defects in the target product using the learned defect diagnosis model 11 created by the defect diagnosis model creation device 1.

[0015] (Defect Diagnosis Device) As shown in FIG. 1, the defect diagnosis device 2 includes a defect diagnosis model storage unit 21 that stores the learned defect diagnosis model 11 created by the defect diagnosis model creation device 1, a target product image acquisition unit 22 that acquires a target product image that is an image of the target product to be inspected for defect diagnosis, a defect diagnosis unit 23 that determines whether there are defects in the target product image using the defect diagnosis model 11 and acquires a determination result, and a determination result output unit 24 that outputs the determination result acquired by the defect diagnosis unit 23.

[0016] FIG. 2 is an explanatory diagram for explaining a convolutional neural network used in one embodiment. The defect diagnosis model 11 stored in the defect diagnosis model storage unit 21 is a learned model that has previously learned the characteristics of defects by machine learning (including deep learning), and is configured to output a determination result indicating whether or not there is a defect in the target product image (target product) by inputting the target product image. The defect diagnosis model 11 may be generated by a learning process for a learning model including a convolutional neural network (neural network) including an input layer, one or more intermediate layers, and an output layer as shown in FIG. 2. In the embodiment shown in FIG. 2, the convolutional neural network includes a convolutional layer 101 that applies a filter for extracting image features to the input image input to the input layer to obtain a feature map (two-dimensional data), and a pooling layer 102 that compresses the size of the feature map obtained by the convolutional layer 101. It includes a plurality of sets of intermediate layers composed of a combination of, a fully connected layer 103 that combines the image data from which the feature portions have been extracted through the plurality of sets of intermediate layers to one node, the input layer, and the output layer. The output layer may be configured to output a determination probability as the determination result.

[0017] The target product image acquisition unit 22 is configured to acquire a target product image from outside the defect diagnosis apparatus 2. The target product image acquisition unit 22 may be, for example, an information communication device such as an input interface or a communication interface, or a photographing device or the like. Note that the corresponding product image may be an image obtained by photographing a part of the target product.

[0018] The defect diagnosis unit 23 is configured to input the target product image (inspection image) acquired by the target product image acquisition unit 22 into the defect diagnosis model 11 stored in the defect diagnosis model storage unit 21 and acquire the determination result output by the defect diagnosis model 11. The determination result output unit 24 is configured to output the determination result acquired by the defect diagnosis unit 23. The determination result output unit 24 may be, for example, an output interface that outputs the determination result to a display device such as a display or a printing device such as a printer, or a communication interface that outputs the determination result to an information terminal external to the defect diagnosis device 2. Various products are applicable to the target product. The target product may be, for example, a supercharger such as a turbocharger, or an impeller or a housing that is a component of the supercharger.

[0019] (Defect Diagnosis Model Creation Device) As shown in FIG. 1, the defect diagnosis model creation device 1 includes an image acquisition unit 3, a pseudo-defect image generation unit 4, a teacher data acquisition unit 5, and a defect diagnosis model creation unit 6.

[0020] (Image Acquisition Unit) The image acquisition unit 3 is configured to acquire at least one base image BI of a product that is the same as or similar to the target product, and a plurality of defect images DI of defects that have occurred in the product. A product similar to the target product may be a product whose feature amount extracted from the product or a captured image of the product has a correlation coefficient of a predetermined value or more with respect to the feature amount extracted from the target product or a captured image of the target product, or may be a product designated by the user of the defect diagnosis model creation device 1. The above feature amount is a number that quantitatively represents features of the product such as color tone, texture, and structure. The image acquisition unit 3 may be an information communication device such as an input interface or a communication interface that acquires at least one base image BI and a plurality of defect images DI from outside the defect diagnosis model creation device 1. Further, when the defect diagnosis model creation device 1 includes an image database 31 that stores at least one base image BI and a plurality of defect images DI as shown in FIG. 1, the image acquisition unit 3 may acquire at least one base image BI and a plurality of defect images DI from the image database 31. The images stored in the image database 31 are associated with labels indicating the presence or absence of defects in the images.

[0021] (Pseudo-defect image generation unit) The pseudo-defect image generation unit 4 is configured to generate a pseudo-defect image PDI by inputting a composite image CI in which a defect image DI is pasted on a part of the base image BI to at least one restorer 7 that has previously learned the features of the base image BI. In the embodiment shown in FIG. 1, the pseudo-defect image generation unit 4 includes a composite image creation unit 41 configured to create a composite image CI from the base image BI and the defect image DI acquired by the image acquisition unit 3, a restorer storage unit 42 that stores at least one restorer 7, an image generation unit 43 configured to generate a pseudo-defect image PDI from the composite image CI using at least one restorer 7, and a pseudo-defect image storage unit 44 that stores the pseudo-defect image PDI generated by the image generation unit 43.

[0022] The base image BI is an image of a product that is the same as or similar to the target product and has no defects. The defect image DI is an image of the defects that have occurred in the product. Note that the defects included in the defect image DI may be not only defects visible from the outside of the product such as cracks, but also defects not visible from the outside of the product such as internal wall thinning. The composite image creation unit 41 creates a composite image CI including defects by pasting the defect image DI including defects onto a part of the image area of the base image BI that does not include defects.

[0023] At least one restorer 7 stored in the restorer storage unit 42 has already learned the features of the base image BI by machine learning (including deep learning) in advance, and is configured to output a pseudo-defect image PDI which is an image obtained by performing image processing to make the defect image DI in the composite image CI fit into the base image BI around the defect image DI when the composite image CI is input. The image processing by the restorer 7 is performed not only on the image area where the defect image DI is pasted in the image area of the composite image CI, but also on the image area of the base image BI around the defect image DI. The pseudo-defect image PDI includes the defect image DI on which the above image processing has been performed.

[0024] The image generation unit 43 is configured to input the composite image CI created by the composite image creation unit 41 into at least one restorer 7 stored in the restorer storage unit 42 and obtain the pseudo-defect image PDI output by the at least one restorer 7. The pseudo-defect image PDI generated in the image generation unit 43 is stored in the pseudo-defect image storage unit 44. Note that for each base image BI, a restorer 7 associated with the base image BI, that is, a restorer 7 that has already learned the features of the base image BI is generated. It is preferable that the image generation unit 43 inputs the composite image CI into the restorer 7 associated with the base image BI included in the composite image CI.

[0025] FIG. 3 is an explanatory diagram for explaining a restorer in one embodiment. FIG. 4 is an explanatory diagram for explaining an adversarial generation network used in one embodiment. The at least one restorer 7 may be generated by learning processing for an adversarial generation network including a generator G that extracts features in an actual image and generates a fake image from the extracted features, and a discriminator D that discriminates whether the fake image generated by the generator G is genuine or not. The at least one restorer 7 includes the generator G in the adversarial generation network and does not include the discriminator D in the adversarial generation network.

[0026] In the embodiment shown in FIG. 4, the adversarial generation network includes a plurality of layers of generators G (G0, G1, G2, G3, ···, Gn) and a plurality of layers (the same number as the generators G) of discriminators D. Each of the plurality of layers of generators G (Gn) is configured to simultaneously learn the generators G of the previous layer (Gn-1) and the layer before the previous layer (Gn-2). For example, the generator G3 of the third layer simultaneously learns the generators G of the second layer G2, which is the previous layer, and the first layer G1, which is the layer before the previous layer. At this time, the learning rate of the generator G of the previous layer G2 (Gn-1) is a fractional multiple (1 / 10 in the example shown in the figure) with respect to the learning rate of the generator G of that layer G3 (Gn), and the generator G of the layer before the previous layer G1 (Gn-2) is a fractional multiple (1 / 10 in the example shown in the figure) with respect to the learning rate of the generator G of the previous layer G2 (Gn-1). The sizes of the images to be discriminated (fake images generated by the generator G of the same layer as the discriminator D and actual images) by the plurality of layers of discriminators D increase as the layer goes deeper.

[0027] (Teacher data acquisition unit) The teacher data acquisition unit 5 is configured to acquire teacher data TD including a group of pseudo defect images PDIG composed of a plurality of pseudo defect images PDI. The teacher data acquisition unit 5 acquires a plurality of pseudo defect images PDI stored in the pseudo defect image storage unit 44. The group of pseudo defect images PDIG composed of the plurality of pseudo defect images PDI acquired by the teacher data acquisition unit 5 becomes the teacher data TD.

[0028] Note that the image database 31 may store actual captured images ASI that are images of products identical or similar to the target product. The actual captured images ASI stored in the image database 31 may include not only images of the products with defects but also images of the products without defects. The teacher data acquisition unit 5 may be configured to acquire at least one actual captured image ASI stored in the image database 31 as teacher data TD in addition to the pseudo defect image group PDIG.

[0029] (Defect diagnosis model creation unit) The defect diagnosis model creation unit 6 is configured to create a defect diagnosis model 11 by learning the teacher data TD. The defect diagnosis model creation unit 6 may generate the defect diagnosis model 11 by performing learning processing on a learning model including the above-described convolutional neural network (neural network). The defect diagnosis model 11 created by the defect diagnosis model creation unit 6 is stored in the above-described defect diagnosis model storage unit 21.

[0030] As shown in FIG. 1, the defect diagnosis model creation apparatus 1 according to some embodiments includes the above-described image acquisition unit 3, the above-described pseudo defect image generation unit 4, the above-described teacher data acquisition unit 5, and the above-described defect diagnosis model creation unit 6. The above-described plurality of defect images DI include at least one rare defect image RDI in which a defect with a generation probability equal to or less than a predetermined value is captured. Among the ranges set based on the normal distribution obtained by approximating the distribution of the generation probabilities of a plurality of pre-sampled defects, defects belonging to a range of not less than the standard deviation σ calculated from the generation probability of the sampled defect (+σ or more or -σ or less) may be regarded as rare defects. It is preferable that defects belonging to a range of 2σ or more (+2σ or more or -2σ or less) are regarded as rare defects.

[0031] According to the above configuration, since the at least one restorer 7 has learned in advance the features (color tone, texture, structure, etc.) of the base image BI, it is possible to generate a pseudo-defect image PDI in which the defect image DI intentionally synthesized into the base image BI is harmonized with the base image BI without a sense of incongruity. Since the composite image CI can be generated by pasting the defect image DI on a part of the base image BI, when generating the composite image CI, it is easy to process the defects (enlargement / reduction, rotation, aspect ratio change, flip, skew, etc.) of the defect image DI, change the position of the defects in the composite image CI, and change the types of defects included in the composite image CI. The defect diagnosis model creation device 1 can include a rare defect image RDI with a rare defect shape in the defect image DI that is the basis of the composite image CI, thereby generating a composite image CI and a pseudo-defect image PDI having a rare defect, and the generated pseudo-defect image PDI having a rare defect can be used as teacher data TD. As a result, the defect diagnosis model creation device 1 can cause the defect diagnosis model 11 to learn teacher data TD that takes into account the range (variation) of the possibility of actual occurrence of defects, so that it is possible to create a defect diagnosis model 11 that can accurately determine not only the presence or absence of frequently occurring defects but also the presence or absence of rare defects.

[0032] FIG. 5 is a schematic configuration diagram schematically showing the configuration of a defect diagnosis model creation device according to an embodiment. In some embodiments, as shown in FIG. 5, the defect diagnosis model creation device 1 described above further includes an image combination adjustment unit 8 configured to adjust the ratio of rare defect images RDI included in a plurality of defect images DI in the pseudo-defect image group PDIG.

[0033] Each of the plurality of pseudo-defect images PDI that make up the pseudo-defect image group PDIG includes a defect image DI. In certain embodiments, the image combination adjustment unit 8 adjusts the proportion of the pseudo-defect images PDI that include rare defect images RDI among the pseudo-defect image group PDIG (a plurality of pseudo-defect images PDI) acquired by the teacher data acquisition unit 5 so that it falls within a predetermined upper and lower limit range set in advance. Specifically, when the proportion of the pseudo-defect images PDI that include rare defect images RDI among the pseudo-defect image group PDIG acquired by the teacher data acquisition unit 5 is less than the lower threshold value, the image combination adjustment unit 8 increases the proportion of the pseudo-defect images PDI that include rare defect images RDI, and when the proportion of the pseudo-defect images PDI that include rare defect images RDI exceeds the upper threshold value, the image combination adjustment unit 8 decreases the proportion of the pseudo-defect images PDI that include rare defect images RDI. Note that the above upper and lower limit ranges (upper threshold value and lower threshold value) in the image combination adjustment unit 8 may be set manually, or may be set automatically by a ratio determination unit 82 or the like described later.

[0034] According to the above configuration, by increasing the proportion of the rare defect images RDI (pseudo-defect images PDI including the same) included in the pseudo-defect image group PDIG by the image combination adjustment unit 8, the proportion of the rare defect images RDI in the teacher data TD can be increased, so that the discrimination accuracy of the defect diagnosis model 11 for the rare defect images RDI can be improved.

[0035] In some embodiments, the above-described image combination adjustment unit 8 includes a detection accuracy evaluation unit 81 that evaluates the detection accuracy of the defect diagnosis model 11, and a ratio determination unit 82 that determines the ratio of the rare defect images RDI so that the detection accuracy of the defect diagnosis model 11 becomes a detection accuracy of a predetermined level or higher based on the evaluation result of the detection accuracy evaluation unit 81.

[0036] When an image whose defect presence or absence is known in advance (an image associated with the above label), such as an actual captured image ASI stored in the image database 31 or a pseudo-defect image PDI stored in the pseudo-defect image storage unit 44, is input to the defect diagnosis model 11, the detection accuracy evaluation unit 81 quantitatively evaluates whether the defect diagnosis model 11 outputs an appropriate determination result, and outputs the evaluation result as the detection accuracy. For example, the detection accuracy evaluation unit 81 may output the correct answer rate of the determination result of the defect diagnosis model 11 as the detection accuracy. The detection accuracy evaluation unit 81 may output the detection accuracy for each type of defect included in the defect image DI (rare defect image RDI). The ratio determination unit 82 determines the ratio of the rare defect image RDI in the training data TD so that the detection accuracy of the defect diagnosis model 11 output as the evaluation result of the detection accuracy evaluation unit 81 becomes a predetermined or higher detection accuracy.

[0037] The detection accuracy evaluation unit 81 may be configured to separately output the evaluation results of the rare defect image RDI and the defect image DI that is not the rare defect image RDI. The ratio determination unit 82 may determine the above ratio of the rare defect image RDI so that the detection accuracy of the defect diagnosis model 11 for the rare defect image RDI output by the detection accuracy evaluation unit 81 becomes a predetermined or higher detection accuracy. Further, the ratio determination unit 82 may determine the above ratio of the rare defect image RDI so that both the detection accuracy of the defect diagnosis model 11 for the rare defect image RDI output by the detection accuracy evaluation unit 81 and the detection accuracy of the defect diagnosis model 11 for the defect image DI that is not the rare defect image RDI become a predetermined or higher detection accuracy. When the detection accuracy evaluation unit 81 outputs the detection accuracy for each type of defect included in the defect image DI (rare defect image RDI), the above ratio of the rare defect image RDI may be determined so that the detection accuracy of the defect diagnosis model 11 for all types of defects becomes a predetermined or higher detection accuracy. Specifically, when the detection accuracy of the rare defect is poor, the detection accuracy evaluation unit 81 increases the ratio of the rare defect image RDI. Further, when the detection accuracy of the non-rare defect deteriorates due to an increase in the ratio of the rare defect image RDI, the detection accuracy evaluation unit 81 decreases the ratio of the rare defect image RDI.

[0038] According to the above configuration, the ratio determination unit 82 determines the ratio of the rare defect images RDI so that the detection accuracy of the defect diagnosis model 11 becomes a detection accuracy equal to or higher than a predetermined value based on the evaluation result of the detection accuracy evaluation unit 81. Thus, the ratio of the rare defect images RDI in the teacher data TD can be adjusted so that the detection accuracy of the defect diagnosis model 11 can be improved, and the discrimination accuracy of the defect diagnosis model 11 for rare defects can be improved.

[0039] (Multiple restorers) In some embodiments, as shown in FIG. 1, at least one of the above-described restorers 7 includes a first restorer 7A that has pre-learned the features of the first base image BI1, and a second restorer 7B that has pre-learned the features of a second base image BI2 whose features are different from those of the first base image BI1. Here, the fact that the features of the base images BI are different means that the user of the defect diagnosis model creation device 1 can determine that they are different base images BI. When the correlation coefficient indicating the strength of the relationship between the feature amount extracted from the second base image BI2 and the feature amount extracted from the first base image BI1 to be compared is less than a predetermined value, it may be determined that the second base image BI2 has different features from the first base image BI1.

[0040] The second base image BI2 may be a captured image of the same product as the first base image BI1. In the visual inspection of the product after use, due to the use conditions of the product (the environment in which it is used), there are differences in the surface color and characteristic shape changes compared to the product before use. Therefore, it is preferable to prepare a restorer 7 that has learned the features of the base image BI corresponding to the use conditions of the product. Also, when there are quality variations in the target product, it is preferable to prepare a plurality of restorers 7 so as to be able to handle each variation.

[0041] The image database 31 stores a plurality of base images BI (BI1, BI2, BI3), and the image acquisition unit 3 may acquire the plurality of base images BI (BI1, BI2, BI3) from the image database 31. The image generation unit 43 generates a pseudo defect image PDI corresponding to the input composite image CI for each number of the composite images CI input to each restorator 7 (7A, 7B). The image generation unit 43 inputs the composite image CI created based on the first base image BI1 to the first restorator 7A that has learned the features of the first base image BI1 in advance, and inputs the composite image CI created based on the second base image BI2 to the second restorator 7B that has learned the features of the second base image BI2 in advance.

[0042] According to the above configuration, the defect diagnosis model creation device 1 includes a plurality of restorators 7 (the first restorator 7A, the second restorator 7B) that have learned in advance the features of the base images BI1 and BI2 with different features individually. Therefore, it is possible to generate pseudo defect images PDI corresponding to the base images BI1 and BI2 with different features individually. As a result, the types of pseudo defect images PDI that the defect diagnosis model creation device 1 can generate can be increased, and the teacher data TD can be given a width. Therefore, the defect identification accuracy of the defect diagnosis model 11 can be improved.

[0043] (Low pixel processing unit, high pixel processing unit) In some embodiments, as shown in FIG. 1, at least one of the above-described restorators 7 (a plurality in the illustrated example) is configured to learn the features of the reduced base image RBI reduced by reducing the base image BI. The reduced base image RBI is used as the teacher data of the restorator 7. The above-described defect diagnosis model creation device 1 further includes a low pixel processing unit 12 configured to perform low pixel processing on at least one of the combination of the base image BI and the defect image DI or the composite image CI, and a high pixel processing unit 13 configured to perform high pixel processing on the pseudo defect image PDI.

[0044] The low-pixelization processing unit 12 performs low-pixelization processing on at least one of the combination of the base image BI and the defect image DI, or the composite image CI so that the image size (number of pixels) becomes the same or similar to the image size (number of pixels) of the reduced base image RBI. In a certain embodiment, when the image size (number of pixels) of the reduced base image RBI is defined as IS1, and the image size (number of pixels) of the base image BI or the composite image CI on which the low-pixelization processing has been performed is defined as IS2, the condition of IS1×0.9≦IS2≦IS1×1.1 is satisfied. The low-pixelization processing unit 12 preferably provides a lower limit for low-pixelization in order to avoid disappearance or deterioration of defects. The reduction rate (the above lower limit) is determined by the image size (the memory capacity of the GPU (graphics card)) that can be processed by the restorer 7 and the base image BI. Examples of the low-pixelization processing method in the low-pixelization processing unit 12 include the area averaging method (average pixel method), the nearest neighbor method, the bilinear method (linear interpolation), the bicubic method, and Lanczos3.

[0045] The high-pixelization processing unit 13 performs high-pixelization (super-resolution) processing on the combination of the base image BI and the defect image DI on which the low-pixelization processing has been performed, or the pseudo-defect image PDI generated from the composite image CI on which the low-pixelization processing has been performed. The high-pixelization processing unit 13 performs high-pixelization processing so that the image size (number of pixels) of the pseudo-defect image PDI becomes the same or similar to the image size (number of pixels) of the inspection image. By creating the defect diagnosis model 11 with the teacher data TD including the pseudo-defect image PDI high-pixelized by the high-pixelization processing unit 13, the defect diagnosis model 11 can be made to support high-pixel inspection. Examples of the high-pixelization processing method in the high-pixelization processing unit 13 include SRCNN (Super-Resolution Convolutional Neural Network), FSRCNN (Fast Super-Resolution Convolutional Neural Network), and the like.

[0046] According to the above configuration, since at least one restorator 7 learns the features of the reduced-base image RBI, the performance required of the hardware used for learning can be suppressed compared to the case of learning the features of the base image BI before the reduction process. Since the image size that can be generated as the pseudo defect image PDI depends on the image size learned by the restorator 7, in the low-pixelization processing unit 12, low-pixelization processing is performed on the combined image CI input to the restorator 7 or the combination of the base image BI and the defect image DI that are the elements of the combined image CI. For products with a large size of the target product and small defects, inspection at a high pixel count is desirable. Therefore, in the high-pixelization processing unit 13, high-pixelization processing is performed on the pseudo defect image PDI that becomes the teacher data TD of the defect diagnosis model 11 so as to support inspection at a high pixel count. According to the above configuration, by providing the low-pixelization processing unit 12 and the high-pixelization processing unit 13, it becomes possible to generate a pseudo defect image PDI for products that cannot be applied due to the performance of the hardware (specifically, products with a large image size and small defects. That is, products that cannot be photographed unless the image size is increased), and the number of applicable products of the defect diagnosis model creation device 1 can be increased.

[0047] In addition, by setting the lower limit of the low-pixelization in the low-pixelization processing unit 12, even when a small defect occurs in a target product with a large image size, it is possible to avoid the disappearance or deterioration of the defect during the generation of the pseudo defect image PDI and maintain the quality of the generated pseudo defect image PDI.

[0048] In some embodiments, at least one restorator 7 (a plurality in the illustrated example) described above may be configured to directly learn the features of the base image BI from the base image BI. The base image BI is used as the teacher data of the restorator 7. In this case, the defect diagnosis model creation device 1 described above may not include the low-pixelization processing unit 12 and the high-pixelization processing unit 13.

[0049] (First fitness evaluation unit) FIG. 6 is a schematic configuration diagram schematically showing the configuration of a defect diagnosis model creation apparatus according to an embodiment. In some embodiments, as shown in FIG. 6, the above-described defect diagnosis model creation apparatus 1 further includes a first fitness evaluation unit 14 including a first fitness evaluation model 15 and a first non-conforming image removal unit 16.

[0050] The first fitness evaluation model 15 is a learned model obtained by pre-training a fitness evaluation teacher data GFE including a defective image of the above product (a product identical or similar to the target product) in which a defect has occurred and a non-defective image of the above product in which no defect has occurred. The first fitness evaluation model 15 has learned the fitness evaluation teacher data GFE including the actually captured images (defective images and non-defective images). By inputting the pseudo-defect image PDI, the similarity between the input pseudo-defect image PDI and the actually captured image is quantitatively evaluated, and the evaluation result is output as the fitness. The higher the similarity of the pseudo-defect image PDI input to the first fitness evaluation model 15 to the actually captured image, the higher the fitness output in the first fitness evaluation model 15.

[0051] The first fitness evaluation model 15 may be generated by a learning process for a learning model including a convolutional neural network (neural network) including an input layer, one or more intermediate layers, and an output layer as shown in FIG. 2.

[0052] FIG. 7 is an explanatory diagram for explaining the evaluation result output by the first fitness evaluation model in an embodiment. In the illustrated embodiment, the evaluation result (fitness) output by the first fitness evaluation model 15 is a numerical value included in the range of 0 (lower limit value) or more and 1 (upper limit value) or less. The higher the similarity of the pseudo-defect image PDI input to the first fitness evaluation model 15 to the actually captured image, the closer the fitness (first fitness DC1) is to 1, which is the upper limit value.

[0053] The first non-conforming image removal unit 16 is configured to exclude, from candidates for the teacher data TD, a pseudo defect image PDI in which the degree of conformity (first degree of conformity DC1) in the first degree of conformity evaluation model 15 is less than the first threshold value FT among the pseudo defect images PDI subjected to the high pixel processing by the high pixel processing unit 13 described above. A pseudo defect image PDI in which the first degree of conformity DC1 is less than the first threshold value FT results in abnormal output in the restorator 7, the high pixel processing unit 13, etc., and is not suitable as the teacher data TD, and thus is removed from the candidates for the teacher data TD by the first non-conforming image removal unit 16. As a result, each of the plurality of pseudo defect images PDI included in the teacher data TD satisfies the condition that the first degree of conformity DC1 is greater than or equal to the first threshold value FT.

[0054] According to the above configuration, the quality of the pseudo defect image PDI output from the restorator 7 and subjected to the high pixel processing (whether the pseudo defect image PDI is an image that conforms to reality in which a physical pseudo defect can occur) can be evaluated by the first degree of conformity evaluation model 15 that has been preliminarily trained with the teacher data GFE for degree of conformity evaluation including the defective image and the non-defective image. The first non-conforming image removal unit 16 can exclude, from the candidates for the teacher data TD, a pseudo defect image PDI in which the degree of conformity in the first degree of conformity evaluation model 15 is insufficient (less than the first threshold value FT). Thereby, it is possible to suppress the defect diagnosis model 11 from learning, as the teacher data TD, a pseudo defect image PDI including an unrealistic defect image DI that does not physically occur and degrading the defect diagnosis accuracy.

[0055] If the first threshold FT is too high, there is a risk that rare defective images RDI with a low occurrence frequency in the first non-conforming image removal unit 16 will be excluded from the candidates for the teacher data TD. In this case, since the defect diagnosis model 11 cannot learn the rare defective images RDI, the detection accuracy of the defect diagnosis model 11 for the rare defective images RDI may decrease. Also, if the first threshold FT is too low, there is a risk that unrealistic defective images DI will not be excluded from the candidates for the teacher data TD in the first non-conforming image removal unit 16. In this case, there is a risk that the defect diagnosis model 11 will learn the unrealistic defective images DI and reduce the defect diagnosis accuracy. Therefore, optimization of the first threshold FT is required. Note that the first threshold FT may be set manually or may be set automatically by a first threshold adjustment unit 91 described later.

[0056] (First Threshold Adjustment Unit) In some embodiments, as shown in FIG. 6, the above-described defect diagnosis model creation device 1 further includes a first threshold adjustment unit 91 configured to adjust the above-described first threshold FT in consideration of the detection accuracy of the above-described defect diagnosis model 11.

[0057] Based on the above-described defect diagnosis unit 23 and determination result output unit 24, a determination result of the presence or absence of a defect by the defect diagnosis model 11 is output. The defect diagnosis model creation device 1 investigates pseudo-defective images PDI and defective images (actual photographed images ASI) that are close to the images with insufficient accuracy in the detection accuracy evaluation unit 81 existing in the defect diagnosis model creation device 1 or the defect diagnosis device 2 (such as the image database 31), and determines an image to be added to the teacher data TD. After that, after the first threshold FT is changed by the first threshold adjustment unit 91, generation of the pseudo-defective images PDI and addition to the teacher data TD are performed, and the defect diagnosis model 11 is reconfigured. Using the reconfigured defect diagnosis model 11, determination of the presence or absence of a defect is performed.

[0058] In one embodiment, the first threshold adjustment unit 91 prepares a plurality of first thresholds FT with different numerical values, compares the detection accuracy of the defect diagnosis model 11 obtained by setting the plurality of first thresholds FT (for example, the evaluation result output by the detection accuracy evaluation unit 81), and determines the optimal value of the first threshold FT that maximizes the detection accuracy of the defect diagnosis model 11. The first threshold adjustment unit 91 may be configured to repeatedly determine the optimal value of the first threshold FT multiple times.

[0059] According to the above configuration, in the first threshold adjustment unit 91, by adjusting the first threshold FT in consideration of the detection accuracy of the defect diagnosis model 11, in the first non-conforming image removal unit 16, while leaving the pseudo-defect image PDI including the rare defect image RDI as a candidate for the teacher data TD, the pseudo-defect image PDI including the unrealistic defect image DI can be accurately removed from the candidates for the teacher data TD. As a result, the detection accuracy of the defect diagnosis model 11 can be improved.

[0060] (Second conformity evaluation unit) In some embodiments, as shown in FIG. 6, the above-described defect diagnosis model creation apparatus 1 further includes a second conformity evaluation unit 17 including a second conformity evaluation model 18 and a second non-conforming image removal unit 19.

[0061] The second conformity evaluation model 18 is a learned model that has been pre-trained with conformity evaluation teacher data GFE including a defective image of the above product (the same or similar product as the target product) in which a defect has occurred and a non-defective image of the above product in which no defect has occurred. The conformity evaluation teacher data GFE is stored in the image database 31. The second conformity evaluation model 18 has been trained with conformity evaluation teacher data GFE including actually captured images (defective images and non-defective images). By inputting the synthetic image CI, the similarity between the input synthetic image CI and the actually captured image is quantitatively evaluated, and the evaluation result is output as the conformity. The higher the similarity of the synthetic image CI input to the second conformity evaluation model 18 to the actually captured image, the higher the conformity output by the second conformity evaluation model 18.

[0062] FIG. 8 is an explanatory diagram for explaining the evaluation result output by the second fitness evaluation model in one embodiment. In the illustrated embodiment, the evaluation result (fitness) output by the second fitness evaluation model 18 is a numerical value included in the range of 0 (lower limit value) or more and 1 (upper limit value) or less. The higher the similarity of the synthetic image CI input to the second fitness evaluation model 18 to the actually captured image, the closer the fitness (second fitness DC2) is to 1, which is the upper limit value.

[0063] The second non-conforming image removal unit 19 is configured to exclude a synthetic image CI in which the fitness (second fitness DC2) in the second fitness evaluation model 18 is less than the second threshold ST from the candidates for the teacher data TD. The synthetic image CI input to the second fitness evaluation model 18 may be the synthetic image CI before the low-pixelation process in the low-pixelation processing unit 12 as shown by the solid line in FIG. 6, or may be the synthetic image CI after the low-pixelation process in the low-pixelation processing unit 12 as shown by the dotted line in FIG. 6. A synthetic image CI in which the second fitness DC2 is less than the second threshold ST is an abnormal output in the synthetic image creation unit 41, the low-pixelation processing unit 12, etc., and is not suitable as the teacher data TD, so it is removed from the candidates for the teacher data TD in the second non-conforming image removal unit 19. As a result, each of the plurality of pseudo-defect images PDI included in the teacher data TD satisfies the condition that the second fitness DC2 is equal to or greater than the second threshold ST.

[0064] According to the above configuration, the second fitness evaluation model 18 that has been pre-trained with the teacher data GFE for fitness evaluation including defective images and non-defective images can evaluate the quality of the composite image CI (whether the composite image CI or the pseudo-defective image PDI is an image that conforms to reality where it can physically occur). The second non-conforming image removal unit 19 can exclude a composite image CI with insufficient fitness (less than the second threshold value) in the second fitness evaluation model 18 (for example, a composite image with a defect in a part where it does not actually occur) from the candidates of the teacher data TD. Thereby, it is possible to suppress the defect diagnosis model 11 from learning a pseudo-defective image PDI including an unrealistic defective image DI that does not physically occur and reducing the defect diagnosis accuracy.

[0065] If the second threshold value ST is too high, there is a risk that a rare defective image RDI with a low occurrence frequency will be excluded from the candidates of the teacher data TD in the second non-conforming image removal unit 19. In this case, since the defect diagnosis model 11 cannot learn the rare defective image RDI, there is a risk that the detection accuracy of the defect diagnosis model 11 for the rare defective image RDI will decrease. Also, if the second threshold value ST is too low, there is a risk that an unrealistic defective image DI will not be excluded from the candidates of the teacher data TD in the second non-conforming image removal unit 19. In this case, there is a risk that the defect diagnosis model 11 will learn the unrealistic defective image DI and reduce the defect diagnosis accuracy. Therefore, optimization of the second threshold value ST is required. Note that the second threshold value ST may be set manually or may be set automatically by a second threshold value adjustment unit 92 described later.

[0066] (Second Threshold Value Adjustment Unit) In some embodiments, as shown in FIG. 6, the above-described defect diagnosis model creation device 1 further includes a second threshold value adjustment unit 92 configured to adjust the second threshold value ST in consideration of the detection accuracy of the defect diagnosis model 11.

[0067] As described above, the defect diagnosis model creation device 1 investigates a pseudo defect image PDI and a defective image (actual captured image ASI) that are similar to an image with insufficient accuracy in the detection accuracy evaluation unit 81, which exist in the defect diagnosis model creation device 1 or the defect diagnosis device 2 (such as the image database 31), and determines an image to be added to the training data TD. After that, after the second threshold adjustment unit 92 changes the second threshold ST, a pseudo defect image PDI is generated and added to the training data TD, and the defect diagnosis model 11 is reconstructed. Using the reconstructed defect diagnosis model 11, a determination is made as to whether or not there is a defect.

[0068] In one embodiment, the second threshold adjustment unit 92 prepares a plurality of second thresholds ST with different numerical values, compares the detection accuracy (for example, the evaluation result output by the detection accuracy evaluation unit 81) of the defect diagnosis model 11 obtained by setting the plurality of second thresholds ST, and may be configured to obtain an optimal value of the second threshold ST at which the detection accuracy of the defect diagnosis model 11 is increased. The second threshold adjustment unit 92 may be configured to repeatedly obtain the optimal value of the second threshold ST a plurality of times.

[0069] According to the above configuration, in the second threshold adjustment unit 92, by adjusting the second threshold ST in consideration of the detection accuracy of the defect diagnosis model 11, in the second non-conforming image removal unit 19, while leaving a composite image CI including a rare defect image RDI as a candidate for the training data TD, a composite image CI including an unrealistic defect image DI can be accurately removed from the candidates for the training data TD. As a result, the detection accuracy of the defect diagnosis model 11 can be improved.

[0070] (Relearning unit) In some embodiments, as shown in FIG. 6, the above-described defect diagnosis model creation device 1 further includes a relearning unit 93 that adds an image similar to a defective image in which the detection accuracy of the defect diagnosis model 11 is equal to or lower than a predetermined value to the fitness evaluation training data GFE.

[0071] The relearning unit 93 investigates pseudo defect images PDI and defect-containing images (actual captured images ASI) that are similar to images with insufficient accuracy in the detection accuracy evaluation unit 81 (such as images stored in the image database 31, the first fitness evaluation model 15 and the second fitness evaluation model 18 whose fitness is satisfied by the teacher data acquisition unit). Based on the investigation results, the relearning unit 93 adds pseudo defect images PDI and defect-containing images (actual captured images ASI) that are similar to images with insufficient accuracy in the detection accuracy evaluation unit 81 to the teacher data GFE for fitness evaluation of the first fitness evaluation model 15 and the second fitness evaluation model 18, and causes the first fitness evaluation model 15 and the second fitness evaluation model 18 to be relearned. The first non-conforming image removal unit 16 and the second non-conforming image removal unit 19 use the relearned first fitness evaluation model 15 and second fitness evaluation model 18.

[0072] The first non-conforming image removal unit 16 excludes pseudo defect images PDI whose fitness in the first fitness evaluation model 15 is less than the first threshold value FT from the candidates for the teacher data TD. Pseudo defect images PDI similar to defect-containing images where the detection accuracy of the defect diagnosis model 11 is below a predetermined level are excluded from the candidates for the teacher data TD in the first non-conforming image removal unit 16, and the number included in the teacher data TD decreases, which may reduce the detection accuracy of the defect diagnosis model 11 for the above-mentioned defect-containing images. The relearning unit 93 adds an image similar to the above-mentioned defect-containing image to the teacher data GFE for fitness evaluation of the first fitness evaluation model 15 and causes the first fitness evaluation model 15 to be relearned, thereby suppressing the pseudo defect image PDI similar to the above-mentioned defect-containing image from being excluded from the candidates for the teacher data TD in the first non-conforming image removal unit 16, and increasing the number included in the teacher data TD, thereby improving the detection accuracy of the defect diagnosis model 11 for images identical or similar to the above-mentioned defect-containing images.

[0073] The second non-conforming image removal unit 19 excludes a composite image CI whose degree of conformity in the second degree-of-conformity evaluation model 18 is less than the second threshold ST from the candidates for the teacher data TD. A composite image CI similar to a defective image in which the detection accuracy of the defect diagnosis model 11 is below a predetermined level is excluded from the candidates for the teacher data TD in the second non-conforming image removal unit 19, and the number included in the teacher data TD decreases, so there is a possibility that the detection accuracy of the defect diagnosis model 11 for the defective image has decreased. The relearning unit 93 adds an image similar to the defective image to the teacher data GFE for degree-of-conformity evaluation of the second degree-of-conformity evaluation model 18 and causes the second degree-of-conformity evaluation model 18 to relearn, thereby suppressing the composite image CI similar to the defective image from being excluded from the candidates for the teacher data TD in the second non-conforming image removal unit 19 and increasing the number included in the teacher data TD, thereby improving the detection accuracy of the defect diagnosis model 11 for an image identical or similar to the defective image.

[0074] According to the above configuration, in the relearning unit 93, by adding an image similar to a defective image in which the detection accuracy of the defect diagnosis model 11 is below a predetermined level to the teacher data GFE for degree-of-conformity evaluation, an image that requires learning by the defect diagnosis model 11 (an image with low detection accuracy of the defect diagnosis model 11) can be learned by the first degree-of-conformity evaluation model 15 and the second degree-of-conformity evaluation model 18, and the evaluation accuracy of the degree of conformity in the first degree-of-conformity evaluation model 15 and the second degree-of-conformity evaluation model 18 for the similar image (image with low detection accuracy of the defect diagnosis model 11) can be improved. By improving the evaluation accuracy of the degree of conformity in the first degree-of-conformity evaluation model 15 and the second degree-of-conformity evaluation model 18, it is possible to suppress an image that requires learning by the defect diagnosis model 11 from being excluded from the candidates for the teacher data TD in the first non-conforming image removal unit 16 and the second non-conforming image removal unit 19. As a result, an image that requires learning by the defect diagnosis model 11 (teacher data TD) can be learned by the defect diagnosis model 11, so the detection accuracy of the defect diagnosis model 11 can be improved.

[0075] FIG. 9 is a schematic configuration diagram schematically showing the configuration of a defect diagnosis model creation device according to an embodiment. In some embodiments, as shown in FIG. 9, the defect diagnosis model creation device 1 described above further includes a defect image processing unit 94 configured to perform an image processing operation that changes the features of the defects included in the defect image DI to features similar to those of the defects.

[0076] By the image processing operation in the defect image processing unit 94, a plurality of defect images DI including defects with features similar to the features of the defects included in one defect image DI can be generated. In one embodiment, the image processing operation in the defect image processing unit 94 includes at least one of enlargement, reduction, rotation, skew (tilting the defect diagonally), inversion (upside-down inversion, left-right inversion), or aspect ratio change of the defects included in the defect image DI. In one embodiment, the above-described image acquisition unit 3 may acquire the defect image DI on which the image processing operation has been performed in the defect image processing unit 94. Further, in one embodiment, the image processing operation on the defect image DI in the defect image processing unit 94 may be executed when the defect image DI is pasted on a part of the base image BI, or may be executed after the defect image DI is pasted on a part of the base image BI. Note that the defect diagnosis model creation device 1 does not necessarily pass all the defect images DI through the defect image processing unit 94. As shown in FIG. 9, the defect diagnosis model creation device 1 may input the defect image DI on which the image processing operation has not been performed in the defect image processing unit 94 to the restorer 7 or the pasting position determination unit 95 described later. Further, the defect image DI that has passed through the defect image processing unit 94 may be input to the pasting position determination unit 95.

[0077] According to the above configuration, the defect image processing unit 94 can increase the number of synthesized images CI created by performing an image processing operation that changes the features of the defects included in the defect image DI to features similar to those of the defects, and can generate pseudo-defect images PDI for each of the created synthesized images CI. By including the defect image processing unit 94, the defect diagnosis model creation device 1 can increase the pseudo-defect images PDI and can cause the defect diagnosis model 11 to learn a large amount of teacher data TD, so that the determination accuracy of the presence or absence of defects in the defect diagnosis model 11 can be improved.

[0078] Note that the defective image DI generated by the defective image processing unit 94 may include an unrealistic defective image DI that does not physically occur. However, a composite image CI or a pseudo-defective image PDI including an unrealistic defective image DI that does not physically occur can be removed by the above-described first non-conforming image removal unit 16 and the above-described second non-conforming image removal unit 19.

[0079] In some embodiments, as shown in FIG. 9, the above-described defective diagnosis model creation apparatus 1 further includes an attachment position determination unit 95 that randomly determines an attachment position of the defective image DI to the base image BI. The above-described composite image creation unit 41 attaches the defective image DI to the attachment position of the base image BI determined by the attachment position determination unit 95. Note that the attachment position determination unit 95 may be configured to randomly determine the attachment position of the defective image DI to the base image BI within a preset range.

[0080] According to the above configuration, in the attachment position determination unit 95, by randomly determining the attachment position of the defective image DI to the base image BI, the number of composite images CI created can be increased, and the pseudo-defective image PDI can be generated for each created composite image CI. The defective diagnosis model creation apparatus 1 includes the attachment position determination unit 95, so that the pseudo-defective image PDI can be increased, and a large amount of teacher data TD can be learned by the defective diagnosis model 11. Therefore, the determination accuracy of the presence or absence of defects in the defective diagnosis model 11 can be improved.

[0081] Note that the composite image CI with the defective image DI attached to the attachment position determined by the attachment position determination unit 95 may include an unrealistic defective image DI that does not physically occur, or the attachment position of the realistic defective image DI may be unrealistic, that is, the composite image CI itself may be an unrealistic image that does not physically occur. However, the composite image CI including an unrealistic defective image DI that does not physically occur, the unrealistic composite image CI that does not physically occur, or the pseudo-defective image PDI generated from these composite images CI can be removed by the above-described first non-conforming image removal unit 16 and the above-described second non-conforming image removal unit 19.

[0082] Each of the above-described defect diagnosis model creation device 1 and defect diagnosis device 2 includes a computer configured to be able to realize a predetermined function according to the processing described in the program. However, the general configuration and control will be omitted as appropriate. Each of the defect diagnosis model creation device 1 and the defect diagnosis device 2 is configured as a microcomputer including a CPU (processor) not shown, a memory such as a ROM and a RAM, a storage device such as an external storage device, an I / O interface, a communication interface, and the like. The defect diagnosis model creation device 1 and the defect diagnosis device 2 may realize the control in each of the above-described units by the CPU operating (for example, performing data calculation) according to the instructions of the program loaded into the main storage device of the memory.

[0083] FIG. 10 is a flowchart of a defect diagnosis model creation method according to an embodiment. The defect diagnosis model creation method 100 according to some embodiments is a method for creating the above-described defect diagnosis model 11. As shown in FIG. 10, the defect diagnosis model creation method 100 includes an image acquisition step S1, a pseudo-defect image generation step S2, a teacher data acquisition step S3, and a defect diagnosis model creation step S4.

[0084] In the image acquisition step S1, at least one of the above-described base images BI and the above-described plurality of defect images DI are acquired. The above-described plurality of defect images DI include at least one rare defect image RDI in which a defect with a generation probability of a predetermined value or less is photographed. In the pseudo-defect image generation step S2, a pseudo-defect image PDI is generated by inputting a composite image CI in which a defect image DI is pasted on a part of the base image BI into at least one restorer 7 that has previously learned the characteristics of the base image BI. Note that a restorer 7 associated with the base image BI for each base image BI, that is, a restorer 7 that has learned the characteristics of the base image BI is generated. In the pseudo-defect image generation step S2, it is preferable to input the composite image CI into the restorer 7 associated with the base image BI included in the composite image CI.

[0085] In the teacher data acquisition step S3, teacher data TD including a pseudo defect image group PDIG composed of a plurality of pseudo defect images PDI is acquired. In the defect diagnosis model creation step S4, a defect diagnosis model 11 is created by training the teacher data TD.

[0086] According to the defect diagnosis model creation method 100, a synthetic image CI or a pseudo defect image PDI having a rare defect can be generated, and the generated pseudo defect image PDI having a rare defect can be used as the teacher data TD. According to the defect diagnosis model creation method 100, since the teacher data TD considering the range (variation) of the possibility of actual defect occurrence can be trained in the defect diagnosis model 11, not only the presence or absence of frequently occurring defects but also the presence or absence of rare defects can be accurately determined, and a defect diagnosis model 11 can be created.

[0087] The defect diagnosis model creation program 100A according to some embodiments is a program for creating the above-described defect diagnosis model 11. The defect diagnosis model creation program 100A is a program for causing a computer to execute the above-described image acquisition step S1, the above-described pseudo defect image generation step S2, the above-described teacher data acquisition step S3, and the above-described defect diagnosis model creation step S4 as shown in FIG. 10. The defect diagnosis model creation program 100A includes a program for realizing the functions of each part in the defect diagnosis model creation apparatus 1. The defect diagnosis model creation program 100A may be stored in a computer-readable recording medium (for example, a flash memory).

[0088] According to the defect diagnosis model creation program 100A, a synthetic image CI or a pseudo-defect image PDI having a rare defect can be generated, and the generated pseudo-defect image PDI having a rare defect can be used as teacher data TD. According to the defect diagnosis model creation program 100A, since the teacher data TD considering the range (variation) of the possibility of actual occurrence of a defect can be learned by the defect diagnosis model 11, not only the presence or absence of a frequently occurring defect but also the presence or absence of a rare defect can be accurately determined, and the defect diagnosis model 11 can be created.

[0089] In this specification, expressions representing relative or absolute arrangements such as "in a certain direction", "along a certain direction", "parallel", "orthogonal", "center", "concentric", or "coaxial" not only strictly represent such arrangements, but also represent a state in which there are tolerances or relative displacements with angles and distances that can obtain the same function. For example, expressions representing that things such as "identical", "equal", and "homogeneous" are in an equal state not only strictly represent an equal state, but also represent a state in which there are tolerances or differences that can obtain the same function. Also, in this specification, expressions representing shapes such as a rectangular shape or a cylindrical shape not only represent shapes such as a rectangular shape or a cylindrical shape in a geometrically strict sense, but also represent shapes including concave and convex portions, chamfered portions, etc. within a range where the same effect can be obtained. Also, in this specification, the expression that a component "comprises", "includes", or "has" is not an exclusive expression excluding the existence of other components.

[0090] The present disclosure is not limited to the above-described embodiments, and includes forms obtained by modifying the above-described embodiments and forms obtained by appropriately combining these forms.

[0091] The content described in some of the above-described embodiments can be understood as follows, for example.

[0092] 1) The defect diagnosis model creation device (1) according to at least one embodiment of the present disclosure is A defect diagnosis model creation device (1) for creating a defect diagnosis model (11) that determines the presence or absence of defects in a target product from an image of the target product, an image acquisition unit (3) that acquires a plurality of defect images (DI) including at least one base image (BI) of a product identical or similar to the target product and at least one rare defect image (RDI) in which a defect with a generation probability equal to or lower than a predetermined value is imaged, a pseudo-defect image generation unit (4) that generates a pseudo-defect image (PDI) by inputting a composite image (CI) in which the defect image (DI) is pasted on a part of the base image (BI) into at least one restorer (7) that has previously learned the characteristics of the base image (BI), a teacher data acquisition unit (5) that acquires teacher data (TD) including a pseudo-defect image group composed of a plurality of the pseudo-defect images (PDI), and a defect diagnosis model creation unit (6) that creates a defect diagnosis model (11) by learning the teacher data (TD).

[0093] According to the configuration of 1) above, since the at least one restorer (7) has pre-learned the features (color tone, texture, structure, etc.) of the base image (BI), it is possible to generate a pseudo-defect image (PDI) in which the defect image (DI) intentionally synthesized in the base image (BI) is harmonized with the base image (BI) without a sense of incongruity. Since the composite image (CI) can be generated by pasting the defect image (DI) on a part of the base image (BI), when generating the composite image (CI), it is easy to process the defect of the defect image (DI), change the position of the defect in the composite image (CI), and change the type of defect included in the composite image (CI). The defect diagnosis model creation device (1) includes a rare defect image (RDI) with a rare defect shape in the defect image (DI) that is a component of the composite image (CI), so that a composite image (CI) or a pseudo-defect image (PDI) having a rare defect can be generated, and the generated pseudo-defect image (PDI) having a rare defect can be used as teacher data (TD). Thereby, since the defect diagnosis model creation device (1) can cause the defect diagnosis model (11) to learn teacher data (TD) considering the range (variation) of the possibility of actually occurring defects, it is possible to create a defect diagnosis model (11) that can accurately determine not only the presence or absence of frequently occurring defects but also the presence or absence of rare defects.

[0094] 2) In some embodiments, it is the defect diagnosis model creation device (1) described in 1) above, The defect diagnosis model creation device (1) is, further provided with an image combination adjustment unit (8) configured to adjust the ratio of the rare defect images (RDI) included in the plurality of defect images (DI) in the pseudo-defect image group.

[0095] According to the configuration of 2) above, by increasing the ratio of the rare defect images (RDI) included in the pseudo-defect image group by the image combination adjustment unit (8), the ratio of the rare defect images (RDI) in the teacher data (TD) can be increased, so that the discrimination accuracy of the defect diagnosis model (11) for the rare defect images (RDI) can be improved.

[0096] 3) In some embodiments, there is provided the defect diagnosis model creation device (1) described in 2) above, wherein the image combination adjustment unit (8) of the defect diagnosis model creation device (1) includes a detection accuracy evaluation unit (81) that evaluates the detection accuracy of the defect diagnosis model (11), and a ratio determination unit (82) that determines the ratio of the rare defect images (RDI) so that the detection accuracy of the defect diagnosis model (11) becomes a predetermined detection accuracy or higher based on the evaluation result of the detection accuracy evaluation unit (81).

[0097] According to the configuration of 3) above, the ratio determination unit (82) determines the ratio of the rare defect images (DI) so that the detection accuracy of the defect diagnosis model (11) becomes a predetermined detection accuracy or higher based on the evaluation result of the detection accuracy evaluation unit (81). Thus, the ratio of the rare defect images (RDI) in the training data (TD) can be adjusted so that the detection accuracy of the defect diagnosis model (11) can be improved, and the discrimination accuracy of the defect diagnosis model (11) for rare defects can be improved.

[0098] 4) In some embodiments, there is provided the defect diagnosis model creation device (1) according to any one of 1) to 3) above, wherein the at least one restorator (7) includes a first restorator (7A) that has learned in advance the features of the first base image (BI1), and a second restorator (7B) that has learned in advance the features of a second base image (BI2) whose features are different from those of the first base image (BI1).

[0099] According to the configuration of 4) above, the defect diagnosis model creation device (1) includes a plurality of restorators (the first restorator 7A and the second restorator 7B) that have learned in advance the features of base images (BI1, BI2) with different features individually. Therefore, pseudo-defect images (PDI) corresponding to base images (BI1, BI2) with different features individually can be generated. As a result, the types of pseudo-defect images (PDI) that the defect diagnosis model creation device (1) can generate can be increased, and the training data (TD) can have a wider range. Thus, the defect discrimination accuracy of the defect diagnosis model (11) can be improved.

[0100] 5) In some embodiments, there is provided a defect diagnosis model creation device (1) according to any one of 1) to 4) above, wherein the at least one restorer (7) is configured to learn features of a reduced base image (RBI) obtained by reducing the base image (BI) through reduction processing, and the defect diagnosis model creation device (1) further includes a low pixelation processing unit (12) configured to perform low pixelation processing on at least one of the combination of the base image (BI) and the defect image (DI) or the composite image (CI), and a high pixelation processing unit (13) configured to perform high pixelation processing on the pseudo defect image (PDI).

[0101] According to the configuration of 5) above, since the at least one restorer (7) learns features of the reduced base image (RBI), the performance required for the hardware used for learning can be suppressed as compared with the case of learning features of the base image (BI) before reduction processing. Since the image size that can be generated as the pseudo defect image (PDI) depends on the image size learned by the restorer (7), in the low pixelation processing unit (12), low pixelation processing is performed on the composite image (CI) input to the restorer (7) or the combination of the base image (BI) and the defect image (DI) that is the basis of the composite image (CI). For products with a large target product size and small defects, inspection at a high pixel count is desirable. Therefore, in the high pixelation processing unit (13), high pixelation processing is performed on the pseudo defect image (PDI) that serves as the teacher data (TD) of the defect diagnosis model (11) so as to enable inspection at a high pixel count. According to the configuration of 5) above, by providing the low pixelation processing unit (12) and the high pixelation processing unit (13), it becomes possible to generate a pseudo defect image (PDI) of a product that cannot be applied due to the performance of the hardware (specifically, a product with a large image size and small defects. That is, a product that cannot be photographed without increasing the image size), and the products to which the defect diagnosis model creation device (1) can be applied can be increased.

[0102] 6) In some embodiments, the defect diagnosis model creation apparatus (1) described in 5) above, A first fitness evaluation model (15) that has been pre-trained with teacher data for fitness evaluation (GFE) including a defective image of the product in which a defect has occurred and a non-defective image of the product in which no defect has occurred, and A first non-conforming image removal unit (16) configured to exclude, from candidates for the teacher data (TD), the pseudo-defect image (PDI) in which the fitness in the first fitness evaluation model (15) is less than a first threshold (FT) among the pseudo-defect images (PDI) on which the high-pixel processing has been performed by the high-pixel processing unit (13). The first fitness evaluation unit (14) further includes.

[0103] According to the configuration of 6) above, the quality of the pseudo-defect image (PDI) output from the restorator (7) and on which high-pixel processing has been performed can be evaluated by the first fitness evaluation model (15) that has been pre-trained with teacher data for fitness evaluation (GFE) including a defective image and a non-defective image. The first non-conforming image removal unit (16) can exclude, from candidates for the teacher data (TD), the pseudo-defect image (PDI) in which the fitness in the first fitness evaluation model (15) is insufficient (less than the first threshold FT). Thereby, it is possible to suppress the defect diagnosis model (11) from learning an unrealistic defect image (DI) that does not physically occur as the teacher data (TD) and degrading the defect diagnosis accuracy.

[0104] 7) In some embodiments, the defect diagnosis model creation apparatus (1) described in any one of 1) to 6) above, A second fitness evaluation model (18) that has been pre-trained with teacher data for fitness evaluation (GFE) including a defective image of the product in which a defect has occurred and a non-defective image of the product in which no defect has occurred, and A second non-conforming image removal unit (19) configured to exclude, from candidates for the teacher data (TD), the composite image (CI) in which the fitness in the second fitness evaluation model (18) is less than a second threshold (ST). The second fitness evaluation unit (17) further includes.

[0105] According to the configuration of 7) above, the quality of the synthetic image (CI) can be evaluated by the second fitness evaluation model (18) that has been pre-trained with the fitness evaluation teacher data (GFE) including the defective image and the non-defective image. The second non-conforming image removal unit (19) can exclude from the candidates of the teacher data (TD) the synthetic images (CI, for example, synthetic images with defects in parts that do not actually occur) whose fitness in the second fitness evaluation model (18) is insufficient (less than the second threshold value). Thereby, it is possible to suppress the defect diagnosis model (11) from learning a pseudo-defect image (PDI) including an unrealistic defect image (DI) that does not physically occur as the teacher data (TD) and reducing the defect diagnosis accuracy.

[0106] 8) In some embodiments, there is provided the defect diagnosis model creation device (1) described in 6) above, further comprising a first threshold adjustment unit (91) configured to adjust the first threshold value (FT) in consideration of the detection accuracy of the defect diagnosis model (11).

[0107] According to the configuration of 8) above, in the first threshold adjustment unit (91), by adjusting the first threshold value (FT) in consideration of the detection accuracy of the defect diagnosis model (11), in the first non-conforming image removal unit (16), it is possible to accurately remove from the candidates of the teacher data (TD) the pseudo-defect images (PDI) including unrealistic defect images (DI) while leaving the pseudo-defect images (PDI) including rare defect images (RDI) as candidates of the teacher data (TD). Thereby, the detection accuracy of the defect diagnosis model (11) can be improved.

[0108] 9) In some embodiments, there is provided the defect diagnosis model creation device (1) described in 7) above, further comprising a second threshold adjustment unit (92) configured to adjust the second threshold value (ST) in consideration of the detection accuracy of the defect diagnosis model (11).

[0109] According to the configuration of the above (9), in the second threshold adjustment unit (92), by adjusting the second threshold (ST) in consideration of the detection accuracy of the defect diagnosis model (11), in the second non-conforming image removal unit (19), a synthetic image (CI) including a rare defect image (RDI) as a candidate for the teacher data (TD) can be left while accurately removing a synthetic image (CI) including an unrealistic defect image (DI) from the candidates for the teacher data (TD). Thereby, the detection accuracy of the defect diagnosis model (11) can be improved.

[0110] 10) In some embodiments, there is provided a defect diagnosis model creation device (1) according to any one of the above (6) to (9), further comprising a relearning unit (93) that adds an image similar to the defective image when the detection accuracy of the defect diagnosis model (11) is equal to or lower than a predetermined value to the fitness evaluation teacher data (GFE).

[0111] According to the configuration of the above (10), in the relearning unit (93), by adding an image similar to a defective image when the detection accuracy of the defect diagnosis model (11) is equal to or lower than a predetermined value to the fitness evaluation teacher data (GFE), an image that the defect diagnosis model (11) needs to learn (an image with low detection accuracy of the defect diagnosis model) can be learned by the first fitness evaluation model (15) and the second fitness evaluation model (18), and the evaluation accuracy of the fitness in the first fitness evaluation model (15) and the second fitness evaluation model (18) for the above similar images (images with low detection accuracy of the defect diagnosis model) can be improved. By improving the evaluation accuracy of the fitness in the first fitness evaluation model (15) and the second fitness evaluation model (18), it is possible to suppress the image that the defect diagnosis model (11) needs to learn from being excluded from the candidates for the teacher data (TD) in the first non-conforming image removal unit (16) and the second non-conforming image removal unit (19). Thereby, since an image (teacher data TD) that the defect diagnosis model (11) needs to learn can be learned by the defect diagnosis model (11), the detection accuracy of the defect diagnosis model (11) can be improved.

[0112] 11) In some embodiments, there is provided a defect diagnosis model creation device (1) according to any one of the above (1) to (10), The apparatus further includes a defective image processing unit (94) configured to perform an image processing operation for changing the features of the defect included in the defective image (DI) to features similar to the defect.

[0113] According to the configuration of 11) above, the defective image processing unit (94) performs an image processing operation for changing the features of the defect included in the defective image (DI) to features similar to the defect, thereby increasing the number of synthesized images (CI) to be created, and generating pseudo-defective images (PDI) corresponding to the created synthesized images (CI). By including the defective image processing unit (94), the defective diagnosis model creation apparatus (1) can increase the pseudo-defective images (PDI), and can cause the defective diagnosis model (11) to learn a large amount of teacher data (TD). Therefore, the determination accuracy of the presence or absence of defects in the defective diagnosis model (11) can be improved.

[0114] 12) In some embodiments, the defective diagnosis model creation apparatus (1) according to any one of 1) to 11) above, further includes a pasting position determination unit (95) configured to randomly determine a pasting position of the defective image (DI) on the base image (BI).

[0115] According to the configuration of 12) above, the pasting position determination unit (95) randomly determines the pasting position of the defective image (DI) on the base image (BI), thereby increasing the number of synthesized images (CI) to be created, and generating pseudo-defective images (PDI) corresponding to the created synthesized images (CI). By including the pasting position determination unit (95), the defective diagnosis model creation apparatus (1) can increase the pseudo-defective images (PDI), and can cause the defective diagnosis model (11) to learn a large amount of teacher data (TD). Therefore, the determination accuracy of the presence or absence of defects in the defective diagnosis model (11) can be improved.

[0116] 13) A defective diagnosis model creation method (100) according to at least one embodiment of the present disclosure is a defective diagnosis model creation method (100) for creating a defective diagnosis model (11) that determines the presence or absence of defects in a target product from an image of the target product, An image acquisition step (S1) of acquiring a plurality of defect images (DI) including at least one base image (BI) in which a product identical or similar to the target product is photographed, and at least one rare defect image (RDI) in which a defect occurring in the product with a generation probability equal to or less than a predetermined value is photographed. A pseudo-defect image generation step (S2) of generating a pseudo-defect image (PDI) by inputting a composite image (CI) in which the defect image (DI) is pasted on a part of the base image (BI) into at least one restorer (7) that has previously learned the features of the base image (BI). A teacher data acquisition step (S3) of acquiring teacher data (TD) including a pseudo-defect image group composed of a plurality of the pseudo-defect images (PDI). A defect diagnosis model creation step (S4) of creating a defect diagnosis model (11) by learning the teacher data (TD).

[0117] According to the defect diagnosis model creation method (100) described in the above 13), a composite image (CI) or a pseudo-defect image (PDI) having a rare defect can be generated, and the generated pseudo-defect image (PDI) having a rare defect can be used as teacher data (TD). According to the defect diagnosis model creation method (100), since the teacher data (TD) considering the range (variation) of the possibility of actual occurrence of a defect can be learned by the defect diagnosis model (11), not only the presence or absence of a frequently occurring defect but also the presence or absence of a rare defect can be accurately determined, and a defect diagnosis model (11) can be created.

[0118] 14) The defect diagnosis model creation program (100A) according to at least one embodiment of the present disclosure is A defect diagnosis model creation program (100A) for creating a defect diagnosis model (11) for determining the presence or absence of a defect in a target product from an image of the target product, An image acquisition step (S1) of acquiring a plurality of defect images (DI) including at least one base image (BI) in which a product identical or similar to the target product is photographed, and at least one rare defect image (RDI) in which a defect that has occurred in the product and has a generation probability equal to or less than a predetermined value is photographed. A pseudo-defect image generation step (S2) of generating a pseudo-defect image (PDI) by inputting a composite image (CI) in which the defect image (DI) is pasted on a part of the base image (BI) into at least one restorer (7) that has previously learned the features of the base image (BI). A teacher data acquisition step (S3) of acquiring teacher data (TD) including a group of pseudo-defect images composed of a plurality of the pseudo-defect images (PDI). A defect diagnosis model creation step (S4) of creating a defect diagnosis model (11) by learning the teacher data (TD). This is a program for causing a computer to execute the steps.

[0119] According to the defect diagnosis model creation program (100A) described in 14) above, a composite image (CI) having a rare defect and a pseudo-defect image (PDI) can be generated, and the generated pseudo-defect image (PDI) having a rare defect can be used as teacher data (TD). According to the defect diagnosis model creation program (100A), since the teacher data (TD) considering the range (variation) of the possibility of actual defect occurrence can be learned by the defect diagnosis model (11), not only the presence or absence of frequently occurring defects but also the presence or absence of rare defects can be accurately determined. A defect diagnosis model (11) can be created.

Explanation of Signs

[0120] 1 Defect diagnosis model creation device 2 Defect diagnosis device 3 Image acquisition unit 4 Pseudo-defect image generation unit 5 Teacher data acquisition unit 6 Defect diagnosis model creation unit 7, 7A, 7B Restorers 8 Image combination adjustment unit 10 Defect Diagnosis System 11 Defect Diagnosis Model 12 Low Pixelation Processing Unit 13 High Pixelation Processing Unit 14 First Fitness Evaluation Unit 15 First Fitness Evaluation Model 16 First Non - conforming Image Removal Unit 17 Second Fitness Evaluation Unit 18 Second Fitness Evaluation Model 19 Second Non - conforming Image Removal Unit 21 Defect Diagnosis Model Memory Unit 22 Target Product Image Acquisition Unit 23 Defect Diagnosis Unit 24 Judgment Result Output Unit 31 Image Database 41 Composite Image Creation Unit 42 Restorer Memory Unit 43 Image Generation Unit 44 Suspected Defect Image Memory Unit 81 Detection Accuracy Evaluation Unit 82 Ratio Determination Unit 91 First Threshold Adjustment Unit 92 Second Threshold Adjustment Unit 93 Re - learning Unit 94 Defect Image Processing Unit 95 Attachment Position Determination Unit 100 Defect Diagnosis Model Creation Method 100A Defect Diagnosis Model Creation Program ASI Actual Shooting Image BI, BI1, BI2, BI3 Base Image CI Composite Image DI Defect Image FT First Threshold GFE Fitness Evaluation Teacher Data PDI Suspected Defect Image PDIG Suspected Defect Image Group RBI Reduced Base Image RDI Rare Defect Image S1 Image Acquisition Step S2 Suspected Defect Image Generation Step S3 Teacher Data Acquisition Step S4 Defect Diagnosis Model Creation Step ST Second Threshold TD Training Data

Claims

1. A defect diagnosis model creation device for creating a defect diagnosis model for determining the presence or absence of defects in a target product from an image of the target product, An image acquisition unit that acquires at least one base image of a product that is the same as or similar to the target product, and a plurality of defect images in which defects that have occurred in the product are photographed, including at least one rare defect image in which a defect with a probability of occurrence of a predetermined value or less is photographed, A pseudo-defect image generation unit that generates a pseudo-defect image by inputting a composite image in which the defect image is pasted on a part of the base image into at least one restorer that has previously learned the characteristics of the base image, A teacher data acquisition unit that acquires teacher data including a pseudo-defect image group composed of a plurality of the pseudo-defect images, A defect diagnosis model creation unit that creates a defect diagnosis model by learning the teacher data, Defect diagnosis model creation device.

2. The defect diagnosis model creation device, Further includes an image combination adjustment unit configured to adjust the ratio of the rare defect images included in the plurality of defect images in the pseudo-defect image group, The defect diagnosis model creation device according to claim 1.

3. The image combination adjustment unit, A detection accuracy evaluation unit that evaluates the detection accuracy of the defect diagnosis model, Based on the evaluation result of the detection accuracy evaluation unit, a ratio determination unit that determines the ratio of the rare defect images so that the detection accuracy of the defect diagnosis model becomes a detection accuracy of a predetermined value or more, The defect diagnosis model creation device according to claim 2.

4. The at least one restorer includes a first restorer that has previously learned the characteristics of a first base image, and a second restorer that has previously learned the characteristics of a second base image that is different in characteristics from the first base image, The defect diagnosis model creation device according to any one of claims 1 to 3.

5. The at least one restorator is configured to learn features of a reduced base image obtained by reducing the base image. The defect diagnosis model creation device A low-pixelation processing unit configured to perform low-pixelation processing on at least one of the combination of the base image and the defect image or the composite image, A high-pixelation processing unit configured to perform high-pixelation processing on the pseudo defect image, further comprising: The defect diagnosis model creation device according to any one of claims 1 to 4.

6. A first fitness evaluation model that has been pre-learned with teacher data for fitness evaluation including a defective image of the product in which a defect has occurred and a non-defective image of the product in which no defect has occurred, and A first non-conforming image removal unit configured to exclude, from candidates for the teacher data, the pseudo defect images in which the fitness in the first fitness evaluation model is less than a first threshold among the pseudo defect images on which the high-pixelation processing has been performed by the high-pixelation processing unit, further comprising a first fitness evaluation unit. The defect diagnosis model creation device according to claim 5.

7. A second fitness evaluation model that has been pre-learned with teacher data for fitness evaluation including a defective image of the product in which a defect has occurred and a non-defective image of the product in which no defect has occurred, and A second non-conforming image removal unit configured to exclude, from candidates for the teacher data, the composite images in which the fitness in the second fitness evaluation model is less than a second threshold, further comprising a second fitness evaluation unit. The defect diagnosis model creation device according to any one of claims 1 to 6.

8. Further comprising a first threshold adjustment unit configured to adjust the first threshold in consideration of the detection accuracy of the defect diagnosis model. The defect diagnosis model creation device according to claim 6.

9. Further comprising a second threshold adjustment unit configured to adjust the second threshold in consideration of the detection accuracy of the defect diagnosis model. The defect diagnosis model creation device according to claim 7.

10. Further comprising a relearning unit configured to add, to the fitness evaluation teacher data, an image similar to the defective image in which the detection accuracy of the defect diagnosis model is equal to or lower than a predetermined value. The defect diagnosis model creation device according to any one of claims 6 to 9.

11. Further comprising a defective image processing unit configured to perform an image processing operation for changing the characteristics of the defect included in the defective image to characteristics similar to the defect. The defect diagnosis model creation device according to any one of claims 1 to 10.

12. Further comprising a pasting position determination unit configured to randomly determine a pasting position of the defective image on the base image. The defect diagnosis model creation device according to any one of claims 1 to 11.

13. A defect diagnosis model creation method for creating a defect diagnosis model for determining the presence or absence of a defect in a target product from an image of the target product, comprising: An image acquisition step of acquiring a plurality of defective images including at least one base image of a product identical or similar to the target product and a plurality of defective images of defects occurring in the product, the plurality of defective images including at least one rare defective image in which a defect having a generation probability equal to or lower than a predetermined value is photographed; A pseudo-defective image generation step of generating a pseudo-defective image by inputting a composite image in which the defective image is pasted on a part of the base image into at least one restorer that has previously learned the characteristics of the base image; A teacher data acquisition step of acquiring teacher data including a pseudo-defective image group composed of a plurality of the pseudo-defective images; A defect diagnosis model creation step of creating a defect diagnosis model by learning the teacher data, and A method for creating a defect diagnosis model.

14. A defect diagnosis model creation program for creating a defect diagnosis model that determines the presence or absence of a defect in a target product from an image of the target product, An image acquisition step of acquiring a plurality of defect images including at least one base image of a product identical or similar to the target product and a plurality of defect images of defects that have occurred in the product, and including at least one rare defect image of a defect having a generation probability equal to or less than a predetermined value, A pseudo-defect image generation step of generating a pseudo-defect image by inputting a composite image in which the defect image is pasted on a part of the base image into at least one restorer that has previously learned the characteristics of the base image, A teacher data acquisition step of acquiring teacher data including a pseudo-defect image group composed of a plurality of the pseudo-defect images, A defect diagnosis model creation program for causing a computer to execute a defect diagnosis model creation step of creating a defect diagnosis model by learning the teacher data.

Images