Classification model generation system, classification model generation method, and program

Abstract

This classification model generating system comprises: an acquiring unit (11) for acquiring a non-defective product image group including a plurality of non-defective product images; a defective part image generating unit (12) for generating a plurality of defective part images on the basis of a seed image group obtained by geometrically transforming seed images which are generated by artificial rendering and which simulate defective parts; a combination processing unit (13) for generating a defective image group by combining each of the plurality of defective part images with the non-defective product image group; and a classification model generating unit (23) for generating a classification model by performing classification learning using the non-defective product image group and a portion of the defective image group as a training image group.

Description

Technical Field

[0001] The present disclosure relates to a classification model generation system, a classification model generation method, and a program.

Background Art

[0002] When performing machine learning such as deep learning, it is known that a large number of images are required as learning data in order to ensure the accuracy of the model after machine learning.

[0003] However, for example, when performing machine learning on a model for inspection purposes in a manufacturing process, although many good product images can be easily obtained, it is often difficult to obtain many defective product images. This is because the number of defective product images obtained in the inspection process is often small relative to the population, that is, the total number of images for which the inspection process was performed. Further, even if a large number of defective product images can be obtained, it is common for there to be a bias in the defective modes. Therefore, it is difficult to secure variations in the defective product images used as learning data, and even if machine learning is performed on the model using the obtained defective product images and good product images, there is a problem that it is difficult to secure inspection accuracy that can withstand practical use.

[0004] Furthermore, even if a large number of defective product images can be obtained, a great deal of man-hours are required for the sorting work, and there is also a problem in terms of maintenance management that when reconstructing the model due to a change or addition of the defective mode, it is necessary to perform the sorting work again.

[0005] On the other hand, for example, Patent Document 1 discloses a defective product image generation method and the like that can proliferate defective product images. According to the method disclosed in Patent Document 1, a defective product image closer to the real thing can be generated from a good product image and a partial image including a defective part.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

[0007] [Non-Patent Document 1] Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, Hayit Greenspan: GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification. Neurocomputing Volume 321, 10 December 2018, Pages 321-331 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, the technology disclosed in Patent Document 1 requires creating a database of defective areas and then collecting and selecting defective images that contain defective areas in order to increase the number of defective images.

[0009] This disclosure is made in view of the circumstances described above, and aims to provide a classification model generation system, etc., that can generate a classification model capable of classifying good product images from defective images without collecting and sorting defective images. [Means for solving the problem]

[0010] To solve the above problems, a classification model generation system according to one embodiment of the present disclosure includes: an acquisition unit that acquires a group of good product images including a plurality of good product images; a defective part image generation unit that generates a plurality of defective part images based on a group of seed images obtained by geometrically transforming seed images that are artificially drawn and mimic defective parts; a synthesis processing unit that generates a group of defective images by synthesizing each of the plurality of defective part images with the group of good product images; and a classification model generation unit that generates a classification model by performing classification learning using the group of good product images and a part of the group of defective images as a group of training images.

[0011] These general or specific embodiments may be implemented as devices, methods, systems, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs, or as any combination of devices, methods, systems, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0012] According to the classification model generation system of this disclosure, it is possible to generate a classification model capable of classifying good product images from defective images without collecting and sorting defective images. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 shows an example of the configuration of a classification model generation system according to an embodiment. [Figure 2A] Figure 2A is a block diagram showing an example of the detailed configuration of an image generation device according to an embodiment. [Figure 2B] Figure 2B is a block diagram showing an example of the detailed configuration of the pre-processing unit shown in Figure 2A. [Figure 3A] Figure 3A is a conceptual diagram showing an example of a seed image according to the embodiment. [Figure 3B] Figure 3B is a conceptual diagram showing an example of a group of seed images obtained by geometrically transforming seed images according to the embodiment. [Figure 4]FIG. 4 is a diagram conceptually showing an example of a plurality of defective part images according to the embodiment. [Figure 5A] FIG. 5A is a diagram conceptually showing an example of a defective part image according to the embodiment. [Figure 5B] FIG. 5B is a diagram conceptually showing an example of an image obtained by adding an alpha channel to the background of the defective part image shown in FIG. 5A. [Figure 5C] FIG. 5C is a diagram conceptually showing an example of an image obtained by performing a geometric transformation of rotation on an image having a background with an added alpha channel shown in FIG. 5B. [Figure 5D] FIG. 5D is a diagram conceptually showing an example of a non-defective product image according to the embodiment. [Figure 5E] FIG. 5E is a diagram conceptually showing an example of a defective image candidate according to the embodiment. [Figure 6A] FIG. 6A is a diagram conceptually showing an example of a mask image according to the embodiment. [Figure 6B] FIG. 6B is a diagram conceptually showing an example of a non-defective product image with a mask according to the embodiment. [Figure 6C] FIG. 6C is a diagram conceptually showing an example of a defective image candidate with a mask according to the embodiment. [Figure 6D] FIG. 6D is a diagram conceptually showing an example of a difference image according to the embodiment. [Figure 7] FIG. 7 is a block diagram showing an example of a detailed configuration of a model generation device according to the embodiment. [Figure 8] FIG. 8 is a flowchart showing the processing of a classification model generation method of a classification model generation system according to the embodiment. [[ID=3~4]] [Figure 9] FIG. 9 is a diagram showing the flow of the processing of the classification model generation method according to Example 1.

Embodiments for Carrying Out the Invention

[0014] The embodiments described below are all specific examples of this disclosure. The numerical values, shapes, components, steps, and order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components. In addition, the contents of each embodiment can be combined.

[0015] (Embodiment) The following describes the classification model generation method of the classification model generation system 1 in the embodiment, with reference to the drawings.

[0016] [1 Classification Model Generation System 1] Figure 1 shows an example of the configuration of the classification model generation system 1 according to the embodiment.

[0017] The classification model generation system 1 according to this embodiment is a system for generating a classification model that can classify good product images and defective images without collecting and sorting defective images. A defective image is an image in which defects such as scratches or foreign objects are visible. For example, a defective image in the inspection process is an image that shows a defect in the manufactured product and indicates that the manufactured product is a defective product. A good product image is an image in which no defects are visible. For example, a good product image in the inspection process is an image that shows no defects in the manufactured product and indicates that the manufactured product has passed inspection and is a good product.

[0018] The classification model generation system 1, as shown in Figure 1, comprises an image generation device 10 and a model generation device 20. These may be connected by a communication network or by physical communication lines such as a bus. In this embodiment, the image generation device 10 and the model generation device 20 are described as having different hardware configurations, but this is not limited to this. The image generation device 10 and the model generation device 20 may be built on a single piece of hardware to constitute the classification model generation system 1.

[0019] The image generation device 10 generates a group of defective images from a seed image artificially depicting a defective area such as a scratch or foreign object, and a group of good product images obtained by augmenting acquired good product images. Further details will be described later.

[0020] The model generation device 20 uses the defective image group generated by the image generation device 10 and the good image group, which is augmented with good images by the image generation device 10, to generate a classification model of the defective teaching type, that is, a classification model capable of classifying good images and defective images. Further details will be described later.

[0021] [2 Image generation device 10] Figure 2A is a block diagram showing an example of the detailed configuration of the image generation device 10 according to the embodiment.

[0022] The image generation device 10 is implemented as a computer equipped with, for example, a processor (microprocessor), memory, a communication interface, etc. The image generation device 10 performs various functions by having the processor execute a control program stored in memory. It should be noted that some components of the image generation device 10 may be included in a cloud server and operate within it.

[0023] In this embodiment, the image generation device 10, as shown in Figure 1, comprises an acquisition unit 11, a defective area image generation unit 12, and a synthesis processing unit 13. Each component will be described below.

[0024] [2.1 Acquisition part 11] As shown in Figure 2A, the acquisition unit 11 comprises an input unit 111 and a good product image generation unit 112, and acquires a group of good product images that include multiple good product images.

[0025] The input unit 111 receives multiple images of good products. The input unit 111 is implemented, for example, by a communication interface or an input interface, and can acquire multiple images of good products by receiving multiple pre-prepared images of good products. These pre-prepared images of good products can be obtained, for example, in a manufacturing process such as an inspection process. The number of pre-prepared images of good products may be around 1,000, but it may also be around tens.

[0026] The good product image generation unit 112 generates one or more new good product images by geometrically transforming multiple good product images input to the input unit 111. The good product image generation unit 112 then obtains a group of good product images from the multiple good product images and the one or more newly generated good product images. Here, geometric transformations include, but are not limited to, scaling (resizing), rotation, translation, X-axis mirroring (X-axis symmetrical movement), Y-axis mirroring (Y-axis symmetrical movement), swapping of x and y coordinates (mirror), shearing, cropping, contrast change processing, brightness change processing, filtering processing, blurring processing, etc. Any geometric transformation that does not destroy the characteristics of the shape is acceptable. In this way, the good product image generation unit 112 augments (multiplies) the multiple good product images input to the input unit 111 to obtain a group of good product images. The number of good product images in the group is, for example, around 10,000, but it may be several thousand or even tens of thousands.

[0027] [2.2 Defective part image generation unit 12] As shown in Figure 2A, the defective area image generation unit 12 comprises a pre-processing unit 121, a GAN model 122, and a generation unit 123. The defective area image generation unit 12 generates multiple defective area images based on a group of seed images obtained by geometrically transforming seed images artificially generated to mimic defective areas. Here, a defective area image is an image showing a defect such as a scratch or foreign object that constitutes a defective area in a manufactured product, for example.

[0028] [2.2.1 GAN122] GAN model 122 is a generative model composed of an architecture called GAN (Generative Adversarial Networks) and is trained using a set of seed images. In this embodiment, GAN122 is composed of, for example, DCGAN (Deep Convolutional GAN) or WGAN (Wasserstein GAN), but is not limited to these. GAN122 may also be composed of SNGAN (Spectral Normalization for GAN) or LSGAN (Least Squares GAN), as long as it is composed of a GAN that has the feature to increase similar variations by referring to the trained images.

[0029] [2.2.2 Pre-processing step 121] Figure 2B is a block diagram showing an example of the detailed configuration of the pre-processing unit 121 shown in Figure 2A.

[0030] As shown in Figure 2B, the preprocessing unit 121 comprises a seed image generation unit 1211, a geometric transformation unit 1212, and a learning unit 1213, and performs preprocessing for generating defective area images, such as generating seed images and training the GAN model 122. The preprocessing unit 121 includes, for example, a computer including memory and a processor (microprocessor), and the processor executes a predetermined program stored in memory to realize the functions of each unit.

[0031] The seed image generation unit 1211 generates a seed image that mimics the defective area by artificially depicting it. Here, the seed image generation unit 1211 generates a seed image that consists of a defective area mimicking the defective area and a background, and the defective area and the background are separable by a single threshold. The background is preferably painted with a single color.

[0032] In this embodiment, the seed image generation unit 1211 runs an application such as paint software, and, in accordance with the user's operation, artificially draws a figure that mimics the defective part as the defect, thereby generating one seed image, which is an artificial image that will serve as the seed of the defect.

[0033] Figure 3A is a conceptual diagram showing an example of a seed image according to the embodiment.

[0034] In the example shown in Figure 3A, a rectangular frame is artificially drawn to represent the defect. Furthermore, the defect in the seed image is depicted in white, while the background is black. Ideally, the defect should be drawn with a color similar to the defect to be recognized, and the background should be a single color that does not intersect with the defect. By depicting a background painted in a single color in this way, the defect in the seed image and the background can be separated using a single threshold.

[0035] The geometric transformation unit 1212 performs geometric transformations on the seed images generated by the seed image generation unit 1211 to obtain a group of seed images. The geometric transformation unit 1212 includes a processor (not shown) and memory, and the geometric transformation process is realized by the processor executing a predetermined program stored in the memory.

[0036] In this embodiment, the geometric transformation unit 1212 applies geometric transformation processing, such as resizing, contrast changing, rotation, and stretching, to a single seed image generated by the seed image generation unit 1211, thereby increasing (adding) seed images that may contain defects. As a result, the geometric transformation unit 1212 can obtain a group of seed images consisting of the original single seed image and the multiple increased seed images.

[0037] Figure 3B is a conceptual diagram showing an example of a group of seed images obtained by geometrically transforming seed images according to the embodiment.

[0038] In the example shown in Figure 3B, a portion of the seed images shown in Figure 3A have undergone geometric transformations, such as resizing, rotation, and stretching.

[0039] The learning unit 1213 uses a set of seed images obtained by geometrically transforming the seed images to train the GAN model 122 on the features of the seed image set. The learning unit 1213 includes a processor and memory (not shown), and the processor executes a predetermined program stored in the memory to realize the learning process that trains the GAN model 122 using the seed image set. As a result, the learning unit 1213 can use the seed image set to train the GAN model 122 on the features of defect variations.

[0040] [2.2.3 Generation unit 123] The generation unit 123 generates multiple images of defective areas using a pre-trained GAN model 122. The generation unit 123 includes a processor (not shown) and memory, and the processor executes a predetermined program stored in the memory to realize a generation process that causes the pre-trained GAN model 122 to generate images of defective areas.

[0041] In this embodiment, the generation unit 123 can generate images of defective areas by applying noise to the trained GAN model 122. Therefore, the generation unit 123 can generate multiple images of defective areas by applying noise with varying values ​​to the trained GAN model 122.

[0042] Figure 4 is a conceptual diagram showing an example of multiple defective area images according to the embodiment.

[0043] In the example shown in Figure 4, some of the images of defective areas generated by the trained GAN model 122 by applying varied noise are shown. In Figure 4, each of the white shapes represents a defective area, or defect. As shown in Figure 4, it can be seen that by using the trained GAN model 122, it is possible to generate not only defects similar to the seed image set, but also defects that are similar to but different from the original defect.

[0044] In this way, the generation unit 123 can cause the trained GAN model 122 to generate not only images of defective regions with defects similar to those in the seed image set from which features were learned, but also images of defective regions with defects that are similar to but different from those defects.

[0045] [2.3 Synthesis Processing Unit 13] As shown in Figure 2A, the synthesis processing unit 13 comprises a geometric transformation unit 131, a synthesis unit 132, and an image selection unit 133. The synthesis processing unit 13 generates a group of defective images by combining each of the multiple defective area images with a group of good product images.

[0046] [2.3.1 Geometric transformation section 131] The geometric transformation unit 131 performs geometric transformations on multiple defective area images generated by the defective area image generation unit 12. The geometric transformation unit 131 includes a processor (not shown) and memory, and the processor executes a predetermined program stored in the memory to realize each process, including the geometric transformation process.

[0047] In this embodiment, the geometric transformation unit 131 separates each of the multiple defective area images generated by the defective area image generation unit 12 into a background and a defective area by binarizing them, and adds an alpha channel to the background, which acts as a transparent layer. The geometric transformation unit 131 then performs a geometric transformation on the image with the background to which the alpha channel has been added.

[0048] Figure 5A is a conceptual diagram showing an example of a defective area image according to the embodiment. Figure 5B is a conceptual diagram showing an example of an image in which an alpha channel has been added to the background of the defective area image shown in Figure 5A. Figure 5C is a conceptual diagram showing an example of an image in which a geometric transformation, which is rotation, has been performed on the image with the alpha channel added to the background shown in Figure 5B.

[0049] Specifically, the geometric transformation unit 131 separates the defective area image, for example shown in Figure 5A, generated by the defective area image generation unit 12, into a background and a defective area by binarizing it, and then adds an alpha channel to the separated background to obtain an image (image of the defect) for example shown in Figure 5B. The geometric transformation unit 131 then performs a rotational geometric transformation on the image with the background to which the alpha channel has been added, as shown in Figure 5B, to obtain the image shown in Figure 5C.

[0050] [2.3.2 Synthesis section 132] The synthesis unit 132 synthesizes each of the multiple defective area images into a group of good product images. The synthesis unit 132 includes a processor (not shown) and memory, and the synthesis process is realized by the processor executing a predetermined program stored in the memory.

[0051] In this embodiment, the synthesis unit 132 can obtain candidate defective images by combining an image that has undergone geometric transformation or other processing by the geometric transformation unit 131 and has a background with an alpha channel added, with a good product image. Therefore, the synthesis unit 132 can obtain candidate defective image groups by combining each of the multiple defective area images that have undergone geometric transformation or other processing by the geometric transformation unit 131 with the good product image group. If the candidate defective image groups are not selected by the image selection unit 133 described later, all candidate defective image groups become a defective image group; if they are selected, only the selected candidate defective image groups become a defective image group.

[0052] Figure 5D is a conceptual diagram showing an example of a good product image 50 according to the embodiment. Figure 5E is a conceptual diagram showing an example of a defective image candidate 51 according to the embodiment.

[0053] In other words, the synthesis unit 132 can obtain a candidate defective image 51, such as the one shown in Figure 5E, by synthesizing the image shown in Figure 5C, which has been processed by the geometric transformation unit 131 on the image shown in Figure 5B, with the image of a good product shown in Figure 5D. The image shown in Figure 5C is just one example, and various geometric transformations can be performed on the image shown in Figure 5B. Therefore, the synthesis unit 132 can obtain variations of the candidate defective image 51 by synthesizing images of defective areas that have undergone various geometric transformations with a group of good product images.

[0054] [2.3.3 Image sorting unit 133] The image selection unit 133 determines whether the image (candidate for a defective image) synthesized by the synthesis unit 132 is valid as a defective image. The image selection unit 133 includes a processor (not shown) and memory, and the processor executes a predetermined program stored in the memory to perform processing such as the discrimination process.

[0055] More specifically, the image selection unit 133 determines that a composite image is valid as a defective image if the area and brightness of the defective parts shown in the composite image meet predetermined criteria. The image selection unit 133 determines that a composite image is not valid as a defective image if the area and brightness of the defective parts shown in the composite image do not meet predetermined criteria. Alternatively, the image selection unit 133 may use a difference image obtained by taking the difference between the good product image 50 shown in Figure 5D and the defective image candidate 51 shown in Figure 5E to determine whether the area and brightness of the defective parts (defective areas) included in the difference image meet predetermined criteria. The image selection unit 133 then includes only the images that it has determined to be valid from the composite image into the group of defective images.

[0056] Furthermore, if an image of a good product includes an area outside the manufactured product region, a candidate image of a defective product with a defective area (flaw) superimposed on that area is not effective for training the classification model. In other words, a good product image may contain areas where we do not want to superimpose a defective area. In such cases, a mask image is prepared by filling the areas where we do not want to superimpose a defective area with black, and the image (candidate image of a defective product) synthesized by the synthesis unit 132 can be used to determine whether it is a valid defective image.

[0057] More specifically, first, the image selection unit 133 obtains a masked defective image candidate by combining a prepared mask image with a defective image candidate, and a masked good image by combining the mask image with the good image that was the source of the defective image candidate, from the synthesis unit 132. Next, the image selection unit 133 generates a difference image by taking the difference between these (masked defective image candidate and masked good image). The image selection unit 133 may then measure the brightness and area of ​​the defective parts included in the generated difference image and determine whether it meets a predetermined standard. As a result, defective image candidates in which defective parts are combined with areas of good images where defective parts are not to be combined are excluded and excluded from the training image group used to train the classification model described later.

[0058] Figure 6A is a conceptual diagram showing an example of a mask image 52 according to the embodiment. Figure 6B is a conceptual diagram showing an example of a masked good product image 50a according to the embodiment. In other words, Figure 6B shows an example of a masked good product image 50a obtained by the image sorting unit 133 and the synthesis unit 132 synthesizing the mask image 52 shown in Figure 6A and the good product image 50 shown in Figure 5D.

[0059] Figure 6C is a conceptual diagram showing an example of a masked defective image candidate 51a according to the embodiment. Specifically, Figure 6C shows an example of a masked defective image candidate 51a obtained by the image sorting unit 133 and the synthesis unit 132 synthesizing the mask image 52 shown in Figure 6A and the defective image candidate 51 shown in Figure 5E.

[0060] Furthermore, Figure 6D is a conceptual diagram showing an example of a difference image 53 according to the embodiment. Specifically, Figure 6D shows an example of a difference image 53 generated by the image sorting unit 133 by taking the difference between the masked good product image 50a shown in Figure 6B and the masked defective image candidate 51a shown in Figure 6C.

[0061] In this way, the image sorting unit 133 generates a difference image 53, for example, as shown in Figure 6D, using the mask image shown in Figure 6A. The image sorting unit 133 then determines whether the brightness and area of ​​the defective parts included in the difference image 53 meet predetermined criteria. As a result, if the defective image candidate 51 shown in Figure 5E is a defective image candidate in which a defective part is superimposed on a region of a good image where the defective part is not to be superimposed, the image sorting unit 133 can exclude the defective image candidate 51 shown in Figure 5E from the group of defective images.

[0062] [3 Model generation device 20] Figure 7 is a block diagram showing an example of the detailed configuration of the model generation device 20 according to the embodiment.

[0063] The model generation device 20 generates a classification model by performing classification learning based on the good product image group and defective product image group generated by the image generation device 10.

[0064] The model generation device 20 is implemented, for example, by a computer equipped with a processor (microprocessor), memory, communication interface, etc., and various functions are realized by the processor executing a control program stored in memory. The model generation device 20 may also operate with some components included in a cloud server.

[0065] In this embodiment, the model generation device 20, as shown in Figure 1, comprises a storage unit 21, an extraction unit 22, a classification model generation unit 23, and a learning DB 24. Each component will be described below.

[0066] [3.1 Storage section 21] The storage unit 21 is implemented by, for example, an HDD (Hard Disk Drive) or flash memory.

[0067] In this embodiment, the storage unit 21 stores the good image group and the defective image group generated by the image generation device 10 as a generated image group. The number of images included in the generated image group is, for example, on the order of several hundred thousand, but is not limited to this. The number of defective images included in the generated image group may be on the order of several tens to several hundred. In this case, the defective image group generated by the image generation device 10 is added to the generated image group, for example, when the model generation device 20 requests the image generation device 10 to add the defective image group.

[0068] [3.2 Extraction part 22] The extraction unit 22 extracts a portion of the generated images stored in the memory unit 21 and outputs them to the classification model generation unit 23 or stores them in the learning DB 24.

[0069] In this embodiment, the extraction unit 22 extracts a portion of the generated image group stored in the storage unit 21, namely a portion of the good image group and a portion of the defective image group, as initial training images and outputs them to the training DB 24 and the training unit 231. The initial training images extracted by the extraction unit 22 only need to be a small number; for example, they may contain 5 to 10 good images and 10 defective images, but are not limited to these. The number of good images and defective images included in the initial training images can be selected as appropriate, but if there are too few, the classification model will not work, and if there are too many, the training cost will be high, so it is desirable to have around 5 to 100 of each.

[0070] Alternatively, the extraction unit 22 may extract the generated images other than the initial training images from the generated image group stored in the storage unit 21 as evaluation images and output them to the evaluation unit 233.

[0071] [3.3 Classification Model Generation Unit 23] As shown in Figure 7, the classification model generation unit 23 comprises a learning unit 231, a classification model 232, an evaluation unit 233, and a determination unit 234. The classification model generation unit 23 generates the classification model 232 by performing classification learning using a portion of the good product image group and the defective product image group as training image groups.

[0072] The following describes each component.

[0073] [3.3.1 Classification Model 232] The classification model 232 is a discriminant model consisting of a model with an architecture having one or more convolutional layers. In this embodiment, the classification model 232 is composed of, for example, a CNN (Convolutional Neural Network), but is not limited to these. The classification model 232 is a model having one or more convolutional layers and is capable of classification learning to classify good product images and defective product images using a set of training images.

[0074] [3.3.2 Learning Section 231] The learning unit 231 generates a classification model by having the classification model 232 classify and learn a set of training images. The learning unit 231 includes a processor and memory (not shown), and the processor executes a predetermined program stored in the memory to perform a learning process that trains the classification model 232.

[0075] More specifically, the learning unit 231 first uses initial training images, which are a portion of the good product image group and the defective product image group, as the training image group to perform classification training on the classification model 232. The learning unit 231 then uses the training image group updated by the judgment unit 234 to train the classification model 232. In this way, the learning unit 231 generates a trained classification model 232 by repeating the classification training.

[0076] In this embodiment, the learning unit 231 first trains the classification model 232 using a small number of initial training images as a set of training images. Next, the learning unit 231 trains the classification model 232 with the updated set of training images, which is obtained by adding a set of incorrect images consisting of one or more incorrect images (described later) to the initial training images. Then, the learning unit 231 generates a trained classification model 232 by repeatedly training the classification model 232 with the repeatedly updated set of training images.

[0077] Furthermore, when the learning unit 231 trains the classification model 232 with the updated training image set, it may retrain the classification model 232. Alternatively, the parameters of the previously trained classification model 232 may be used as initial values, the layers constituting the previously trained classification model 232 may be frozen, and then the previously trained classification model 232 may be transferred and trained using the updated training image set.

[0078] [3.3.3 Evaluation Section 233] The evaluation unit 233 causes the classification model 232 to evaluate at least a portion of the good image group and the bad image group, excluding the training image group used for classification learning of the classification model 232. The evaluation result includes, but is not limited to, a determination result indicating whether the image input to the classification model 232 is a good image or a bad image. The evaluation result may include a determination result indicating whether the image input to the classification model 232 is a good image or a bad image, and the certainty of said determination result. The certainty of said determination result can be easily obtained from the layer immediately preceding the output layer of the classification model 232, which outputs a value indicating whether the image is a good image or a bad image. The evaluation unit 233 includes a processor and memory (not shown), and the evaluation process is realized by the processor executing a predetermined program stored in the memory.

[0079] In this embodiment, the evaluation unit 233 has the classification model 232 evaluate the evaluation images acquired from the extraction unit 22. As described above, the evaluation images are a part of the generated image group. In other words, the evaluation images are at least a part of the good image group and the bad image group, excluding the training image group used for classification learning of the classification model 232.

[0080] [3.3.4 Judgment unit 234] The determination unit 234 determines whether the evaluation result evaluated by the classification model 232 is incorrect or not. Here, the determination unit 234 determines that the evaluation result is incorrect if the determination result is incorrect or if the certainty of the determination result is less than or equal to a predetermined value. The determination unit 234 also updates the training image group by adding only the good product images or defective images that have been determined to be incorrect to the training image group.

[0081] Furthermore, the determination unit 234 can define learning termination conditions, such as the number of learning iterations and the number of evaluation images, and cause the learning unit 231 to perform classification learning of the classification model 232 using the updated set of learning images. In this case, the determination unit 234 terminates the classification learning of the learning unit 231 when the learning termination conditions are met.

[0082] The determination unit 234 includes a processor and memory (not shown), and these various processes are realized by the processor executing a predetermined program stored in the memory.

[0083] In this embodiment, the defective image group included in the evaluation images evaluated by the classification model 232 is generated by the image generation device 10. Therefore, it is already known whether the evaluation images evaluated by the classification model 232 are good images or defective images. Accordingly, the determination unit 234 can determine that the evaluation result evaluated by the classification model 232 is incorrect or that the certainty of the determination result is below a predetermined value. The determination unit 234 can then update the learning image group by adding only the incorrect images, which are good images or defective images, to the learning image group. In other words, the determination unit 234 can add only the incorrect images to the learning image group without performing a sorting operation, thereby increasing the number (data) of learning images.

[0084] Furthermore, the judgment unit 234 may update the training image group by adding only the incorrect images to the training image group as soon as even one of the evaluation images is found to be incorrect, and then perform classification learning of the classification model 232 using the updated training image group. Alternatively, the judgment unit 234 may update the training image group by adding the incorrect images to the training image group when it is determined that a predetermined number of evaluation images are incorrect, and then perform classification learning of the classification model 232 using the updated training image group. In this case, in order to improve the efficiency of learning, it is preferable to decrease the value set by the predetermined number as classification learning is repeated. This is because as the learning of the classification model 232 progresses, the number of times the evaluation result is determined to be incorrect decreases.

[0085] [3.4 Learning DB24] The learning DB24 is implemented using, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0086] The learning DB24 stores a set of training images, including at least the initial training images. The training image set is updated by the judgment unit 234. Initially, the training image set contains only the initial training images during the initial training of the classification model 232, but thereafter, it is updated by adding one or more incorrect images. As the training of the classification model 232 progresses, the training image set is updated to include both the initial training images and the incorrect image set.

[0087] [4 Operation of Classification Model Generation System 1] Next, an example of the operation of the classification model generation system 1, configured as described above, will be explained below.

[0088] Figure 8 is a flowchart showing the processing of the classification model generation method of the classification model generation system 1 according to the embodiment.

[0089] The classification model generation system 1 comprises a processor and memory, and uses the processor and the program stored in memory to perform the following steps S1 to S4.

[0090] More specifically, first, the classification model generation system 1 acquires a set of good product images containing multiple good product images (S1).

[0091] Next, the classification model generation system 1 generates multiple images of defective parts based on a set of seed images obtained by geometrically transforming seed images that mimic defective parts, which are seed images that have been artificially drawn (S2).

[0092] Next, the classification model generation system 1 generates a group of defective images by combining each of the multiple defective area images with a group of good product images (S3).

[0093] Next, the classification model generation system 1 generates a classification model by performing classification learning using a portion of the good product image set and the defective product image set as training image sets (S4).

[0094] (Example 1) Figure 9 shows the processing flow of the classification model generation method according to Example 1.

[0095] Figure 9 shows a specific example of the processing of a classification model generation method performed using the classification model generation system 1. It shows that after the preparation phase, an image generation phase is performed, followed by a learning and evaluation phase. It also shows that the image generation phase and the learning and evaluation phase can be looped.

[0096] As shown in Figure 9, the preliminary preparation phase is carried out first.

[0097] More specifically, the classification model generation system 1 first generates a seed image, which is an artificial image that will serve as the seed of a defect, by artificially depicting a shape that mimics the defective part as the defect, in accordance with the user's operation (S11). Next, it performs a geometric transformation process on the seed image generated in step S11 (S12). This allows it to obtain a set of seed images that augment the number of seed images that could become defects. Next, it performs a training process on the GAN model using the set of seed images (S13). This allows the GAN model to learn the characteristics of variations in defective parts (defects) from the set of seed images.

[0098] After this preliminary preparation phase is completed, the image generation phase takes place. Multiple images of good products are prepared in advance, for example, by collecting them from the actual manufacturing process.

[0099] More specifically, the classification model generation system 1 first receives a set of pre-prepared images of good products as input, and then performs a geometric transformation on the input images of good products (S14). As a result, the classification model generation system 1 can augment the multiple images of good products to obtain a diverse set of images of good products.

[0100] Furthermore, the classification model generation system 1 performs a generation process to generate multiple images of defective areas by applying noise with varying values ​​to the GAN model that was trained in the preparation stage (S15). By using the GAN model in this way, the classification model generation system 1 can generate variations in shape that would be difficult to achieve with geometric transformations alone, thus enabling the acquisition of diverse images of defective areas.

[0101] Next, the classification model generation system 1 performs geometric transformation processing on the multiple defective area images generated in step S15 (S16) and then performs image synthesis processing to combine them with the group of good product images (S17). As a result, the classification model generation system 1 can obtain candidate defective area images in which various defective areas with different shapes or positions are combined with the good product images.

[0102] Next, the classification model generation system 1 prepares a mask image in which the areas where defective parts should not be synthesized are filled with black (S18), and performs an image selection process to determine whether the candidate defective images synthesized in step S17 are valid as defective images (S19). As a result, the classification model generation system 1 can obtain a group of defective images by selecting and excluding candidate defective images in which defective parts have been synthesized in areas of good images where defective parts should not be synthesized. The group of good images obtained in step S14 and the group of defective images obtained in step S19 are used as a group of generated images in the learning and evaluation phase.

[0103] After the image generation phase is completed as described above, the learning and evaluation phase is performed.

[0104] More specifically, the classification model generation system 1 first extracts a portion of the generated image set, namely a portion of the good product image set and a portion of the defective product image set, as initial training images (S20). Here, the extracted initial training images consist of about 10 to 30 good product images and defective product images. Next, the classification model generation system 1 performs training processing on the classification model using the initial training images extracted in step S20 (S21). Next, the classification model generation system 1 extracts the generated image set other than the initial training images from the generated image set as evaluation images (S22), and performs evaluation processing to have the classification model evaluate the extracted evaluation images (S23). As a result, the classification model generation system 1 can obtain evaluation results evaluated by the classification model trained on the initial training images.

[0105] Next, the classification model generation system 1 performs a determination process to determine whether the evaluation result, evaluated by the classification model trained on the initial training images, is incorrect or not (S24). Here, the evaluation images are generated in the image generation phase, and it is already known whether each evaluation image is a good product image or a defective product image. Therefore, in the determination process of step S24, if the evaluation result is incorrect and the evaluation image of the evaluation result is determined to be an incorrect image, the training image group is updated by adding only the incorrect image to the training image group. In this way, the classification model generation system 1 can increase the number of training images by accumulating only the incorrect images in the training image group.

[0106] Next, the classification model generation system 1 determines the learning termination conditions and performs a learning process to train the classification model using the updated training image set in step S24 (S25), while making a learning termination determination. As a result, the classification model generation system 1 can terminate the learning process of the classification model when the learning termination conditions are reached, and can repeat the learning process while only incorrect answers are fed back.

[0107] Note that there may be insufficient evaluation images to meet the learning termination conditions. In this case, the image generation phase can be repeated to increase the number of generated images, and then the evaluation process in step S23 to the learning termination determination in step S25 can be performed. The image generation phase and the learning / evaluation phase can be looped until the learning termination conditions are met (S26).

[0108] (Example 2) In Example 2, the effectiveness of the classification model generation method according to the embodiment was verified. The verification results are briefly described below.

[0109] In this verification, we used one artificially generated seed image to represent a foreign object and 1000 pre-collected images of good products. The learning completion condition was defined as passing 30,000 evaluation images without errors, or completing 30 learning loops. In this verification, we used VGG16, a 16-layer CNN model, as the classification model, Adam as the optimizer (optimization algorithm), and a learning rate of 10 -6 The number of epochs was set to 30. The learning rate was varied depending on the number of training iterations.

[0110] As a result of this verification, the following findings were obtained. • Learning loop: 7 times • Final number of training images: 517 OK / 102 NG • Final accuracy: 100%

[0111] From the verification results above, it was found that the classification model generation method according to the embodiment can achieve 100% accuracy in the classification model.

[0112] In this way, the effectiveness of the classification model generation method according to the embodiment was verified.

[0113] [5 Effects, etc.] As described above, according to the embodiment, the classification model generation system 1, etc., can generate a classification model capable of classifying good product images and defective images without collecting and sorting defective images.

[0114] For example, when collecting images of defective products from actual manufacturing processes, it requires the effort of collecting the images and then sorting them by adding annotations. Although there is a method of collecting only images of good products and generating a classification model using only those images, it becomes difficult to set criteria for judging defective images, such as defining the defect modes (defects) that you want to identify.

[0115] On the other hand, in this embodiment, good product images are collected from the actual manufacturing process, but defective product images are not collected from the actual manufacturing process. Instead, defective product images are generated based on seed images that mimic artificially created defects (defective parts) such as scratches and foreign objects. This allows a classification model to be generated using the generated defective product images and a set of training images generated from the collected good product images. In other words, a classification model capable of classifying good product images and defective product images can be generated without collecting and sorting defective product images.

[0116] Furthermore, according to this embodiment, the collected images of good products can be augmented by geometric transformation or the like, and by combining the augmented images of good products with images of defective parts generated based on the seed image, a variety of defective images can be generated.

[0117] Here, to generate images of defective areas, a GAN model trained on a set of seed images augmented by geometrically transforming seed images can be used. This allows for the rough setting of the range and shape of the actual defective area by artificially creating (generating) seed images, thus enabling the generation of variations in the shape of defective area images that would be difficult to achieve with geometric transformation alone.

[0118] Furthermore, according to this embodiment, the generated classification model is evaluated, and only incorrect answers are fed back, while the classification model is repeatedly trained using the training image set, including the incorrectly identified images. This allows the classification model to grow to a higher accuracy while keeping the computational cost required for the training process low.

[0119] Furthermore, the training image set generated as a by-product includes only incorrect images, in addition to the images initially used as training images, resulting in a carefully selected training image set. This makes it easier to generate a more accurate classification model than training a classification model using images randomly selected from the generated defective and good images. In addition, since the number of images included in the training image set is relatively small, a more accurate classification model can be trained at a faster rate.

[0120] (Note) Furthermore, the following technologies are disclosed based on the above description of embodiments.

[0121] (Technology 1) A classification model generation system comprising: an acquisition unit that acquires a group of good product images including multiple good product images; a defective part image generation unit that generates multiple defective part images based on a group of seed images obtained by geometrically transforming seed images that are artificially drawn and mimic defective parts; a synthesis processing unit that generates a group of defective images by synthesizing each of the multiple defective part images with the group of good product images; and a classification model generation unit that generates a classification model by performing classification learning using the group of good product images and a part of the group of defective images as a group of training images.

[0122] In this way, defective images can be generated based on seed images that mimic artificially created defects (defective parts) such as scratches and foreign objects, without collecting defective images from actual manufacturing processes. This allows for the generation of a classification model using a set of training images generated from the generated defective images and collected good product images. In other words, a classification model capable of distinguishing between good product images and defective images can be generated without collecting and sorting defective images.

[0123] (Technology 2) The classification model generation system according to Technology 1, wherein the acquisition unit comprises an input unit into which the plurality of good product images are input, and a good product image generation unit that geometrically transforms the plurality of good product images input to the input unit to generate one or more new good product images, and acquires the plurality of good product images and the one or more new good product images as the good product image group.

[0124] In this way, multiple pre-prepared high-quality images can be augmented through geometric transformation. As a result, multiple high-quality images are augmented, yielding a diverse collection of high-quality images.

[0125] (Technology 3) The classification model generation system according to Technology 1 or Technology 2, wherein the classification model generation unit comprises a learning unit that causes the model to classify and learn the training image group, an evaluation unit that causes the model to evaluate at least a portion of the good product image group and the bad product image group, excluding the training image group used for the model's classification learning, and a determination unit that determines whether the evaluation result evaluated by the model is incorrect or not, and updates the training image group by adding only the good product image or bad product image determined to be incorrect to the training image group, and the learning unit generates the classification model by causing the model to classify and learn the updated training image group.

[0126] In this way, the generated classification model can be evaluated, and only incorrect answers can be fed back. The model can then be repeatedly trained using the training image set, including the incorrectly identified images. This allows for the development of a more accurate classification model while keeping the computational cost required for training low.

[0127] Furthermore, the training image set generated as a byproduct will only include incorrect images, in addition to the images initially used as training images, resulting in a carefully selected training image set. This makes it easier to generate a more accurate classification model than training a classification model using images randomly selected from the generated defective and good images. In addition, since the number of images included in the training image set is relatively small, a more accurate classification model can be trained at a faster rate.

[0128] (Technology 4) The classification model generation system according to Technology 3, wherein the evaluation result includes a determination result indicating whether the image input to the model is a good product image or a defective product image, and the likelihood of the determination result.

[0129] (Technical 5) The classification model generation system according to Technical 4, wherein the determination unit determines that the evaluation result is an incorrect answer if the determination result is incorrect or the probability is less than or equal to a predetermined value.

[0130] (Technical 6) The classification model generation system according to any one of Technical 1 to Technical 5, wherein the synthesis processing unit comprises a synthesis unit that synthesizes each of the plurality of defective part images into the group of good product images, and an image selection unit that determines whether the image synthesized by the synthesis unit is appropriate as a defective image.

[0131] This allows for the identification and exclusion of unsuitable, defective images, enabling the generation of more appropriate defective images.

[0132] (Technical 7) The classification model generation system according to Technical 6, wherein the image sorting unit determines that the synthesized image is valid as the defective image if the area and brightness of the defective parts shown in the synthesized image meet predetermined criteria, and determines that the synthesized image is not valid as the defective image if the area and brightness of the defective parts shown in the synthesized image do not meet predetermined criteria, and includes only the images that have been determined to be valid among the synthesized images in the group of defective images.

[0133] In this way, for example, defective images in which defective parts are superimposed on areas of good images where defective parts are not desired can be identified and excluded as invalid. This allows the classification model to be trained to accurately classify defective images that have a specific defect mode that you want to identify as defective images.

[0134] (Technology 8) The classification model generation system according to any one of Techniques 1 to 7, wherein the defective area image generation unit generates the defective area image using a trained GAN (Generative Adversarial Network) model, and the GAN model is trained using the seed image group.

[0135] In this way, by using a GAN model trained on a set of seed images based on seed images, it is possible to generate variations in the shape of defective areas that would be difficult to achieve with geometric transformations alone. This allows for the acquisition of diverse images of defective areas.

[0136] (Technology 9) The classification model generation system according to any one of Techniques 1 to 7, wherein the seed image consists of a defective part that mimics a defective area and a background, and the defective part and the background can be separated by a single threshold.

[0137] In this way, the defective parts of the seed image and the background can be separated by a single threshold, making it easy to perform processing such as making the background transparent. This allows for easy synthesis of a geometrically transformed seed image with a pre-prepared good image, thus easily generating defective images.

[0138] (Technology 10) The classification model generation system described in Technology 9, wherein the background is painted in a single color.

[0139] This makes it easy to perform processes such as making the background transparent.

[0140] (Possibility of other embodiments) Although the classification model generation system, classification model generation method, and program of this disclosure have been described in the embodiments above, there are no particular limitations on the entities or devices on which each process is performed. Processing may be performed by a processor (described below) embedded in a specific device located locally. Alternatively, processing may be performed by a cloud server or the like located in a different location from the local device.

[0141] This disclosure is not limited to the embodiments described above. For example, other embodiments realized by arbitrarily combining the components described herein, or by excluding some of the components, may also be considered embodiments of this disclosure. Furthermore, modifications obtained by applying various modifications to the above embodiments that a person skilled in the art could conceive of, without departing from the spirit of this disclosure, i.e., the meaning of the language used in the claims, are also included in this disclosure.

[0142] Furthermore, this disclosure also includes the following cases:

[0143] (1) Specifically, the above system is a computer system consisting of a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The microprocessor operates according to the computer program, and each device performs its function. Here, the computer program is composed of a combination of multiple instruction codes that indicate commands to the computer in order to achieve a predetermined function.

[0144] (2) Some or all of the components constituting the above system may be made up of a single system LSI (Large Scale Integration). The system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, it is a computer system that includes a microprocessor, ROM, RAM, etc. The RAM stores a computer program. The system LSI achieves its function by operating the microprocessor in accordance with the computer program.

[0145] (3) Some or all of the components constituting the above system may consist of detachable IC cards or standalone modules attached to each device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned multi-functional LSI. The microprocessor operates according to a computer program, thereby enabling the IC card or module to perform its function. The IC card or module may be tamper-resistant.

[0146] (4) The present disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of the computer program.

[0147] (5) The disclosure may also be a computer program or a digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, the digital signal may be recorded on one of these recording media.

[0148] Furthermore, this disclosure may also describe transmitting the computer program or digital signal via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0149] Furthermore, the present disclosure may also provide a computer system comprising a microprocessor and memory, wherein the memory stores the computer program, and the microprocessor operates in accordance with the computer program.

[0150] Furthermore, the program or digital signal may be implemented by another independent computer system by recording and transferring it on the recording medium, or by transferring the program or digital signal via the network or the like. [Industrial applicability]

[0151] This disclosure can be used in classification model generation systems, classification model generation methods, and programs, and in particular in classification model generation systems, classification model generation methods, and programs used in factories for image appearance inspection, image monitoring, and the like. [Explanation of Symbols]

[0152] 1. Classification Model Generation System 10 Image generation device 11 Acquisition Department 12 Defective area image generation unit 13 Synthesis Processing Unit 20 Model Generators 21 Memory section 22 Extraction part 23 Classification Model Generation Unit 24 Learning DB 111 Input Section 112 Good Image Generation Unit 121 Pre-processing 122 GAN Model 123 Generation part 131, 1212 Geometric transformation section 132 Synthesis section 133 Image sorting section 231, 1213 Learning Department 232 Classification Models 233 Evaluation Department 234 Judgment section 1211 Image Generation Unit

Claims

1. An acquisition unit that acquires a group of good product images, which includes multiple good product images, A geometric transformation unit that obtains a set of seed images by geometrically transforming seed images that are artificially generated and mimic defective parts, A defective area image generation unit generates multiple defective area images based on the aforementioned seed image group, A synthesis processing unit that generates a group of defective images by combining each of the multiple defective part images with the group of good product images, The system includes a classification model generation unit that generates a classification model by classifying and learning using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups. A classification model generation system.

2. An acquisition unit that acquires a group of good product images including multiple good product images, A defective area image generation unit generates multiple defective area images based on a group of seed images obtained by geometrically transforming seed images that mimic defective areas, which are seed images generated by artificial depiction. A synthesis processing unit that generates a group of defective images by combining each of the multiple defective part images with the group of good product images, The system includes a classification model generation unit that generates a classification model by classifying and learning using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups. The classification model generation unit, A learning unit that classifies and trains the aforementioned set of training images into a model, An evaluation unit that causes the model to evaluate at least a portion of the good image group and the defective image group, excluding the training image group used for classification learning of the model, The system includes a determination unit that determines whether the evaluation result evaluated by the aforementioned model is incorrect or not, and updates the learning image group by adding only the good product images or defective images determined to be incorrect to the learning image group. The learning unit generates the classification model by classifying the updated set of training images into the model. A classification model generation system.

3. An acquisition unit that acquires a group of good product images including multiple good product images, A defective area image generation unit generates multiple defective area images based on a group of seed images obtained by geometrically transforming seed images that mimic defective areas, which are seed images generated by artificial depiction. A synthesis processing unit that generates a group of defective images by combining each of the multiple defective part images with the group of good product images, The system includes a classification model generation unit that generates a classification model by classifying and learning using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups. The aforementioned synthesis processing unit, A synthesis unit that combines each of the multiple defective part images with the group of good product images, The system includes an image selection unit that determines whether the image synthesized by the synthesis unit is suitable as a defective image, The aforementioned image sorting unit is: If the area and brightness of the defective parts shown in the synthesized image meet predetermined standards, it is determined that the synthesized image is valid as the defective image. If the area and brightness of the defective parts shown in the synthesized image do not meet the predetermined standards, the synthesized image is determined to be unsuitable as the defective image. Only the images deemed valid from the synthesized images are included in the group of defective images. A classification model generation system.

4. An acquisition unit that acquires a group of good product images including multiple good product images, A defective area image generation unit generates multiple defective area images based on a group of seed images obtained by geometrically transforming seed images that mimic defective areas, which are seed images generated by artificial depiction. A synthesis processing unit that generates a group of defective images by combining each of the multiple defective part images with the group of good product images, The system includes a classification model generation unit that generates a classification model by classifying and learning using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups. The defective area image generation unit is, The defective area image is generated using a pre-trained GAN (Generative Adversarial Network) model. The aforementioned GAN model is trained using the aforementioned set of seed images. A classification model generation system.

5. The acquisition unit is, The input unit into which the aforementioned multiple images of good products are input, The system includes a good product image generation unit that geometrically transforms the plurality of good product images input to the input unit to generate one or more new good product images, and acquires the plurality of good product images and the one or more new good product images as a group of good product images. A classification model generation system according to any one of claims 1 to 4.

6. The classification model generation unit, A learning unit that classifies and trains the aforementioned set of training images into a model, An evaluation unit that causes the model to evaluate at least a portion of the good image group and the defective image group, excluding the training image group used for classification learning of the model, The system includes a determination unit that determines whether the evaluation result evaluated by the aforementioned model is incorrect or not, and updates the learning image group by adding only the good product images or defective images determined to be incorrect to the learning image group. The learning unit generates the classification model by classifying the updated set of training images into the model. A classification model generation system according to claim 1, 3, or 4.

7. The evaluation results include a determination result indicating whether the image input to the model is a good image or a bad image, and the likelihood of the determination result. The classification model generation system according to claim 6.

8. The determination unit determines that the evaluation result is an incorrect answer if the determination result is incorrect or if the probability is less than or equal to a predetermined value. The classification model generation system according to claim 7.

9. The aforementioned synthesis processing unit, A synthesis unit that combines each of the multiple defective part images with the group of good product images, The system includes an image selection unit that determines whether the image synthesized by the synthesis unit is a suitable image for a defective image. A classification model generation system according to claim 1, 2, or 4.

10. The aforementioned image sorting unit is: If the area and brightness of the defective parts shown in the synthesized image meet predetermined standards, it is determined that the synthesized image is valid as the defective image. If the area and brightness of the defective parts shown in the synthesized image do not meet the predetermined standards, the synthesized image is determined to be unsuitable as the defective image. Only the images deemed valid from the synthesized images are included in the group of defective images. The classification model generation system according to claim 9.

11. The defective area image generation unit is, The defective area image is generated using a pre-trained GAN (Generative Adversarial Network) model. The aforementioned GAN model is trained using the aforementioned set of seed images. A classification model generation system according to any one of claims 1 to 3.

12. The aforementioned seed image consists of a defective area that mimics a faulty part and a background, and the defective area and the background can be separated by a single threshold. A classification model generation system according to any one of claims 1 to 4.

13. The aforementioned background is painted in a single color. The classification model generation system according to claim 12.

14. A step to acquire a group of good product images that include multiple good product images, A geometric transformation step in which a seed image, which is artificially generated and mimics a defective part, is geometrically transformed to obtain a set of seed images, A defective area image generation step that generates multiple defective area images based on the aforementioned set of seed images, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, The process includes a classification model generation step, which generates a classification model by using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups to perform classification learning. A method for generating classification models.

15. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, The process includes a classification model generation step, which generates a classification model by using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups to perform classification learning. The aforementioned classification model generation step is: A learning step in which the aforementioned set of training images is used to classify and train a model, An evaluation step in which the model is made to evaluate at least a portion of the good image group and the defective image group, excluding the training image group used for classification learning of the model, The process includes a determination step of determining whether the evaluation result evaluated by the aforementioned model is incorrect or not, and updating the training image group by adding only the good product images or defective images determined to be incorrect to the training image group. In the learning step, the updated set of training images is used to classify the model, thereby generating the classification model. A method for generating classification models.

16. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, The process includes a classification model generation step, which generates a classification model by using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups to perform classification learning. The aforementioned synthesis process step is A synthesis step in which each of the multiple defective area images is combined with the group of good product images, The image selection step includes determining whether the image synthesized by the synthesis step is suitable as a defective image, In the aforementioned image selection step, If the area and brightness of the defective parts shown in the synthesized image meet predetermined standards, it is determined that the synthesized image is valid as the defective image. If the area and brightness of the defective parts shown in the synthesized image do not meet the predetermined standards, the synthesized image is determined to be unsuitable as the defective image. Only the images deemed valid from the synthesized images are included in the group of defective images. A method for generating classification models.

17. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, The process includes a classification model generation step, which generates a classification model by using a portion of the above-mentioned good product image group and the above-mentioned defective product image group as training image groups to perform classification learning. In the aforementioned defective area image generation step, The defective area image is generated using a pre-trained GAN (Generative Adversarial Network) model. The aforementioned GAN model is trained using the aforementioned set of seed images. A method for generating classification models.

18. A step to acquire a group of good product images that include multiple good product images, A geometric transformation step in which a seed image, which is artificially generated and mimics a defective part, is geometrically transformed to obtain a set of seed images, A defective area image generation step that generates multiple defective area images based on the aforementioned set of seed images, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, A classification model generation step is performed by generating a classification model by using the aforementioned good product image group and a portion of the aforementioned defective product image group as training image groups to perform classification learning. A program that is executed by a computer.

19. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, A classification model generation step is performed by generating a classification model by using the aforementioned good product image group and a portion of the aforementioned defective product image group as training image groups to perform classification learning. Let the computer run it, The aforementioned classification model generation step is: A learning step in which the aforementioned set of training images is used to classify and train a model, An evaluation step in which the model is made to evaluate at least a portion of the good image group and the defective image group, excluding the training image group used for classification learning of the model, The process includes a determination step of determining whether the evaluation result evaluated by the aforementioned model is incorrect or not, and updating the training image group by adding only the good product images or defective images determined to be incorrect to the training image group. In the learning step, the updated set of training images is used to classify the model, thereby generating the classification model. program.

20. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, A classification model generation step is performed by generating a classification model by using the aforementioned good product image group and a portion of the aforementioned defective product image group as training image groups to perform classification learning. Let the computer run it, The aforementioned synthesis process step is A synthesis step in which each of the multiple defective area images is combined with the group of good product images, The image selection step includes determining whether the image synthesized by the synthesis step is suitable as a defective image, In the aforementioned image selection step, If the area and brightness of the defective parts shown in the synthesized image meet predetermined standards, it is determined that the synthesized image is valid as the defective image. If the area and brightness of the defective parts shown in the synthesized image do not meet the predetermined standards, the synthesized image is determined to be unsuitable as the defective image. Only the images deemed valid from the synthesized images are included in the group of defective images. program.

21. An acquisition step of acquiring a group of good product images including multiple good product images, A seed image generation step in which multiple images of defective parts are generated based on a seed image set obtained by geometrically transforming a seed image that mimics a defective part, which is an artificially drawn seed image, A synthesis process step to generate a group of defective images by combining each of the multiple defective area images with the group of good product images, A classification model generation step is performed by generating a classification model by using the aforementioned good product image group and a portion of the aforementioned defective product image group as training image groups to perform classification learning. Let the computer run it, In the aforementioned defective area image generation step, The defective area image is generated using a pre-trained GAN (Generative Adversarial Network) model. The aforementioned GAN model is trained using the aforementioned set of seed images. program.

Images