Classification model generation system, classification model generation method, and program

Abstract

The present invention is provided with: an acquisition unit (11) that acquires a group of qualified product images including a plurality of qualified product images; a disqualified part image generation unit (12) that generates a plurality of disqualified part images on the basis of a seed image group obtained by geometrically transforming a seed image that is generated by manual drawing and that imitates a disqualified part; a combination processing unit (13) that generates a group of defective images by combining each of the plurality of defective site images with the group of defective images; and a classification model generation unit (23) that generates a classification model by performing classification learning using a part of the qualified product image group and the unqualified product image group as a learning image group.

Description

Technical Field

[0001] The present disclosure relates to a classification model generation system, a classification model generation method, and a program. Background Art

[0002] In the case of performing machine learning such as deep learning, it is known that a large number of images are required as learning data in order to ensure the accuracy of the model after machine learning.

[0003] However, for example, in the case of performing machine learning on a model for inspection use in a manufacturing process, although a large number of qualified product images can be easily obtained, it is often difficult to obtain a large number of unqualified images. This is because the unqualified images obtained in the inspection process are often small relative to the denominator, that is, the total number of images when the inspection process is performed. In addition, even assuming that a large number of unqualified images are obtained, generally the unqualified patterns are biased. Therefore, there is a problem that the variability of the unqualified images used as learning data cannot be ensured, and it is difficult to ensure practical inspection accuracy even when the obtained unqualified images and qualified product images are used for machine learning of the model.

[0004] Moreover, even when a large number of unqualified images are obtained, the selection work requires a large amount of man-hours, and there are also maintenance management problems as follows: when rebuilding the model due to changes or additions of unqualified patterns, etc., the selection work needs to be performed again.

[0005] In response to this, for example, Patent Document 1 discloses a defective product image generation method that can increase the number of unqualified images. According to the method disclosed in Patent Document 1, it is possible to generate more realistic defective product images based on qualified product images and partial images including defective parts.

[0006] Prior Art Documents

[0007] Patent Documents

[0008] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2021-135903

[0009] Non-Patent Documents

[0010] Non-Patent Document 1: Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, Hayit Greenspan: GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification. Neurocomputing Volume 321, 10 December 2018, Pages 321-331 Summary of the Invention

[0011] Problems to be Solved by the Invention

[0012] However, in the technology disclosed in Patent Document 1, a database of non-conforming parts needs to be created. On this basis, in order to increase the number of non-conforming images, non-conforming images including non-conforming parts need to be collected and selected.

[0013] The present disclosure has been completed in view of the above circumstances, and an object thereof is to provide a classification model generation system and the like that can generate a classification model capable of classifying conforming images and non-conforming images without collecting and selecting non-conforming images.

[0014] Solutions to the Problems

[0015] In order to solve the above problems, a classification model generation system according to an aspect of the present disclosure includes: an acquisition unit that acquires a group of conforming images including a plurality of conforming images; a non-conforming part image generation unit that generates a plurality of non-conforming part images based on a group of seed images, where the group of seed images is obtained by geometric transformation of seed images that are generated by manual drawing and imitate non-conforming parts; a synthesis processing unit that generates a group of non-conforming images by synthesizing the plurality of non-conforming part images with the group of conforming images respectively; and a classification model generation unit that uses a part of the group of conforming images and the group of non-conforming images as a learning image group to perform classification learning, thereby generating a classification model.

[0016] In addition, these overall or specific aspects can be implemented by a device, a method, a system, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM, or can also be implemented by any combination of a device, a method, a system, an integrated circuit, a computer program, and a recording medium.

[0017] Effects of the Invention

[0018] A classification model generation system according to the present disclosure, etc., can generate a classification model capable of classifying qualified product images and unqualified product images without collecting and selecting unqualified images. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a diagram showing an example of the structure of the classification model generation system according to the embodiment.

[0020] Figure 2A It is a block diagram showing an example of the detailed structure of the image generation device according to the embodiment.

[0021] Figure 2B It shows Figure 2A It is a block diagram showing an example of the detailed structure of the preprocessing unit shown.

[0022] Figure 3A It is a diagram conceptually showing an example of the seed image according to the embodiment.

[0023] Figure 3B It is a diagram conceptually showing an example of a group of seed images obtained by geometric transformation of the seed image according to the embodiment.

[0024] Figure 4 It is a diagram conceptually showing an example of a plurality of unqualified part images according to the embodiment.

[0025] Figure 5A It is a diagram conceptually showing an example of the unqualified part image according to the embodiment.

[0026] Figure 5B It conceptually shows an image obtained by Figure 5A adding an alpha channel to the background of the unqualified part image shown.

[0027] Figure 5C It conceptually shows an image obtained by Figure 5B performing a geometric transformation such as rotation on the image with the background to which the alpha channel has been added shown.

[0028] Figure 5D It is a diagram conceptually showing an example of the qualified product image according to the embodiment.

[0029] Figure 5E It is a diagram conceptually showing an example of the unqualified image candidate according to the embodiment.

[0030] Figure 6A It is a diagram conceptually showing an example of the mask image according to the embodiment.

[0031] Figure 6B It is a diagram conceptually showing an example of a masked qualified product image related to an embodiment.

[0032] Figure 6C It is a diagram conceptually showing an example of a masked unqualified image candidate related to an embodiment.

[0033] Figure 6D It is a diagram conceptually showing an example of a difference image related to an embodiment.

[0034] Figure 7 It is a block diagram showing an example of the detailed structure of a model generation device related to an embodiment.

[0035] Figure 8 It is a flowchart showing the processing of a classification model generation method of a classification model generation system related to an embodiment.

[0036] Figure 9 It is a diagram showing the flow of the processing of the classification model generation method related to Example 1. Detailed implementation manners

[0037] Each of the implementation manners described below represents a specific example of the present disclosure. The numerical values, shapes, constituent elements, steps, order of steps, etc. shown in the following implementation manners are examples and are not intended to limit the present disclosure. In addition, among the constituent elements in the following implementation manners, the constituent elements not described in the independent claims representing the most general concept are described as optional constituent elements. In addition, in all the implementation manners, each content can also be combined.

[0038] (Implementation manner)

[0039] Hereinafter, a classification model generation method and the like of the classification model generation system 1 in the implementation manner will be described with reference to the drawings.

[0040] [1 Classification model generation system 1]

[0041] Figure 1 It is a diagram showing an example of the structure of the classification model generation system 1 related to an embodiment.

[0042] The classification model generation system 1 according to this embodiment is a system for generating a classification model capable of classifying qualified product images and unqualified product images without collecting and selecting unqualified images. An unqualified image is an image in which defects such as damage or foreign objects are captured. For example, an unqualified image in an inspection process is an image indicating that the manufactured product captured in the image contains defects and the manufactured product is unqualified. A qualified product image is an image in which no defects are captured. For example, a qualified product image in an inspection process is an image indicating that the manufactured product captured in the image does not contain defects and the manufactured product has passed the inspection and is a qualified product.

[0043] As Figure 1 shown, the classification model generation system 1 includes an image generation device 10 and a model generation device 20. They can be connected through a communication network or through physical communication lines such as a bus. In addition, in this embodiment, it is described that the image generation device 10 and the model generation device 20 have different hardware structures, but this is not limited thereto. The image generation device 10 and the model generation device 20 can also be constructed on one piece of hardware to form the classification model generation system 1.

[0044] The image generation device 10 generates a group of unqualified images based on an unqualified part image and a group of qualified product images. The unqualified part image is generated based on a seed image manually drawn by imitating unqualified parts such as damage or foreign objects, and the group of qualified product images is obtained by augmenting the acquired qualified product images. Details will be described later.

[0045] The model generation device 20 uses the group of unqualified images generated by the image generation device 10 and the group of qualified product images obtained by augmenting the qualified product images by the image generation device 10 to generate a classification model of the unqualified teaching type, that is, a classification model capable of classifying qualified product images and unqualified images. Details will be described later.

[0046] [2 Image Generation Device 10]

[0047] Figure 2A is a block diagram showing an example of the detailed structure of the image generation device 10 according to the embodiment.

[0048] The image generation device 10 is implemented by a computer having, for example, a processor (microprocessor), a memory, a communication interface, etc. The image generation device 10 realizes various functions by the processor executing a control program stored in the memory. In addition, the image generation device 10 can also be configured to operate in a manner that part of its structure is included in a cloud server.

[0049] In this embodiment, as Figure 1As shown in the figure, the image generation device 10 includes an acquisition unit 11, a non-conforming part image generation unit 12, and a synthesis processing unit 13. Hereinafter, each component will be described.

[0050] [2.1 Acquisition Unit 11]

[0051] As Figure 2A shown, the acquisition unit 11 includes an input unit 111 and a qualified product image generation unit 112, and the acquisition unit 11 acquires a group of qualified product images including a plurality of qualified product images.

[0052] A plurality of qualified product images are input to the input unit 111. The input unit 111 is implemented by, for example, a communication interface or an input interface, etc., and can acquire a plurality of qualified product images by inputting a plurality of pre-prepared qualified product images. The plurality of pre-prepared qualified product images can be obtained, for example, through manufacturing processes such as inspection processes. In addition, the number of the plurality of pre-prepared qualified product images is, for example, about one thousand, but can also be about several tens.

[0053] The qualified product image generation unit 112 performs geometric transformation on the plurality of qualified product images input to the input unit 111 to generate one or more new qualified product images. Then, the qualified product image generation unit 112 acquires the plurality of qualified product images and the one or more generated new qualified product images as a group of qualified product images. Here, geometric transformation refers to, for example, zooming (size change), rotation, translation, X-axis mirroring (X-axis symmetric movement), Y-axis mirroring (Y-axis symmetric movement), exchange of x coordinate and y coordinate (mirroring), shearing, cropping, contrast change processing, brightness change processing, filter processing, blur processing, etc., but is not limited to these. Any geometric transformation that does not destroy the shape features is acceptable. In this way, the qualified product image generation unit 112 augments (increases) the plurality of qualified product images input to the input unit 111 to obtain a group of qualified product images. In addition, the number of the group of qualified product images is, for example, about ten thousand, but can also be several thousand, or can also be several tens of thousands.

[0054] [2.2 Non-conforming Part Image Generation Unit 12]

[0055] As Figure 2A shown, the non-conforming part image generation unit 12 includes a pre-processing unit 121, a GAN model 122, and a generation unit 123. The non-conforming part image generation unit 12 generates a plurality of non-conforming part images based on a group of seed images, where the group of seed images is obtained by performing geometric transformation on artificially generated seed images that imitate non-conforming parts. Here, the non-conforming part image is, for example, an image representing defects such as damage or foreign objects that become non-conforming parts in manufactured products, etc.

[0056] [2.2.1 GAN 122]

[0057] The GAN model 122 is a generative model constructed by an architecture called GAN (Generative Adversarial Networks), and it uses a group of seed images for learning. In this embodiment, the GAN 122 is constituted by, for example, DCGAN (Deep Convolutional GAN), WGAN (Wasserstein GAN), etc., but is not limited to these. The GAN 122 can also be constituted by SNGAN (Spectral Normalization for GAN) or LSGAN (Least Squares GAN), etc., as long as it is constituted by a GAN having the feature of increasing similar variants with reference to the learned images.

[0058] [2.2.2 Preprocessing Unit 121]

[0059] Figure 2B is a block diagram showing Figure 2A an example of the detailed structure of the preprocessing unit 121 shown.

[0060] As Figure 2B shown, the preprocessing unit 121 includes a seed image generation unit 1211, a geometric transformation unit 1212, and a learning unit 1213. The preprocessing unit 121 performs preprocessing for generating a seed image and causing the GAN model 122 to learn, etc., for generating an image of a defective part. The preprocessing unit 121, for example, includes a computer including a memory and a processor (microprocessor), and the functions of each unit are realized by the processor executing a prescribed program stored in the memory.

[0061] The seed image generation unit 1211 generates a seed image imitating the defective part by manual drawing. Here, the seed image generation unit 1211 generates a seed image in such a way that it is composed of a background and a defective part imitating the defective part, and the defective part and the background can be separated by a single threshold. The background is preferably painted in a single color.

[0062] In this embodiment, the seed image generation unit 1211, for example, executes an application such as a drawing software, and according to the operation of the user, manually draws a figure imitating the defective part as the defective part, thereby generating an artificial image as a seed of the defective part, that is, a seed image.

[0063] Figure 3A is a diagram conceptually showing an example of the seed image according to the embodiment.

[0064] In Figure 3AIn the example shown, a quadrilateral frame is manually drawn as a defective part. In addition, the defective part of the seed image is drawn in white and the background is drawn in black. Furthermore, it is desirable that the defective part is drawn in a color close to the defective part to be recognized, and the background is drawn in a single color that does not mix with the defective part. In this way, by drawing the background painted in a single color, the defective part of the seed image can be separated from the background with a single threshold value.

[0065] The geometric transformation unit 1212 performs geometric transformation on the seed image generated by the seed image generation unit 1211 to obtain a seed image group. The geometric transformation unit 1212 includes a processor and a memory (not shown), and implements the geometric transformation process by the processor executing a predetermined program stored in the memory.

[0066] In the present embodiment, the geometric transformation unit 1212 increases (augments) the seed images that can be defects by performing geometric transformation processes such as size change, contrast change, rotation, stretching, etc. on one seed image generated by the seed image generation unit 1211. As a result, the geometric transformation unit 1212 can obtain a seed image group composed of the original one seed image and a plurality of seed images obtained by the increase.

[0067] Figure 3B It is a diagram conceptually showing an example of a seed image group obtained by performing geometric transformation on a seed image according to an embodiment.

[0068] In Figure 3B the example shown, a part of a seed image group obtained by performing geometric transformation processes such as size change, rotation, stretching, etc. on one seed image shown in Figure 3A is shown.

[0069] The learning unit 1213 uses the seed image group obtained by performing geometric transformation on the seed image to make the GAN model 122 learn the features of the seed image group. The learning unit 1213 includes a processor and a memory (not shown), and implements the learning process of making the GAN model 122 learn using the seed image group by the processor executing a predetermined program stored in the memory. As a result, the learning unit 1213 can use the seed image group to make the GAN model 122 learn the features of defective variants.

[0070] [2.2.3 Generation unit 123]

[0071] The generation unit 123 uses the learned GAN model 122 to generate a plurality of nonconforming part images. The generation unit 123 includes a processor and a memory (not shown), and implements the generation process of making the learned GAN model 122 generate nonconforming part images by the processor executing a predetermined program stored in the memory.

[0072] In the present embodiment, the generation unit 123 can generate an unqualified part image by applying noise to the learned GAN model 122. Therefore, the generation unit 123 can generate a plurality of unqualified part images by applying noise that varies in value to the learned GAN model 122.

[0073] Figure 4 FIG. is a diagram conceptually showing an example of a plurality of unqualified part images according to the embodiment.

[0074] In Figure 4 the example shown, a part of the plurality of unqualified part images generated by the learned GAN model 122 by applying varying noise is shown. In Figure 4 , the white figures respectively show the unqualified parts, i.e., the defects. As Figure 4 shown, by using the learned GAN model 122, it is possible to generate not only defects similar to the seed image group but also defects similar to but different from the defects.

[0075] In this way, the generation unit 123 can cause the learned GAN model 122 to generate not only unqualified part images having defects similar to the seed image group whose features have been learned, but also unqualified part images having defects similar to but different from the defects.

[0076] [2.3 Composite processing unit 13]

[0077] As Figure 2A shown, the composite processing unit 13 includes a geometric transformation unit 131, a composite unit 132, and an image selection unit 133. The composite processing unit 13 generates a group of unqualified images by respectively combining a plurality of unqualified part images with the group of qualified product images.

[0078] [2.3.1 Geometric transformation unit 131]

[0079] The geometric transformation unit 131 performs geometric transformation on the plurality of unqualified part images generated by the unqualified part image generation unit 12. The geometric transformation unit 131 includes a processor and a memory (not shown), and implements each process including geometric transformation processing by the processor executing a prescribed program stored in the memory.

[0080] In the present embodiment, the geometric transformation unit 131 binarizes each of the plurality of unqualified part images generated by the unqualified part image generation unit 12 to separate it into a background and a defect part, and assigns an alpha channel as a transparent layer to the background. In addition, the geometric transformation unit 131 performs geometric transformation on the image having the background with the alpha channel added.

[0081] Figure 5A This is a diagram conceptually showing an example of an image of a nonconforming part related to an embodiment. Figure 5B This is conceptually showing Figure 5A a diagram of an example of an image obtained by adding an alpha channel to the background of the nonconforming part image shown. Figure 5C This is conceptually showing Figure 5B a diagram of an example of an image obtained by performing a geometric transformation such as rotation on the image with the background to which an alpha channel has been added as shown.

[0082] That is, the geometric transformation unit 131 binarizes, for example, the nonconforming part image generated by the nonconforming part image generation unit 12 as shown in Figure 5A to separate it into the background and the defect part, and adds an alpha channel to the separated background, thereby obtaining an image (image of the defect) such as that shown in Figure 5B . The geometric transformation unit 131 performs a rotational geometric transformation on the image with the background to which an alpha channel has been added as shown in Figure 5B to obtain an image such as that shown in Figure 5C .

[0083] [2.3.2 Composition Unit 132]

[0084] The composition unit 132 composes each of a plurality of nonconforming part images with a group of conforming product images. The composition unit 132 includes a processor and a memory (not shown), and realizes the composition process by the processor executing a prescribed program stored in the memory.

[0085] In the present embodiment, the composition unit 132 can compose an image with a background to which an alpha channel has been added and that has been subjected to geometric transformation processing such as by the geometric transformation unit 131 with a conforming product image, thereby obtaining a nonconforming image candidate as a candidate for the nonconforming image. Therefore, the composition unit 132 can obtain a group of nonconforming image candidates by composing each of a plurality of nonconforming part images that have been subjected to geometric transformation processing such as by the geometric transformation unit 131 with a group of conforming product images. Further, in the case where the group of nonconforming image candidates is not selected by the image selection unit 133 described later, the group of nonconforming image candidates all become a group of nonconforming images, and in the case where the group of nonconforming image candidates is selected by the image selection unit 133 described later, only the selected nonconforming image candidates in the group of nonconforming image candidates become a group of nonconforming images.

[0086] Figure 5D This is a diagram conceptually showing an example of a conforming product image 50 related to an embodiment. Figure 5E This is a diagram conceptually showing an example of a nonconforming image candidate 51 related to an embodiment.

[0087] That is, the composition unit 132 can, for example, combine, for example, an image obtained by subjecting the image shown in Figure 5B to geometric transformation and other processes by the geometric transformation unit 131 with Figure 5C the image shown in Figure 5D the qualified product image shown in Figure 5E to obtain an unqualified image candidate 51 as shown in Figure 5C The image shown in Figure 5B is an example, and various geometric transformations can be performed on the image shown in

[0088] [2.3.3 Image selection unit 133]

[0089] The image selection unit 133 determines whether the image (unqualified image candidate) synthesized by the composition unit 132 is appropriate as an unqualified image. The image selection unit 133 includes a processor and a memory (not shown), and can implement processes such as determination processing by executing a prescribed program stored in the memory by the processor.

[0090] More specifically, when the area and brightness of the unqualified part reflected in the synthesized image satisfy a prescribed standard, the image selection unit 133 determines that the synthesized image is appropriate as an unqualified image. When the area and brightness of the unqualified part reflected in the synthesized image do not satisfy the prescribed standard, the image selection unit 133 determines that the synthesized image is inappropriate as an unqualified image. In addition, the image selection unit 133 can also use, for example, a difference image obtained by taking the difference between the qualified product image 50 shown in Figure 5D and the unqualified image candidate 51 shown in Figure 5E to determine whether the area and brightness of the defect part (unqualified part) included in the difference image satisfy the prescribed standard. Then, the image selection unit 133 includes only the images determined to be appropriate in the synthesized image in the unqualified image group.

[0091] In addition, when other areas such as areas other than the manufactured product area are captured in the qualified product image, the unqualified image candidate obtained by combining the unqualified part (defect part) with the other area is not effective for the learning of the classification model. That is, there may be areas in the qualified product image where it is not desired to combine the unqualified part. In this case, it is only necessary to prepare a mask image in which the areas where it is not desired to combine the unqualified part are filled with black, and determine whether the image (unqualified image candidate) synthesized by the composition unit 132 is appropriate as an unqualified image.

[0092] More specifically, first, the image selection unit 133 acquires a masked defective image candidate obtained by the composition unit 132 composing the prepared mask image and the defective image candidate, and a masked non-defective image obtained by composing the mask image and the non-defective image that is the basis of the defective image candidate. Next, the image selection unit 133 generates a difference image obtained by taking the difference between them (the masked defective image candidate and the masked non-defective image). Then, the image selection unit 133 may measure the brightness and area of the defective part included in the generated difference image and determine whether a specified standard is satisfied. As a result, the defective image candidate obtained by composing the defective part in the area where the defective part is not desired to be composed in the non-defective image is excluded, and thus is excluded from the learning image group for learning the classification model described later.

[0093] Figure 6A FIG. is a diagram conceptually showing an example of the mask image 52 according to the embodiment. Figure 6B FIG. is a diagram conceptually showing an example of the masked non-defective image 50a according to the embodiment. That is, in Figure 6B FIG. shows an example of the masked non-defective image 50a obtained by the image selection unit 133 causing the composition unit 132 to compose the Figure 6A mask image 52 shown in Figure 5D and the non-defective image 50 shown in

[0094] Figure 6C FIG. is a diagram conceptually showing an example of the masked defective image candidate 51a according to the embodiment. That is, in Figure 6C FIG. shows an example of the masked defective image candidate 51a obtained by the image selection unit 133 causing the composition unit 132 to compose the Figure 6A mask image 52 shown in Figure 5E and the defective image candidate 51 shown in

[0095] In addition, Figure 6D FIG. is a diagram conceptually showing an example of the difference image 53 according to the embodiment. That is, in Figure 6D FIG. shows an example of the difference image 53 generated by the image selection unit 133 taking the difference between the Figure 6B masked non-defective image 50a shown in Figure 6C and the masked defective image candidate 51a shown in

[0096] In this way, the image selection unit 133 uses the Figure 6A mask image shown in Figure 6D to generate, for example, the Figure 5EWhen the defective image candidate 51 shown is a defective image candidate obtained by synthesizing a defective part in a region where a defective part is undesirably synthesized in a non-defective product image, the image selection unit 133 can Figure 5E exclude the defective image candidate 51 shown from the group of defective images.

[0097] [3 Model Generation Device 20]

[0098] Figure 7 is a block diagram showing an example of the detailed structure of the model generation device 20 according to the embodiment.

[0099] The model generation device 20 performs classification learning based on the group of non-defective product images and the group of defective images generated by the image generation device 10, and thereby generates a classification model.

[0100] The model generation device 20 is implemented by a computer including, for example, a processor (microprocessor), a memory, a communication interface, etc., and various functions are implemented by the processor executing a control program stored in the memory. In addition, the model generation device 20 may be configured to operate in a manner that a part of its structure is included in a cloud server.

[0101] In the present embodiment, as Figure 1 shown, the model generation device 20 includes a storage unit 21, an extraction unit 22, a classification model generation unit 23, and a learning DB 24. Below, each component will be described.

[0102] [3.1 Storage Unit 21]

[0103] The storage unit 21 is implemented by, for example, an HDD (Hard Disk Drive) or a flash memory.

[0104] In the present embodiment, the storage unit 21 stores the group of non-defective product images and the group of defective images generated by the image generation device 10 as a group of generated images. The number of images included in the group of generated images is, for example, on the order of several hundred thousand, but is not limited thereto. The number of the group of defective images included in the group of generated images may also be on the order of several tens to several hundreds. In this case, by the model generation device 20 requesting the image generation device 10 to add a group of defective images, etc., the group of defective images generated by the image generation device 10 is added to the group of generated images.

[0105] [3.2 Extraction Unit 22]

[0106] The extraction unit 22 extracts a part from the group of generated images stored in the storage unit 21 and outputs it to the classification model generation unit 23 or stores it in the learning DB 24.

[0107] In the present embodiment, the extraction unit 22 extracts a part of the generated image group stored in the storage unit 21, that is, a group of qualified product images and a part of the group of unqualified images, as initial learning images and outputs them to the learning DB 24 and the learning unit 231. The initial learning images extracted by the extraction unit 22 can be a small number. For example, it is sufficient to include 5 to 10 qualified product images and unqualified images respectively, but it is not limited thereto. The number of qualified product images and unqualified images included in the initial learning images can be selected appropriately. However, if the number is too small, the classification model cannot be established, and if the number is too large, the learning cost becomes high. Therefore, it is desirable that each is about 5 to 100.

[0108] In addition, the extraction unit 22 may extract the generated image group other than the initial learning images from the generated image group stored in the storage unit 21 as evaluation images and output them to the evaluation unit 233.

[0109] [3.3 Classification model generation unit 23]

[0110] As Figure 7 shown, the classification model generation unit 23 includes a learning unit 231, a classification model 232, an evaluation unit 233, and a determination unit 234. The classification model generation unit 23 uses a part of the group of qualified product images and the group of unqualified images as a learning image group to perform classification learning, thereby generating the classification model 232.

[0111] Next, each component will be described.

[0112] [3.3.1 Classification model 232]

[0113] The classification model 232 is an identification model composed of the following model: This model is composed of an architecture having one or more convolutional layers. In the present embodiment, the classification model 232 is composed of, for example, a CNN (Convolutional Neural Network), but is not limited to these. The classification model 232 is a model having one or more convolutional layers, and is a model that uses the learning image group for classification learning to be able to classify qualified product images and unqualified images.

[0114] [3.3.2 Learning unit 231]

[0115] The learning unit 231 generates a classification model by causing the classification model 232 to perform classification learning on the learning image group. The learning unit 231 includes a processor and a memory (not shown), and realizes the learning process of causing the classification model 232 to learn by the processor executing a predetermined program stored in the memory.

[0116] More specifically, first, the learning unit 231 uses the initial learning images that are part of the qualified product image group and the unqualified image group as the learning image group to perform classification learning of the classification model 232. In addition, the learning unit 231 uses the learning image group updated by the determination unit 234 to perform classification learning of the classification model 232. In this way, the learning unit 231 generates the learned classification model 232 by repeatedly performing classification learning.

[0117] In this embodiment, the learning unit 231 first uses a small number of initial learning images as the learning image group to make the classification model 232 learn. Next, the learning unit 231 uses the learning image group updated by adding the wrong answer image group composed of one or more wrong answer images described later to the initial learning images to make the classification model 232 learn. Moreover, the learning unit 231 uses the repeatedly updated learning image group to make the classification model 232 repeatedly learn, thereby generating the learned classification model 232.

[0118] In addition, when the learning unit 231 makes the classification model 232 learn using the updated learning image group, the learning unit 231 may make the classification model 232 relearn. Alternatively, the value of the parameter of the classification model 232 obtained in the previous learning may be used as the initial value, and the layers constituting the classification model 232 obtained in the previous learning may be frozen. On this basis, the classification model 232 obtained in the previous learning may be made to perform transfer learning using the updated learning image group.

[0119] [3.3.3 Evaluation Unit 233]

[0120] The evaluation unit 233 makes the classification model 232 evaluate at least a part of the qualified product image group and the unqualified image group other than the learning image group used in the classification learning of the classification model 232. The evaluation result includes the determination result indicating whether the image input to the classification model 232 is a qualified product image or an unqualified image, but is not limited thereto. The evaluation result may also include the determination result indicating whether the image input to the classification model 232 is a qualified product image or an unqualified image, and the accuracy of the determination result. The accuracy of the determination result can be easily obtained from the layer before the output layer of the classification model 232 that outputs a value indicating whether the image is a qualified product image or an unqualified image. In addition, the evaluation unit 233 includes a processor and a memory (not shown), and implements the evaluation process by the processor executing a predetermined program stored in the memory.

[0121] In this embodiment, the evaluation unit 233 causes the classification model 232 to evaluate the evaluation image obtained from the extraction unit 22. As described above, the evaluation image is a part of the generated image group. In other words, the evaluation image is at least a part of the qualified product image group and the unqualified image group other than the learning image group used in the classification learning of the classification model 232 in the qualified product image group and the unqualified image group.

[0122] [3.3.4 Determination Unit 234]

[0123] The determination unit 234 determines whether the evaluation result obtained by evaluating with the classification model 232 is a wrong answer. Here, the determination unit 234 determines that the evaluation result is a wrong answer when the determination result is incorrect or the accuracy of the determination result is below a specified value. In addition, the determination unit 234 updates the learning image group by adding only the qualified product image or unqualified image determined to be a wrong answer to the learning image group.

[0124] In addition, the determination unit 234 can, for example, specify a learning end condition including the number of learning times and the number of evaluation images, and cause the learning unit 231 to perform classification learning of the classification model 232 using the updated learning image group. In this case, the determination unit 234 ends the classification learning of the learning unit 231 at the time point when the learning end condition is reached.

[0125] In addition, the determination unit 234 includes a processor and a memory (not shown), and implements these various processes by the processor executing a specified program stored in the memory.

[0126] In this embodiment, the unqualified image group included in the evaluation image evaluated by the classification model 232 is generated by the image generation device 10. Therefore, the evaluation image evaluated by the classification model 232 is in a state where it is known whether it is a qualified product image or an unqualified image. Thus, the determination unit 234 can determine that the evaluation result is a wrong answer when the evaluation result obtained by evaluating with the classification model 232 is incorrect or the accuracy of the determination result is below a specified value. Then, the determination unit 234 can add only the qualified product image or unqualified image determined to be a wrong answer, that is, the wrong answer image, to the learning image group, thereby updating the learning image group. That is, the determination unit 234 can add only the wrong answer images to the learning image group without performing a selection operation, and can increase the number (data) of the learning image group.

[0127] In addition, when there is even one mis-answer image in the evaluation image, the determination unit 234 can update the learning image group by adding only the mis-answer image to the learning image group, and use the updated learning image group to perform classification learning of the classification model 232. Additionally, when it is determined that a specified number of images in the evaluation image are mis-answer images, the determination unit 234 can update the learning image group by adding these mis-answer images to the learning image group, and use the updated learning image group to perform classification learning of the classification model 232. And, in this case, in order to achieve efficient learning, it is preferable to decrease the value determined in the specified number as classification learning is repeatedly performed. This is because, corresponding to the progress of the learning of the classification model 232, the cases where the evaluation result is determined to be a mis-answer decrease.

[0128] [3.4 Learning DB 24]

[0129] The learning DB 24 is implemented by, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0130] The learning DB 24 stores a learning image group including at least initial learning images. The learning image group is updated by the determination unit 234. The learning image group includes only initial learning images at the initial stage of the learning of the classification model 232, but is updated by adding one or more mis-answer images thereafter. When the learning of the classification model 232 progresses, the learning image group is updated to include initial learning images and a mis-answer image group.

[0131] [4 Operations of the Classification Model Generation System 1]

[0132] Next, an example of the operations of the classification model generation system 1 configured as described above will be described.

[0133] Figure 8 is a flowchart showing the processing of the classification model generation method of the classification model generation system 1 according to the embodiment.

[0134] The classification model generation system 1 includes a processor and a memory, and uses the processor and a program recorded in the memory to perform the following processing of step S1 to step S4.

[0135] More specifically, first, the classification model generation system 1 acquires a group of qualified product images including a plurality of qualified product images (S1).

[0136] Next, the classification model generation system 1 generates a plurality of unqualified part images based on a seed image group, where the seed image group is obtained by geometric transformation of seed images that are manually drawn and imitate unqualified parts (S2).

[0137] Next, the classification model generation system 1 generates a group of defective images by synthesizing a plurality of defective part images with a group of qualified product images respectively (S3).

[0138] Next, the classification model generation system 1 uses a part of the group of qualified product images and the group of defective images as a learning image group to perform classification learning, thereby generating a classification model (S4).

[0139] (Example 1)

[0140] Figure 9 It is a diagram showing the process of the classification model generation method according to Example 1.

[0141] In Figure 9 A specific manner of the process of the classification model generation method using the classification model generation system 1 is shown, showing that after the pre-preparation stage, an image generation stage is performed, and a learning and evaluation stage is performed. In addition, it is also shown that the image generation stage and the learning and evaluation stage can be cycled.

[0142] As Figure 9 shown, first, the pre-preparation stage is performed.

[0143] More specifically, the classification model generation system 1 first manually depicts a graph imitating the defective part as a defective part according to the user's operation, thereby generating an artificial image, that is, a seed image, which is a seed of the defect (S11). Next, geometric transformation processing is performed on the one seed image generated in step S11 (S12). Thereby, a group of seed images obtained by augmenting the seed image that can be a seed of the defect can be obtained. Next, the GAN model is learned using the group of seed images (S13). Thereby, the GAN model can learn the characteristics of the variants of the defective part (defect) according to the group of seed images.

[0144] After performing the pre-preparation stage in this way, the image generation stage is performed. In addition, a plurality of qualified product images are prepared in advance by collecting from actual manufacturing processes and the like.

[0145] More specifically, the classification model generation system 1 is first input with a plurality of pre-prepared qualified product images, and geometric transformation processing is performed on the input plurality of qualified product images (S14). Thereby, the classification model generation system 1 can obtain a group of qualified product images with rich variability obtained by augmenting the plurality of qualified product images.

[0146] In addition, the classification model generation system 1 performs the following generation process: by applying noise that causes value changes to the GAN model that has been learned in advance preparation, multiple defective part images are generated (S15). By using the GAN model in this way, the classification model generation system 1 can generate variant changes in shapes that are difficult to achieve only through geometric transformation, and thus can obtain diverse defective part images.

[0147] Next, the classification model generation system 1 performs geometric transformation processing (S16) on the multiple defective part images generated in step S15, and performs image synthesis processing (S17) of synthesizing with the group of qualified product images. Thereby, the classification model generation system 1 can obtain defective part image candidates obtained by synthesizing diverse defective parts with different shapes, positions, etc. on the qualified product images.

[0148] Next, the classification model generation system 1 prepares a mask image in which the areas where defective parts are not desired to be synthesized are filled with black, and performs image selection processing (S19) for determining whether the defective image candidates synthesized in step S17 are appropriate as defective images. Thereby, the classification model generation system 1 can obtain a group of defective images in which defective image candidates obtained by synthesizing defective parts in areas where defective parts are not desired to be synthesized on the qualified product images are selected and excluded. The group of qualified product images obtained in step S14 and the group of defective images obtained in step S19 are used as the generated image group in the learning and evaluation stages.

[0149] After the image generation stage is performed as described above, the learning and evaluation stages are carried out.

[0150] More specifically, the classification model generation system 1 first extracts a part of the generated image group, that is, a part of the group of qualified product images and the group of defective images, as the initial learning images (S20). Here, the extracted initial learning images are composed of about 10 to 30 qualified product images and defective images. Next, the classification model generation system 1 uses the initial learning images extracted in step S20 to perform the learning process of the classification model (S21). Next, the classification model generation system 1 extracts the generated image group other than the initial learning images from the generated image group as the evaluation images (S22), and performs an evaluation process (S23) for making the classification model evaluate the extracted evaluation images. Thereby, the classification model generation system 1 can obtain the evaluation results obtained by evaluating with the classification model learned using the initial learning images.

[0151] Next, the classification model generation system 1 performs the following determination process: determining whether the evaluation result obtained by evaluating with the classification model learned using the initial learning images is a wrong answer (S24). Here, the evaluation images are generated in the image generation stage, and it is known whether each evaluation image is a qualified product image or a non-qualified image. Therefore, in the determination process of step S24, when this evaluation result is a wrong answer and the evaluation image of this evaluation result is determined to be a wrong answer image, the learning image group is updated by adding only the wrong answer images to the learning image group. In this way, the classification model generation system 1 can accumulate only the wrong answer images into the learning image group to increase the number of the learning image group.

[0152] Next, the classification model generation system 1 stipulates the learning end condition, makes a learning end determination while performing a learning process of making the classification model learn using the learning image group updated in step S24 (S25). Thereby, the classification model generation system 1 can end the learning process of the classification model at the time point when the learning end condition is reached, and thus can repeat the learning by only feeding back the wrong answers.

[0153] In addition, sometimes the number of evaluation images is not enough to meet the learning end condition. In this case, it is only necessary to repeat the image generation stage again, increase the generated image group, and on this basis, perform the evaluation process in step S23 to the learning end determination in step S25, and make the image generation stage and the learning and evaluation stages cycle until the learning end condition is met (S26).

[0154] (Example 2)

[0155] In Example 2, the effectiveness of the classification model generation method according to the embodiment was verified. The verification results will be briefly described.

[0156] In this verification, one artificially generated seed image regarded as a foreign object and 1000 pre-collected and prepared qualified product images were used. The learning end condition was determined to be passing 30000 evaluation images without wrong answers or performing 30 learning cycles. In addition, in this verification, a CNN model consisting of 16 layers, namely VGG16, was used as the classification model, Adam was used as the optimizer (optimization algorithm), and the learning rate was set to 10 -6 , and the number of epochs was set to 30. In addition, the learning rate varies according to the number of learning times.

[0157] Then, as a result of this verification, the following achievements were obtained.

[0158] · Learning cycle: 7 times

[0159] ·Final number of learning images: 517 for OK and 102 for NG

[0160] ·Final accuracy: 100%

[0161] According to the above verification results, based on the classification model generation method according to the embodiment, the accuracy of the classification model can be 100%.

[0162] In this way, the effectiveness of the classification model generation method according to the embodiment is verified.

[0163] [5 Effects, etc.]

[0164] As described above, according to the embodiment, the classification model generation system 1 etc. can generate a classification model capable of classifying qualified product images and unqualified product images without collecting and selecting unqualified images.

[0165] For example, in the case of collecting unqualified images from an actual manufacturing process etc., it is necessary to spend time collecting unqualified images and perform selection operations such as giving annotations. Although there is also a method of only collecting qualified product images and generating a classification model only using qualified product images, it is difficult to set the judgment criteria for unqualified images such as the definition of the unqualified patterns (defects) to be recognized.

[0166] On the other hand, in the present embodiment, although qualified product images are collected from an actual manufacturing process etc., unqualified images are not collected from an actual manufacturing process etc., but unqualified images are generated based on seed images artificially generated and imitating defects (unqualified parts) such as damage and foreign matters. Thus, a learning image group generated from the generated unqualified images and the collected qualified product images can be used to generate a classification model. That is, a classification model capable of classifying qualified product images and unqualified product images can be generated without collecting and selecting unqualified images.

[0167] In addition, according to the embodiment, the collected qualified product images can be augmented by geometric transformation etc., so diverse unqualified images can be generated by synthesizing the augmented qualified product images and the unqualified part images generated based on the seed images.

[0168] Here, in the generation of unqualified part images, a GAN model can be used, and this GAN model is learned using a group of seed images obtained by augmenting the seed images through geometric transformation. Thus, as long as the seed images are artificially made (generated), the range and shape where the actual unqualified parts are located can be roughly set, so variations of unqualified part images with shapes that are difficult to achieve only by geometric transformation can be generated.

[0169] Further, according to an embodiment, an evaluation of the generated classification model is performed, and learning of the classification model is repeated while using a learning image group including mis-answered images and feeding back only mis-answers. Thereby, it is possible to grow into a classification model with better accuracy while suppressing the computational cost required for the learning process.

[0170] In addition, the learning image group that appears as a by-product includes only mis-answered images in addition to the images that were initially used as learning images, and thus is a carefully selected learning image group. Thereby, compared to the case of using images randomly selected from the generated non-conforming images and conforming images to make the classification model learn, it is possible to easily generate a classification model with better accuracy. Further, since the number of images included in the learning image group can also be relatively small, it is possible to learn a classification model with better accuracy at high speed.

[0171] (Supplementary Note)

[0172] In addition, through the description of the above embodiment, the following technology is disclosed.

[0173] (Technology 1) A classification model generation system includes: an acquisition unit that acquires a conforming image group including a plurality of conforming images; a non-conforming part image generation unit that generates a plurality of non-conforming part images based on a seed image group, where the seed image group is obtained by geometric transformation of seed images that are generated by manual drawing and imitate non-conforming parts; a synthesis processing unit that generates a non-conforming image group by synthesizing the plurality of non-conforming part images with the conforming image group respectively; and a classification model generation unit that performs classification learning using the conforming image group and a part of the non-conforming image group as a learning image group, thereby generating a classification model.

[0174] In this way, it is possible to generate non-conforming images based on seed images that are manually generated and imitate defects (non-conforming parts) such as damage and foreign matters, instead of collecting non-conforming images from actual manufacturing processes and the like. Thereby, it is possible to generate a classification model using a learning image group generated from the generated non-conforming images and the collected conforming images. That is, it is possible to generate a classification model that can classify conforming images and non-conforming images without collecting and selecting non-conforming images.

[0175] (Technology 2) The classification model generation system according to Technology 1, wherein the acquisition unit includes: an input unit into which the plurality of conforming images are input; and a conforming image generation unit that performs geometric transformation on the plurality of conforming images input to the input unit to generate one or more new conforming images, and acquires the plurality of conforming images and the one or more new conforming images as the conforming image group.

[0176] In this way, it is possible to augment a plurality of pre-prepared qualified product images through geometric transformation. As a result, the plurality of qualified product images are augmented, and a group of qualified product images with rich variability can be obtained.

[0177] (Technique 3) A classification model generation system according to Technique 1 or Technique 2, wherein the classification model generation unit includes: a learning unit that causes the model to perform classification learning on the learning image group; an evaluation unit that causes the model to evaluate at least a part of the qualified product image group and the unqualified image group other than the learning image group used in the classification learning of the model among the qualified product image group and the unqualified image group; and a determination unit that determines whether the evaluation result obtained by the evaluation of the model is a wrong answer, and only adds the qualified product image or unqualified image determined to be a wrong answer to the learning image group, thereby updating the learning image group, and the learning unit causes the model to perform classification learning on the updated learning image group, thereby generating the classification model.

[0178] In this way, it is possible to perform the evaluation of the generated classification model, and while only feeding back the wrong answers, repeat the learning of the classification model using the learning image group including the images with wrong answers. As a result, it is possible to grow into a classification model with better accuracy while suppressing the computational cost required for the learning process.

[0179] In addition, the learning image group that appears as a by-product includes only the images with wrong answers in addition to the images initially used as learning images, so it is a strictly selected learning image group. Therefore, compared with the case of using randomly selected images from the generated unqualified images and qualified product images to cause the classification model to learn, it is possible to easily generate a classification model with better accuracy. In addition, since the number of images included in the learning image group can also be relatively small, it is possible to learn a classification model with better accuracy at high speed.

[0180] (Technique 4) A classification model generation system according to Technique 3, wherein the evaluation result includes a determination result indicating whether the image input to the model is a qualified product image or an unqualified image and the accuracy of the determination result.

[0181] (Technique 5) A classification model generation system according to Technique 4, wherein the determination unit determines that the evaluation result is a wrong answer when the determination result is incorrect or when the accuracy is below a specified value.

[0182] (Technology 6) A classification model generation system according to any one of Technologies 1 to 5, wherein the synthesis processing unit includes: a synthesis unit that synthesizes each of the plurality of non-conforming part images with the group of conforming product images; and an image selection unit that determines whether the image synthesized by the synthesis unit is appropriate as a non-conforming image.

[0183] Thus, inappropriate non-conforming images can be identified and excluded, and thus more appropriate non-conforming images can be generated.

[0184] (Technology 7) A classification model generation system according to Technology 6, wherein when the area and brightness of the non-conforming part reflected in the synthesized image satisfy a specified standard, the image selection unit determines that the synthesized image is appropriate as the non-conforming image, and when the area and brightness of the non-conforming part reflected in the synthesized image do not satisfy the specified standard, the image selection unit determines that the synthesized image is inappropriate as the non-conforming image, and the image selection unit includes only the images determined to be appropriate in the synthesized images into the group of non-conforming images.

[0185] In this way, for example, a non-conforming image obtained by synthesizing a non-conforming part in an area where it is not desired to synthesize a non-conforming part with a conforming product image can be determined to be inappropriate and excluded. Thus, the classification model can be learned to accurately classify non-conforming images with non-conforming patterns to be recognized as non-conforming images.

[0186] (Technology 8) A classification model generation system according to any one of Technologies 1 to 7, wherein the non-conforming part image generation unit generates the non-conforming part images through a learned GAN (Generative Adversarial Networks) model, and the GAN model is obtained by learning using the group of seed images.

[0187] In this way, by using a GAN model learned using a group of seed images based on seed images, variant changes in the shape of non-conforming parts that are difficult to achieve only through geometric transformation can be generated. Thus, diverse non-conforming part images can be obtained.

[0188] (Technology 9) A classification model generation system according to any one of Technologies 1 to 7, wherein the seed image is composed of a background and a defective part imitating the non-conforming part, and the defective part and the background can be separated by a single threshold.

[0189] In this way, the defective part of the seed image and the background can be separated by a single threshold, so that processing such as making the background part transparent can be easily performed. Thus, it is possible to easily synthesize the geometrically transformed seed image with the pre-prepared qualified product image, and thus it is possible to easily generate a defective image.

[0190] (Technique 10) The classification model generation system according to Technique 9, wherein the background is painted in a single color.

[0191] Thus, processing such as making the background part transparent can be easily performed.

[0192] (Possibility of other embodiments)

[0193] As described above, in the embodiments, the classification model generation system, the classification model generation method, and the program of the present disclosure have been described, but the subject and the device for implementing each process are not particularly limited. The process may also be performed by a processor or the like (described below) assembled in a specific device arranged locally. Alternatively, the process may also be performed by a cloud server or the like arranged at a location different from the local device.

[0194] In addition, the present disclosure is not limited to the above-described embodiments. For example, the constituent elements described in this specification may be arbitrarily combined, and alternatively, other embodiments in which some of the constituent elements are excluded may be set as the embodiments of the present disclosure. In addition, modification examples obtained by applying various modifications conceivable by those skilled in the art to the above-described embodiments within the scope not departing from the gist of the present disclosure, that is, the meaning indicated by the statements described in the claims, are also included in the present disclosure.

[0195] In addition, the present disclosure can also include the following situations.

[0196] (1) Specifically, the above-described system is a computer system composed of a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, etc. A computer program is stored in the RAM or the hard disk unit. The microprocessor operates according to the computer program, whereby each device realizes its function. Here, the computer program is constituted by combining a plurality of instruction codes representing instructions for a computer in order to realize a prescribed function.

[0197] (2) Part or all of the components constituting the above-described system may also be constituted by a single system LSI (Large Scale Integration). A system LSI is a super multi-functional LSI manufactured by integrating a plurality of structural parts on a single chip. Specifically, it is configured to include a computer system such as a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The microprocessor operates according to the computer program, whereby the system LSI realizes its functions.

[0198] (3) Part or all of the components constituting the above-described system may also be constituted by an IC card or a single module that can be detached from and attached to each device. The IC card or the module is a computer system constituted by a microprocessor, ROM, RAM, etc. The IC card or the module may also include the above-described super multi-functional LSI. The microprocessor operates according to a computer program, whereby the IC card or the module realizes its functions. The IC card or the module may also have anti-tampering properties.

[0199] (4) In addition, the present disclosure may also be set as the method shown above. In addition, it may also be set as a computer program for implementing these methods by a computer, and may also be a digital signal constituted by the computer program.

[0200] (5) In addition, the present disclosure may also be set as a mode of recording the computer program or the digital signal on a computer-readable recording medium, such as a floppy disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray (registered trademark) Disc, Blu-ray disc), a semiconductor memory, etc. In addition, it may also be set as the digital signal recorded on these recording media.

[0201] In addition, the present disclosure may also be set as a mode of transmitting the computer program or the digital signal via an electrical communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, etc.

[0202] In addition, the present disclosure may also be set as a computer system including a microprocessor and a memory, the memory storing the above computer program, and the microprocessor operating according to the computer program.

[0203] In addition, it may also be implemented by an independent other computer system by recording the program or the digital signal on the recording medium and transferring it, or by transferring the program or the digital signal via the network or the like.

[0204] Industrial Applicability

[0205] The present disclosure can be applied to a classification model generation system, a classification model generation method, and a program, and in particular, can be applied to a classification model generation system, a classification model generation method, and a program used for image appearance inspection, image monitoring, etc. in a factory.

[0206] Description of Reference Numerals

[0207] 1: Classification model generation system; 10: Image generation device; 11: Acquisition unit; 12: Defective part image generation unit; 13: Synthesis processing unit; 20: Model generation device; 21: Storage unit; 22: Extraction unit; 23: Classification model generation unit; 24: Learning DB; 111: Input unit; 112: Qualified product image generation unit; 121: Preprocessing unit; 122: GAN model; 123: Generation unit; 131, 1212: Geometric transformation unit; 132: Synthesis unit; 133: Image selection unit; 231, 1213: Learning unit; 232: Classification model; 233: Evaluation unit; 234: Determination unit; 1211: Seed image generation unit.

Claims

1. A classification model generation system, comprising: An acquisition unit that acquires a group of qualified product images including a plurality of qualified product images; A defective part image generation unit that generates a plurality of defective part images based on a group of seed images, wherein, The group of seed images is obtained by geometric transformation of seed images that are generated by manual drawing and imitate defective parts; A synthesis processing unit that generates a group of defective images by respectively synthesizing the plurality of defective part images with the group of qualified product images; and A classification model generation unit that uses the group of qualified product images and a part of the group of defective images as a learning image group to perform classification learning, thereby generating a classification model.

2. The classification model generation system according to claim 1, wherein, The acquisition unit includes: An input unit to which the plurality of qualified product images are input; and A qualified product image generation unit that performs geometric transformation on the plurality of qualified product images input to the input unit to generate one or more new qualified product images, and acquires the plurality of qualified product images and the one or more new qualified product images as the group of qualified product images.

3. The classification model generation system according to claim 1, wherein, The classification model generation unit includes: A learning unit that causes the model to perform classification learning on the learning image group; An evaluation unit that causes the model to evaluate at least a part of the group of qualified product images and the group of defective images other than the learning image group used in the classification learning of the model in the group of qualified product images and the group of defective images; and A determination unit that determines whether the evaluation result obtained by evaluating with the model is a wrong answer, and only adds the qualified product image or defective image determined to be a wrong answer to the learning image group, thereby updating the learning image group, The learning unit causes the model to perform classification learning on the updated learning image group, thereby generating the classification model.

4. The classification model generation system according to claim 3, wherein, The evaluation result includes a determination result indicating whether the image input to the model is a qualified product image or a defective image and the accuracy of the determination result.

5. The classification model generation system according to claim 4, wherein, The determination unit determines that the evaluation result is a wrong answer when the determination result is incorrect or when the accuracy is below a specified value.

6. The classification model generation system according to any one of claims 1 to 5, wherein, The synthesis processing unit includes: A synthesis unit that respectively synthesizes the plurality of defective part images with the group of qualified product images; and An image selection unit that determines whether the image synthesized by the synthesis unit is appropriate as a defective image.

7. The classification model generation system according to claim 6, wherein, When the area and brightness of the defective part reflected in the synthesized image satisfy a specified criterion, the image selection unit determines that the synthesized image is appropriate as the defective image. In the case where the area and brightness of the non-conforming part reflected in the synthesized image do not meet the specified criteria, the image selection unit determines that the synthesized image is not appropriate as the non-conforming image. The image selection unit includes only the images determined to be appropriate among the synthesized images in the non-conforming image group.

8. The classification model generation system according to any one of claims 1 to 5, wherein, the non-conforming part image generation unit generates the non-conforming part image through a learned generative adversarial network model, i.e., a GAN model, and the GAN model is obtained by learning using the seed image group.

9. The classification model generation system according to any one of claims 1 to 5, wherein, the seed image is composed of a background and a defect part imitating the non-conforming part, and the defect part and the background can be separated by a single threshold.

10. The classification model generation system according to claim 9, wherein, the background is painted in a single color.

11. A classification model generation method, comprising: an acquisition step of acquiring a group of qualified product images including a plurality of qualified product images; a non-conforming part image generation step of generating a plurality of non-conforming part images based on a seed image group, wherein the seed image group is obtained by performing a geometric transformation on a seed image generated by manual drawing and imitating the non-conforming part; a synthesis processing step of generating a non-conforming image group by synthesizing the plurality of non-conforming part images with the group of qualified product images respectively; and a classification model generation step of performing classification learning using the group of qualified product images and a part of the non-conforming image group as a learning image group, thereby generating a classification model.

12. A program that causes a computer to execute the following steps: an acquisition step of acquiring a group of qualified product images including a plurality of qualified product images; a non-conforming part image generation step of generating a plurality of non-conforming part images based on a seed image group, wherein, the seed image group is obtained by performing a geometric transformation on a seed image generated by manual drawing and imitating the non-conforming part; a synthesis processing step of generating a non-conforming image group by synthesizing the plurality of non-conforming part images with the group of qualified product images respectively; and a classification model generation step of performing classification learning using the group of qualified product images and a part of the non-conforming image group as a learning image group, thereby generating a classification model.

Images