Image generation device, image generation method, and program

Abstract

To provide an image generation device, an image generation method, and a program for generating an image for learning.SOLUTION: An image generation device 10, which is an image generation device that generates a learning image for learning a classifier used for appearance inspection of an object, includes a class selection unit 101 configured to select one of a plurality of classes in which images input to the classifier are classified, an image selection unit 102 configured to select N (N is an integer equal to or greater than 2) learning images from a set of learning images belonging to one class, a random number generation unit 103 configured to generate N-1 random numbers that take a value of 0 or greater and 1 or smaller from a predetermined probability distribution, and an image generation unit 104 configured to generate a new learning image by synthesizing the N learning images with the N-1 random numbers as a synthesis ratio.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image generating device, an image generating method, and a program. [Background technology]

[0002] In recent years, with the development of machine learning, classifiers have been put to practical use that input an image of the appearance of an inspection target and classify the image as either a normal image or an abnormal image. Such classifiers are realized, for example, by training a neural network or the like using a supervised learning method using training data that includes an input image and a ground truth label that indicates whether the input image is a normal image or an abnormal image.

[0003] It is generally known that the accuracy of a classifier improves as the amount of training data increases, but there are cases where it is not possible to prepare a sufficient number of normal and abnormal images.

[0004] For example, in a factory production line, the defect rate is generally low, so a sufficient number of abnormal images cannot be obtained. In response to this, a conventional technique for generating abnormal images for learning is known in which defective areas contained in an abnormal image are manually extracted and transplanted into a normal image (for example, Patent Document 1). In addition to this, there is also known a technique for generating normal and abnormal images for learning by performing some kind of transformation or processing on the image (for example, geometric transformation (horizontal flipping or vertical flipping of the image, rotation, enlargement / reduction, translation of defective areas in the image, etc.), addition of noise, filtering, etc.).

[0005] Furthermore, for example, if a system only stores abnormal images, a sufficient number of normal images cannot be obtained. To address this issue, it is possible to introduce a system that also stores normal images. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2018-205123 Summary of the Invention [Problem to be solved by the invention]

[0007] However, if a sufficient number of abnormal images cannot be obtained, the technology described in Patent Document 1 requires manual extraction of the defect area, which places a heavy burden on the user and requires a lot of time when generating many abnormal images. Furthermore, technologies that perform some kind of transformation or processing on images can sometimes result in the loss of defect features. For example, translating a defect area can result in part of the defect area no longer being included in the image, or shrinking a defect area can result in the defect area being crushed.

[0008] On the other hand, if a sufficient number of normal images cannot be obtained, introducing a system that also stores normal images would require a great deal of cost.

[0009] The embodiments of the present invention have been made in view of the above points, and have an object to generate images for learning. [Means for solving the problem]

[0010] In order to achieve the above object, an image generation device according to one embodiment generates training images for training a classifier used in visual inspection of objects, and includes: a class selection unit that selects one of a plurality of classes into which an image input to the classifier is classified; an image selection unit that selects N (where N is an integer equal to or greater than 2) training images from a set of training images belonging to the one class; a random number generation unit that generates N-1 random numbers taking values ​​between 0 and 1 from a predetermined probability distribution; and an image generation unit that generates a new training image by combining the N training images using the N-1 random numbers as a combination ratio. [Effects of the Invention]

[0011] It is possible to generate images for training. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a diagram illustrating an example of the overall configuration of an image generating apparatus according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of an image set. [Figure 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an image generating apparatus according to an embodiment of the present invention. [Figure 4] 10 is a flowchart showing an example of the flow of image generation processing according to the present embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of image synthesis. DETAILED DESCRIPTION OF THE INVENTION

[0013] An embodiment of the present invention will be described below. In this embodiment, an image generating device 10 will be described that generates training images for training a classifier that classifies input images. Here, the classes into which the input images are classified may be two classes: a normal class into which the input image is classified when it is a normal image, and an abnormal class into which the input image is classified when it is an abnormal image, or multiple classes (three or more classes) in which a different abnormal class is classified for each type of abnormality. An example of multiple classes is, for example, when there are two types of abnormalities, "abnormality type A" and "abnormality type B," three classes are defined: a normal class, a first abnormal class into which an abnormal image of abnormality type A is classified, and a second abnormal class into which an abnormal image of abnormality type B is classified. Generally, when C types of abnormalities exist, where C is an integer greater than or equal to 1, C+1 classes are similarly defined.

[0014] The classifier (hereinafter also referred to as "target classifier") is realized, for example, by a machine learning model such as a neural network, and is trained by an arbitrary optimization method using training images in which images are associated with training data indicating which class the images belong to. Hereinafter, a case will be described in which the number of training images belonging to at least one of the classes is assumed to be insufficient, and training images belonging to this class are generated. Note that "the number of training images is insufficient" means that although a certain number of training images (for example, several to several tens of training images) have been obtained, the number of training images obtained is not sufficient to achieve the desired accuracy when training the target classifier.

[0015] <Overall structure> First, the overall configuration of an image generating device 10 according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the overall configuration of an image generating device 10 according to this embodiment.

[0016] As shown in FIG. 1, an image generating device 10 according to this embodiment includes a class selecting unit 101, an image selecting unit 102, a random number generating unit 103, an image generating unit 104, and a storage unit 105.

[0017] The storage unit 105 stores training images for training the target classifier. For example, when training a target classifier that performs three-class classification into a normal class, a first abnormal class, and a second abnormal class, as shown in FIG. 2 , an image set of training images 1100 belonging to the normal class, an image set of training images 1200 belonging to the first abnormal class, and an image set of training images 1300 belonging to the second abnormal class are stored in the storage unit 105. The training images 1100 are normal images, the training images 1200 are abnormal images of an abnormality type A, and the training images 1300 are abnormal images of an abnormality type B. The training images 1200 include an abnormal location 1210 of the abnormality type A, and the training images 1300 include an abnormal location 1310 of the abnormality type B. Here, the normal images refer to, for example, images captured of an object (e.g., a finished product, a semi-finished product, etc.) that is determined to be normal by visual inspection. On the other hand, an abnormal image is, for example, an image taken of an object that is determined to be abnormal in a visual inspection. Note that an object that is determined to be abnormal in a visual inspection may have some kind of abnormality in its appearance (for example, a defect such as a scratch, dent, or color unevenness). In the example shown in FIG. 2, for example, abnormality type A is a scratch A that may occur in a certain process A, and abnormality type B is a scratch B that may occur in another process B. However, this is just one example, and, for example, abnormality type A may be a dent and abnormality type B may be a scratch.

[0018] The class selection unit 101 selects, from among the predetermined classes, a class (hereinafter also referred to as a "target class") that is to be used to generate learning images.

[0019] Image selection unit 102 randomly selects N training images to be used for generating new training images from among the training images belonging to the target class. That is, image selection unit 102 randomly selects N training images from the image set of the target class from among the training images stored in storage unit 105. Here, N is an integer equal to or greater than 2 and is set in advance.

[0020] The random number generation unit 103 generates N-1 random numbers between 0 and 1 in accordance with a predetermined probability distribution.

[0021] The image generation unit 104 generates a new training image by synthesizing the training images selected by the image selection unit 102 using the random number generated by the random number generation unit 103. The new training image is stored in the storage unit 105 as a training image belonging to the target class.

[0022] <Hardware configuration> Next, the hardware configuration of the image generating device 10 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the hardware configuration of the image generating device 10 according to this embodiment.

[0023] 3, the image generating device 10 according to this embodiment is realized by a general computer or computer system, and includes an input device 201, a display device 202, an external I / F 203, a communication I / F 204, a processor 205, and a memory device 206. Each of these pieces of hardware is connected to each other via a bus 207 so as to be able to communicate with each other.

[0024] The input device 201 is, for example, a keyboard, a mouse, a touch panel, etc. The display device 202 is, for example, a display, etc. Note that the image generating device 10 does not necessarily have to have at least one of the input device 201 and the display device 202.

[0025] The external I / F 203 is an interface with an external device. The external device may be a recording medium 203a. The image generating device 10 can read from and write to the recording medium 203a via the external I / F 203. The recording medium 203a may store, for example, one or more programs that implement each functional unit (class selection unit 101, image selection unit 102, random number generation unit 103, and image generation unit 104) of the image generating device 10.

[0026] The recording medium 203a may be, for example, a CD (Compact Disc), a DVD (Digital Versatile Disc), an SD memory card (Secure Digital memory card), or a USB (Universal Serial Bus) memory card.

[0027] The communication I / F 204 is an interface for connecting the image generating device 10 to a communication network. Note that one or more programs for realizing each functional unit of the image generating device 10 may be acquired (downloaded) from a predetermined server device or the like via the communication I / F 204.

[0028] The processor 205 is, for example, a variety of arithmetic devices such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). Each functional unit of the image generating device 10 is realized by, for example, processing that is executed by the processor 205 according to one or more programs stored in the memory device 206 or the like.

[0029] The memory device 206 is, for example, any of various storage devices such as a hard disk drive (HDD), a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), a flash memory, etc. The storage unit 105 of the image generating device 10 can be realized using the memory device 206. Note that the storage unit 105 may also be realized using, for example, a storage device (for example, a database server, etc.) connected to the image generating device 10 via a communication network.

[0030] The image generation device 10 according to this embodiment has the hardware configuration shown in Fig. 3, and is thereby able to realize the image generation process described below. Note that the hardware configuration shown in Fig. 3 is an example, and the image generation device 10 may have other hardware configurations. For example, the image generation device 10 may have multiple processors 205 or multiple memory devices 206.

[0031] <Image generation processing> Next, the flow of image generation processing according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of the flow of image generation processing according to this embodiment.

[0032] First, the class selection unit 101 selects a target class from among the predetermined classes (step S101). At this time, for example, the target class may be selected by a user selecting one class from among the predetermined classes, or a class with a number of training images less than a predetermined reference value or a class with the fewest number of training images may be selected as the target class. Note that the predetermined reference value may be set to, for example, a value that has been empirically confirmed to provide a desired accuracy when training a target classifier.

[0033] Here, all classes or multiple classes may be selected as target classes in the above step S101. In this case, the following steps S102 to S104 may be repeatedly executed for each target class.

[0034] Next, the image selection unit 102 randomly selects N training images to be used for generating new training images from the image set of the target class (step S102).

[0035] Next, the random number generation unit 103 generates N-1 random numbers between 0 and 1 in accordance with a predetermined probability distribution (step S103). As the predetermined probability portion, any probability distribution in which the possible values ​​of the random variable are between 0 and 1 inclusive can be used, for example, a beta distribution. When a beta distribution is used, any value can be set for its parameters, but it is preferable to set the parameters to values ​​that will generate many random numbers in the range of 0.0 to 0.2 or 0.8 to 1.0, for example.

[0036] Hereinafter, N random numbers generated by the random number generation unit 103 are referred to as r1, . . . , r N-1is represented as follows. Note that these N random numbers r1, ···, r N-1 may all be generated according to the same probability distribution, or a plurality of probability distributions may be used to generate a total of N random numbers according to each probability distribution. For example, when two probability distributions are used, the random numbers r1, ···, r K according to the first probability distribution and the random numbers r K+1 , ···, r N-1 according to the second probability distribution may be generated. Similarly, for example, when three probability distributions are used, the random numbers r1, ···, r K according to the first probability distribution, the random numbers r K+1 , ···, r L according to the second probability distribution, and the random numbers r L+1 , ···, r N-1 according to the third probability distribution may be generated. The same applies when four or more probability distributions are used. Note that K and L are integers satisfying 1 < K < L < N - 1.

[0037] Next, the image generation unit 104 generates a new learning image by synthesizing the N learning images selected in step S102 above using the random numbers r1, ···, r N-1 generated in step S103 above (step S104). The new learning image is stored in the storage unit 105 as a learning image belonging to the target class.

[0038] · When N = 2 As an example, when N = 2, the target class is the "first abnormal class", and the learning images 1200 - m1 and 1200 - m2 are selected in step S102 above, this will be described. For simplicity, let r = r1.

[0039] At this time, the image generating unit 104 generates a new training image 1200-m3 by combining each pixel value of the training image 1200-m1 with each pixel value of the training image 1200-m2 using a random number r as a combining ratio, as shown in Fig. 5. Note that the abnormality area 1210-m3 in the training image 1200-m3 corresponds to the abnormality area 1210-m1 in the training image 1200-m1, and the abnormality area 1220-m3 corresponds to the abnormality area 1210-m2 in the training image 1200-m2.

[0040] To explain the above synthesis in more detail, for example, let the number of pixels in the horizontal direction (horizontal size) of the learning image 1200 be I, and the number of pixels in the vertical direction (vertical size) be J. Then, let each pixel value of the learning image 1200-m1 be x 1ij , the pixel value of the training image 1200-m2 is x 2ij In this case, for each i and j, the pixel value y ij is calculated as follows:

[0041] y ij =rx 1ij +(1-r)x 2ij As a result, a new training image 1200-m3 belonging to the target class is generated.

[0042] As mentioned above, when using the beta distribution as the probability distribution for generating the random number r, it is preferable to set the parameters to values ​​that generate many values ​​within the range of 0.0 to 0.2 or 0.8 to 1.0. This is because it has been experimentally confirmed that when the value of the random number r is within the range of 0.0 to 0.2 or 0.8 to 1.0, it is possible to generate training images that can achieve higher accuracy when training the target classifier.

[0043] When N≧3 More generally, we will explain the case where N≧3. In this case, the random numbers r1, , r N-1A new training image is generated by synthesizing the pixel values ​​of the N training images with the synthesis ratio x. nij (where n = 1, , N), then for each i, j, each pixel value y ij is calculated as follows:

[0044] y ij =r1x 1ij +(1-r1)r2x 2ij +(1-r1)(1-r2)r3x 3ij +···+(1-r1)(1-r2)···(1-r N-1 )x Nij As a result, new learning images belonging to the target class are generated.

[0045] Note that, although the image generation process shown in FIG. 4 has been described as a case in which one new training image is generated, when multiple new training images are generated, the above steps S101 to S104 can be repeatedly executed. Furthermore, when generating multiple new training images, if it is desired to fix the target class, only the above steps S102 to S104 can be repeated. Similarly, if it is desired to fix the random number used as the blending ratio, the above step S103 can be excluded from the repetition. Furthermore, for example, the above step S103 can be re-executed every time a predetermined number of new training images are generated, to generate a new random number to be used as the blending ratio.

[0046] <Summary> As described above, the image generating device 10 according to this embodiment uses N training images belonging to the same class and N-1 combination ratios according to a predetermined probability distribution to combine these N training images to generate a new training image. This enables the image generating device 10 according to this embodiment to increase the number of training images belonging to the class (in other words, to expand the data for the class).

[0047] Furthermore, since combinations of N training images can be randomly selected and composite ratios can be generated from various probability distributions, it is possible to diversify the features of the training images belonging to the class and increase the variation. Furthermore, since new training images can be generated by the simple process of combining N training images, it is possible to perform data expansion without relying on expert knowledge or the type of data (images), and without incurring additional effort or cost. Therefore, by performing training using these training images, it is expected that a target classifier with desired accuracy can be constructed.

[0048] The present invention is not limited to the specifically disclosed embodiments above, and various modifications, changes, and combinations with known techniques are possible without departing from the scope of the claims. For example, known data extension (geometric transformation, noise addition, filter processing, etc.) may be performed on new training images to further increase the number of new training images, or known data extension may be performed on training images stored in storage unit 105 to increase the number of training images, and then new training images may be generated. [Explanation of symbols]

[0049] 10 Image generation device 101 Class Selection Section 102 Image selection section 103 Random Number Generation Unit 104 Image generation unit 105 Storage section 201 Input Device 202 Display device 203 External I / F 203a Recording media 204 Communication I / F 205 processors 206 Memory Device 207 Bus

Claims

1. An image generation device that generates learning images for training a classifier used in visual inspection of an object, a class selection unit that selects one of a plurality of classes into which an image input to the classifier is classified; an image selection unit that selects N (where N is an integer equal to or greater than 2) training images from a set of training images that belong to the one class and have the same size; a random number generator that generates N-1 random numbers having values ​​between 0 and 1 using M (where M is an integer between 2 and N) different probability distributions; an image generation unit that generates a new training image by combining pixel values ​​at the same positions of the N training images using the N-1 random numbers as a combination ratio; An image generating device having:

2. The image generating device of claim 1 , wherein the M different probability distributions include a beta distribution.

3. 3. The image generating device according to claim 2, wherein parameters of the beta distribution are set so that the values ​​of the N-1 random numbers are within a range of either 0.0 to 0.2 or 0.8 to 1.

0.

4. The image generation unit The size of each of the N learning images is I×J, and each pixel value of each of the N learning images is x ｎｉｊ (where n=1, ..., N, 1≦i≦I, 1≦j≦J), and the N-1 random numbers are １ , ..., r Ｎ－１ In this case, r １ x １ｉｊ + (1-r １ )r ２ x ２ｉｊ + (1-r １ ) (1-r ２ )r ３ x ３ｉｊ + ... + (1-r １ ) (1-r ２ ) ... (1-r Ｎ－１ ) x Ｎｉｊ The pixel value y of the new learning image is ｉｊ The image generating device according to claim 1 , wherein the new training image is generated by synthesizing the N training images by calculating:

5. a storage unit for storing the learning image; The class selection unit 5. The image generating device according to claim 1, wherein the one class is selected from the plurality of classes, the class having the smallest number of learning images among the learning images stored in the memory unit.

6. 1. An image generation method for generating learning images for training a classifier used in visual inspection of an object, comprising: a class selection step of selecting one of a plurality of classes into which an image input to the classifier is classified; an image selection step of selecting N (where N is an integer equal to or greater than 2) training images from a set of training images that belong to the one class and have the same size; A random number generation procedure that generates N-1 random numbers having values ​​between 0 and 1 using M (where M is an integer between 2 and N) different probability distributions; an image generation step of generating a new training image by combining pixel values ​​at the same positions of the N training images using the N-1 random numbers as a combination ratio; A computer-implemented image generation method.

7. An image generation device that generates learning images for training a classifier used in visual inspection of an object, a class selection unit that selects one class from a plurality of classes into which the image input to the classifier is classified; an image selection unit that selects N (where N is an integer equal to or greater than 2) training images from a set of training images that belong to the one class and have the same size; a random number generator that generates N-1 random numbers having values ​​between 0 and 1 using M (where M is an integer between 2 and N) different probability distributions; an image generation unit that generates a new learning image by combining pixel values ​​at the same positions of the N learning images using the N-1 random numbers as a combination ratio; A program to function as a

Images