Inspection device, learning device, inspection method, method for generating generative model, inspection program, and learning program

Abstract

To provide an inspection apparatus capable of detecting a defective area similar in appearance to a normal area.SOLUTION: An inspection apparatus (2) includes: an image generation unit (201) that generates a generation image from an inspection image of an object with unknown defects by using a generative model (113) that is configured to generate a generated image emphasizing the defective area; and a quality determination unit (202) that determines the presence / absence of defective portion of the object using the inspection image.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an inspection device or the like that judges the quality of an object by using an image of the object. [Background technology]

[0002] Conventionally, a pass / fail judgment of an object has been performed using a machine-learned model called a generative model such as an autoencoder. For example, the following Patent Document 1 describes a method in which an image of an object to be inspected that has been judged to be a pass is used to input the image to be inspected to an autoencoder that has been machine-learned using an image of the object to be inspected that has been judged to be a pass / fail product, and a restored image is generated, and the pass / fail judgment of the object to be inspected is performed based on the difference between the image to be inspected and the restored image.

[0003] When an image of an object with defects is input to an autoencoder that has undergone the above-mentioned training, the defects are difficult to restore because they have not been trained, resulting in a difference between the image to be inspected and the restored image. This makes it possible to determine that the object has a defect. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2018-205163 Summary of the Invention [Problem to be solved by the invention]

[0005] However, while the above-mentioned conventional techniques are effective in detecting defective areas whose appearance is significantly different from normal areas, there is a problem in that for defective areas whose appearance is similar to that of normal areas, there is not enough difference for detection, making detection difficult.

[0006] An object of one aspect of the present invention is to provide an inspection device or the like capable of detecting defective portions whose appearance resembles that of normal portions. [Means for solving the problem]

[0007] In order to solve the above problems, an inspection device according to one embodiment of the present invention includes an image generation unit that uses a generative model constructed by learning to generate a generated image that emphasizes a defective portion as an input image of an object having a defective portion from an inspection image taken of an object for which the presence or absence of a defective portion is unknown, and a pass / fail judgment unit that uses the generated image generated by the image generation unit to judge the presence or absence of a defective portion of the object depicted in the inspection image.

[0008] In order to solve the above problems, a learning device according to one embodiment of the present invention includes a learning data acquisition unit that acquires an image of an object having a defective portion as an input image for machine learning, and a learning unit that constructs a generative model by learning to generate a generated image that emphasizes the defective portion when the input image is input. In addition, in order to solve the above-mentioned problems, an inspection method according to one embodiment of the present invention is an inspection method executed by an inspection device, and includes the steps of: generating a generated image from an inspection image of an object having a defect using a generative model constructed by learning to generate a generated image that emphasizes the defect using an input image of the object having a defect; and determining whether or not the object shown in the inspection image has a defect using the generated image generated in the above step.

[0009] In order to solve the above problems, a method for generating a generative model according to one embodiment of the present invention is a method for generating a generative model executed by a learning device, and includes the steps of acquiring an image of an object having a defective portion as an input image for machine learning, and constructing a generative model by learning to generate a generated image in which the defective portion is emphasized when the input image is input. Effect of the Invention

[0010] According to one aspect of the present invention, it is possible to realize an inspection device or the like capable of detecting defective portions whose appearance resembles that of normal portions. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a configuration of a main part of a learning device according to an embodiment of the present invention; [Diagram 2] 1 is a block diagram showing an example of a configuration of a main part of an inspection device according to an embodiment of the present invention; [Diagram 3] 1A to 1C are diagrams illustrating an example of inspection performed by the inspection device. [Figure 4] 1A and 1B are diagrams illustrating examples of a defective product image and an emphasized image. [Diagram 5] 11A and 11B are diagrams illustrating an example of generation of a composite image and an emphasized image. [Figure 6] 10 is a flowchart illustrating an example of a method for generating a generative model executed by the learning device. [Figure 7] 4 is a flowchart showing an example of an inspection method executed by the inspection device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] [Configuration of the learning device] The configuration of a learning device 1 according to one embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the main parts of the learning device 1. The learning device 1 is a device that uses an image of a specific object to construct a machine learning model (a generative model, described later) for determining whether the object is a good product or a defective product.

[0013] The above-mentioned object may be any object that can be judged as being good or bad based on its appearance. Specific examples of the above-mentioned object include various industrial products (which may be finished products, parts, or intermediate products) and food. A defective product is an object that does not meet the required requirements, such as an object with scratches, dirt, burns, discoloration, etc. on the surface, an object with foreign matter attached, or an object with a different shape, printing, engraving, etc. from a good product.

[0014] As shown in the figure, the learning device 1 includes a control unit 10 that controls each unit of the learning device 1, and a memory unit 11 that stores various data used by the learning device 1. The learning device 1 also includes an input unit 12 through which the learning device 1 receives input of data, etc., an output unit 13 through which the learning device 1 outputs data, etc., and a communication unit 14 through which the learning device 1 communicates with external devices.

[0015] The control unit 10 includes a composite image generation unit 101, an emphasized image generation unit 102, a learning data acquisition unit 103, and a learning unit 104. The storage unit 11 stores a non-defective product image 111, a defective product image 112, and a generative model 113.

[0016] The composite image generating unit 101 generates a composite image by combining a partial image having an appearance similar to a defective part with a part of an image of an object having no defective part. Specifically, the composite image generating unit 101 generates a composite image by combining the partial image with a non-defective image 111, which is an image of an object having no defective part. The method of generating the composite image will be described in detail later with reference to FIG. 5.

[0017] The emphasized image generating unit 102 generates an emphasized image by emphasizing a portion where the partial images are combined in the composite image generated by the composite image generating unit 101. The emphasized image can be generated from the non-defective product image 111 that is the source of the composite image, or can be generated from the composite image. The emphasized image generating unit 102 can also generate an emphasized image by emphasizing a defective part in a defective product image 112 that is an image of an object having a defective part.

[0018] The learning data acquisition unit 103 acquires an image of an object having a defective portion as an input image for machine learning. More specifically, the learning data acquisition unit 103 acquires a synthetic image for machine learning generated by the synthetic image generation unit 101 as an input image.

[0019] Furthermore, the learning data acquisition unit 103 acquires the enhanced image generated by the enhanced image generation unit 102 as teacher data (which can also be called correct data or correct image) paired with the input image. The pair of the input image and the teacher data is called learning data. The pair of the composite image and the enhanced image generated from the composite image is the learning data.

[0020] The learning data acquisition unit 103 may acquire the defective product image 112 as an input image instead of a composite image. In this case, the highlighted image generation unit 102 generates an highlighted image in which the defective part of the defective product image 112 is highlighted, and the learning data acquisition unit 103 acquires the highlighted image thus generated as training data. In other words, the pair of the defective product image 112 and the highlighted image in which the defective part is highlighted is also learning data.

[0021] The learning unit 104 constructs a generative model by learning to generate a generated image that emphasizes a defective part when an image of an object having a defective part is input as an input image. The constructed generative model is stored in the storage unit 11 as the generative model 113.

[0022] Specifically, the learning unit 104 uses a pair of a composite image and an emphasized image generated from the composite image as learning data, and performs learning so that an emphasized image is generated when the composite image is input. The learning unit 104 may also use a pair of a defective product image 112 and an emphasized image generated from the defective product image 112 as learning data, and perform learning so that an emphasized image is generated when the defective product image 112 is input.

[0023] Furthermore, when constructing the generative model, the learning unit 104 may use a non-defective product image 111, which is an image of an object having no defective parts, as an input image, and perform learning so as to generate a generated image having similar characteristics to the input image. The learning data used in this learning is only the non-defective product image 111. Although details will be described later, by performing learning using the non-defective product image 111 as learning data, it is possible to construct a generative model 113 capable of performing an inspection that achieves both versatility and accuracy.

[0024] As described above, the good-quality image 111 is an image of an object having no defective parts. For example, among images used in a quality inspection, an image of an object that has been confirmed to have no defective parts by visual inspection may be set as the good-quality image 111. Note that when the defective image 112 is used as learning data instead of a composite image and learning is not performed using the good-quality image 111 as learning data, the good-quality image 111 is not necessary.

[0025] As described above, the defective product image 112 is an image of an object having a defective portion. For example, among images used in a pass / fail inspection, an image of an object that has been confirmed to have a defective portion by visual inspection may be set as the defective product image 112. In addition, data indicating the position and range of the defective portion in the image is also recorded for the defective product image 112. Note that if the defective product image 112 is not used as learning data, the defective product image 112 is not necessary.

[0026] The generative model 113 is a trained model constructed as a result of training by the training unit 104. The generative model 113 is constructed by machine learning, and when an image of an object having a defect is input, it is sufficient that the generative model 113 generates a generated image in which the defect is emphasized.

[0027] Examples of such generative models include autoencoders, variational autoencoders, and U-Net. An autoencoder is a combination of an encoder and a decoder, in which the encoder extracts features of an input image, and the decoder decodes the extracted features to generate a generated image.

[0028] When learning is performed with an autoencoder using a non-defective image 111 as learning data, the non-defective image 111 is input to the encoder to extract features, and the extracted features are decoded by a decoder to generate a generated image. Learning is performed by adjusting the values ​​of the weight parameters in the encoder and decoder so that the error between the generated image and the non-defective image 111 is reduced.

[0029] On the other hand, when learning is performed with an autoencoder using a pair of a synthetic image and an emphasized image as learning data, the synthetic image is input to the encoder to extract features, and the extracted features are decoded by a decoder to generate a generated image. Learning is performed by adjusting the values ​​of the weighting parameters in the encoder and decoder so that the error between the generated image and the emphasized image is reduced. The same applies when learning is performed using a pair of a defective product image 112 and an emphasized image as learning data.

[0030] The same applies to the learning of variational autoencoders. Since U-Net outputs a segmentation map from an input image, when using U-Net, it is sufficient to train it to generate an image (generated image) instead of a segmentation map. It is also possible to generate generated images using GAN (Generative Adversarial Networks).

[0031] As described above, the learning device 1 includes a learning data acquisition unit 103 that acquires an image of an object having a defective portion as an input image for machine learning, and a learning unit 104 that constructs a generative model 113 by learning to generate a generated image in which the defective portion is emphasized when the input image is input.

[0032] According to the above configuration, an image of an object having a defect is acquired as an input image, and using this, a generative model 113 is constructed by learning to generate a generated image in which the defect is emphasized. In a generated image generated using this generative model 113 from an inspection image of an object in which the presence or absence of a defect is unknown, the defect is emphasized more than in the original inspection image. This makes it easier to detect a defect that has an appearance similar to a normal part. Therefore, the above configuration has the effect of making it possible to detect a defect that has an appearance similar to a normal part.

[0033] As described above, the learning device 1 includes a composite image generating unit 101 that generates a composite image by combining a partial image having an appearance similar to that of a defective part with a part of a non-defective product image 111, and an emphasized image generating unit 102 that generates an emphasized image that emphasizes the part in the composite image. Then, the learning data acquiring unit 103 acquires the composite image as an input image, and acquires the emphasized image as teacher data.

[0034] According to the above configuration, a composite image and an enhanced image are generated and acquired from a non-defective product image 111. This makes it possible to construct a generative model 113 capable of generating a generated image with sufficient accuracy for judgment, even if a sufficient number of defective product images 112 required for learning are not available.

[0035] [Configuration of the inspection device] A configuration of an inspection device 2 according to an embodiment of the present invention will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of a configuration of a main part of the inspection device 2. The inspection device 2 is a device that uses an image of an object to determine whether the object is a pass or fail product.

[0036] As shown in the figure, the inspection device 2 includes a control unit 20, a storage unit 21, an input unit 22, an output unit 23, and a communication unit 24. The control unit 20 includes an image generation unit 201 and a pass / fail determination unit 202. The storage unit 21 stores an inspection image 211, determination result data 213, and a generation model 113.

[0037] The image generating unit 201 uses an image of an object having a defect as an input image, and generates a generated image in which the defect is emphasized, from an inspection image 211 obtained by capturing an object for which the presence or absence of a defect is unknown, using a generation model 113 constructed by learning to generate a generated image in which the defect is emphasized. The generation model 113 is generated by the learning device 1, and is obtained from the learning device 1 and stored in the storage unit 21.

[0038] The quality determination unit 202 determines the presence or absence of a defective part in the object shown in the inspection image by using the generated image generated by the image generation unit 201. The quality determination unit 202 then stores in the storage unit 21 the determination result data 213 indicating the determination result as to whether the object related to the inspection image 211 is a good product or a defective product.

[0039] As described above, the inspection device 2 includes an image generation unit 201 that generates a generated image from an inspection image 211 captured of an object for which the presence or absence of a defect is unknown, using a generation model 113 constructed by learning to generate a generated image that emphasizes the defect, using an image of an object having a defect as an input image, and a pass / fail judgment unit 202 that judges the presence or absence of a defect in the object captured in the inspection image 211, using the generated image generated by the image generation unit 201.

[0040] According to the above configuration, the presence or absence of a defect is determined using a generated image generated using generative model 113 that is constructed by learning to generate a generated image in which an image of an object having a defect is emphasized as an input image. As described above, the image of an object having a defect used for learning generative model 113 may be defective product image 112 or a composite image generated by composite image generating unit 101.

[0041] In this generated image, defective areas are emphasized more than in inspection image 211, and therefore the above configuration has the effect of making it possible to detect defective areas that have an appearance similar to normal areas.

[0042] As described above, the generative model 113 may be constructed by learning to generate a generated image having similar characteristics to the input image 111 by using the non-defective image 111 as an input image.

[0043] According to the above configuration, by using a non-defective product image 111 as an input image, a generated image is generated from an inspection image 211 using a generative model 113 that has been constructed by learning to generate a generated image having similar characteristics to the input image. Then, the generated image is used to determine whether or not there is a defect.

[0044] This makes it possible to detect defective parts that have not been emphasized because they have not been learned, because such defective parts are not restored in the generated image generated from the inspection image 211 by the generative model, and this causes a difference between the inspection image 211 and the generated image.

[0045] [Examples of tests] An example of inspection by the inspection device 2 will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of inspection by the inspection device 2. More specifically, Fig. 3 shows an example of inspecting the quality of objects A1 to A3.

[0046] (Object A1) The quality inspection of the object A1 is performed using an inspection image B1 obtained by photographing the object A1 from the top side. The image a1 of the object A1 captured in the inspection image B1 includes a discolored part a11, and since the discolored part a11 is included, the object A1 is a defective product.

[0047] It is assumed here that the defective product image 112 including a discoloration similar to the discoloration a11 was used as learning data when learning the generative model 113 used by the image generating unit 201. In other words, the generative model 113 used here is a model constructed by learning using the defective product image 112 including a discoloration similar to the discoloration a11 as an input image and an enhanced image in which the discoloration is enhanced as teacher data.

[0048] In inspecting this object A1, first, the image generating unit 201 of the inspection device 2 inputs the inspection image B1 to the generation model 113 to generate a generated image B1'. As described above, the generation model 113 used here is generated by learning using the defective product image 112 that includes a discoloration similar to the discoloration a11. Therefore, in the generated image B1', the image a1 of the object A1 is roughly the same image a1', while the discoloration a11 is emphasized to become the discoloration a11'.

[0049] Next, the quality determination unit 202 determines whether or not there is a defective part in the object A1 using the generated image B1' generated by the image generation unit 201. Specifically, the quality determination unit 202 generates a difference image B1" between the inspection image B1 and the generated image B1', and determines that there is a defective part if there is a continuous area consisting of a predetermined number or more of pixels in which a difference equal to or greater than a threshold occurs in the generated difference image B1", and determines that there is no defective part if there is no such area.

[0050] It should be noted that the difference image B1" shows the difference between the pixel value of each pixel in the inspection image B1 and the pixel value of each corresponding pixel in the generated image B1'. In the difference image B1" shown in FIG. 3, pixels where the difference is equal to or greater than a threshold are binarized as white, and pixels where the difference is less than the threshold are binarized as black. In this way, a binarized image may be used to determine whether or not there is a defective area. When a binarized image is used, the pass / fail determination unit 202 detects a continuous area consisting of a predetermined number or more of white pixels as a defective area.

[0051] As described above, in the generated image B1', the discolored portion a11 is emphasized as discolored portion a11', and the difference in that portion between the inspection image B1 and the generated image B1' is also emphasized. For this reason, in the difference image B1", that portion is correctly detected as a defective portion a11". Therefore, the quality determination unit 202 can correctly determine that the target object A1 is a defective product.

[0052] (Object A2) Similar to the case of object A1, inspection of object A2 is performed using inspection image B2 obtained by photographing object A2 from the top side. Image a2 of object A2 captured in inspection image B2 includes discolored portion a21, and object A2 is defective because of the inclusion of discolored portion a21. Here, it is assumed that discolored portion a21 is not similar to any of the defective portions of the input image included in the learning data of generative model 113.

[0053] In the inspection of this object A2, the image generating unit 201 first inputs the inspection image B2 to the generative model 113 to generate a generated image B2'. As described above, the training of the generative model 113 used here does not use learning data that includes a defect similar to the discolored part a21. For this reason, the discolored part a21 is not emphasized in the generated image B2'.

[0054] It is assumed here that the generative model 113 is constructed by learning to generate a generated image having similar characteristics to the input image 111 by using the non-defective image 111 as an input image. For this reason, the discolored portion a21 is not restored in the generated image B2'.

[0055] Next, the quality determination unit 202 determines the presence or absence of defective parts in the object A2 using the generated image B2' generated by the image generation unit 201. Specifically, the quality determination unit 202 generates a difference image B2" between the inspection image B2 and the generated image B2', and determines that there is a defective part if there is a continuous area consisting of a predetermined number or more of pixels in which a difference equal to or greater than a threshold value occurs in the generated difference image B2", and determines that there is no defective part if there is no such area. As in the case of inspecting the object A1, a binary image may be used to determine the presence or absence of defective parts.

[0056] Because discolored area a21 is not restored in the generated image B2', the area of ​​discolored area a21 appears in the difference image B2" as the difference between inspection image B2 and generated image B2'. Therefore, the pass / fail judgment unit 202 can correctly detect the area as defective area a21" and correctly judge that the object A2 is a defective product.

[0057] (Object A3) The quality inspection of object A3 is also performed using inspection image B3 obtained by photographing object A3 from the top side, just like objects A1 and A2. The image a3 of object A3 shown in inspection image B3 does not contain any discolored areas. In other words, object A3 is a good product.

[0058] In the inspection of this object A3 as well, first, image generator 201 inputs inspection image B3 to generative model 113 to generate a generated image B3'. Since inspection image B3 does not contain any portion similar to the learned defective portion, a generated image B3' is generated in which an image a3' having similar features to the image a3 in inspection image B3 is captured.

[0059] Next, the quality determination unit 202 determines whether or not there are any defective parts in the object A3 using the generated image B3' generated by the image generation unit 201. Specifically, the quality determination unit 202 generates a difference image B3" between the inspection image B3 and the generated image B3', and determines that there is a defective part if there is a continuous area consisting of a predetermined number or more of pixels in which a difference equal to or greater than a threshold value occurs in the generated difference image B3", and determines that there is no defective part if there is no such area. As in the case of inspecting the object A1, a binary image may be used to determine the presence or absence of defective parts.

[0060] Since the image a3 in inspection image B3 and the image a3' in the generated image B3' have the same appearance, there is no significant difference between inspection image B3 and generated image B3'. Therefore, the quality determination unit 202 can correctly determine that the object A3 is a quality product from the difference image B3".

[0061] [Images used as learning data] The learning unit 104 of the learning device 1 generates a generative model 113 by learning using learning data consisting of an input image and teacher data (correct image) so that when an input image is input, a correct image serving as the teacher data is generated.

[0062] For this reason, the input image is an image that is known in advance to contain a defective part, such as inspection image B1 in Fig. 3, that is, defective product image 112. As defective product image 112, inspection image B1 used in actual inspection as shown in Fig. 3 can also be used. Also, instead of or in addition to defective product image 112, a composite image generated by composite image generating unit 101 may be used as an input image.

[0063] As the teacher data, an emphasized image, which is an image in which a defective part of an input image is emphasized, such as generated image B1' in Fig. 3, is used. The emphasized image is generated by the emphasized image generating unit 102. Specifically, the emphasized image generating unit 102 generates the emphasized image by changing the color of the defective part of the defective product image 112 to a predetermined emphasized color. The emphasized image generating unit 102 may also generate the emphasized image by changing the color of a part where a partial image having an appearance similar to the defective part is synthesized to a predetermined emphasized color.

[0064] As described above, in addition to learning using the above-mentioned learning data, the learning unit 104 may also perform learning using, as an input image, an image of an object having no defects, i.e., a non-defective product image 111. As the non-defective product image 111, an inspection image B3 used in an actual inspection as shown in FIG.

[0065] In learning in which the good product image 111 is used as an input image, the good product image 111 used as the input image is used as teacher data (correct image). In this learning, the learning unit 104 learns from the good product image 111, which is the input image, so that a generated image having similar characteristics to the good product image 111 is generated. Whether the good product image 111, the defective product image 112, or a composite image is used as the input image, the learning framework is the same, so it is also possible to prepare teacher data including each image in advance and learn them all at once. Note that learning in which the good product image 111 is used as an input image is sometimes classified as unsupervised learning, since the input image and the correct image are the same.

[0066] [About accent colors] It is preferable that the above-mentioned highlight color is a color component that appears relatively infrequently among the color components of the input image used for learning. This will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of a defective product image and a highlighted image.

[0067] Defective product image C shown in Fig. 4 contains a defective portion similar to that of inspection image B1 in Fig. 3. Also, Fig. 4 shows the average pixel values ​​of the R, G, and B color components in this defective product image C in a bar graph. As shown in this bar graph, in defective product image C, the average pixel value of B, i.e., the blue component, is relatively small compared to the R and G color components. In other words, the frequency of appearance of blue is relatively low in defective product image C.

[0068] In such a case, it is preferable that the highlighted image generating unit 102 generates highlighted image D as shown in Fig. 4 using blue as the highlighted color. The defective parts in highlighted image D are blue with an average pixel value of 255, as shown by the bar graph in the figure. This makes it possible to highlight the difference between the defective parts in the defective product image C and the highlighted image D. Note that the highlighted color may be determined based on the average pixel value of each color component in a region of the defective product image C in which the target object is captured, instead of the average pixel value of each color component in the entire defective product image C.

[0069] In this way, the generative model 113 may be constructed by learning to generate a generated image in which defective areas are highlighted using a color component that appears relatively infrequently among the color components of the input image used for learning.

[0070] Since the inspection image is an image of the same object as the input image used for learning, the frequency of occurrence of each color component in the inspection image will be the same as that of the input image used for learning. Colors of color components that appear relatively infrequently stand out in the image, and such colored areas are easy to detect. Therefore, the above configuration makes it easy to detect defective areas using the generated image.

[0071] [Examples of synthetic and enhanced image generation] Fig. 5 is a diagram showing examples of generating a composite image and an emphasized image. Fig. 5 shows two examples of generation. One is an example of generating a composite image and an emphasized image using reference shapes E1 to E3, and the other is an example of generating a composite image and an emphasized image using a defective part image F1 and a mask image F2. In both examples of generation, the original image of the composite image is a non-defective product image G in which an image g of a non-defective object is captured.

[0072] (Example of using a reference shape) When the reference shapes E1 to E3 are used, the composite image generating unit 101 first determines the composite position of the reference shapes E1 to E3 in the non-defective image G. At this time, the composite image generating unit 101 specifies the range of the image g of the object in the non-defective image G, and sets the composite position within that range. In the example of Fig. 5, the vicinity of the lower left corner of the image g of the object is set as the composite position.

[0073] Next, the composite image generating unit 101 selects one of the reference shapes E1 to E3 to be used. For example, the composite image generating unit 101 may randomly select the one to be used. Also, when the pattern of the shape of the defective portion is predetermined, the composite image generating unit 101 may select the one close to the pattern. In the example of FIG. 5, the reference shape E1 is selected. Note that only one reference shape may be prepared, in which case there is no need to select. Also, the reference shape may be any shape, for example, a circle, an ellipse, a rectangle, a polygon, a modified shape thereof, or a combination thereof. Also, for example, the shape of the defective portion extracted from the defective product image 112 may be used as the reference shape.

[0074] Next, the composite image generating unit 101 deforms the selected reference shape E1. The method of deformation is not particularly limited. For example, the positions of the vertices of the reference shape E1 may be shifted according to a predetermined rule or randomly. For example, the reference shape E1 may be subjected to a rotation transformation, and the rotation angle in this case may be set according to a predetermined rule or randomly. The reference shape E1 may be deformed by any deformation method such as affine transformation.

[0075] Finally, the composite image generating unit 101 converts the value of each pixel in the post-deformation reference shape E1 into a value corresponding to the color of the defective part. The color of the defective part may be a color extracted from the defective product image 112, or a color generated based on that color. As a result, a composite image H is generated in which a defective part image h having an appearance similar to the defective part is synthesized with a part of the non-defective product image G.

[0076] Then, the highlighted image generating unit 102 generates a highlighted image H' by highlighting a portion of the composite image H where the defective part image h is combined, to generate a defective part image h'. For example, the highlighted image generating unit 102 may generate the highlighted image H' by changing each pixel included in the defective part image h to a predetermined highlight color. As described above, the highlighted image H' can be generated from the composite image H, or can be generated from the non-defective product image G.

[0077] (Example of using defective part image and mask image) Next, an example using the defective part image F1 and the mask image F2 will be described. The defective part image F1 is obtained by cutting out the defective part of the defective product image 112. The mask image F2 is used to mask the background area of ​​the defective part in the defective part image F1 when the defective part image F1 is combined with the good product image G. In the mask image F2, the pixel values ​​of the area corresponding to the defective part in the defective part image F1 are white, and the pixel values ​​of the other areas are black.

[0078] Even when the defective part image F1 and the mask image F2 are used, the composite image generating unit 101 first determines a composite position of the defective part image F1 and the mask image F2 in the good product image G. Then, the composite image generating unit 101 performs conversion processing on the defective part image F1 and the mask image F2, and then composites them at the composite position of the good product image G. As the conversion processing, various types of conversion such as rotation conversion, shape conversion, and color conversion can be applied. As a result, a composite image J including a defective part image j is generated. Moreover, the emphasized image generating unit 102 generates an emphasized image J' by emphasizing the defective part image j in the composite image J to generate a defective part image j'. The emphasized image J' can be generated from the composite image J, or can be generated from the good product image G.

[0079] (Regarding the scope of emphasis) 5, the highlighted image generating unit 102 may generate a highlighted image by highlighting only the defective parts in the composite image. This is also true when generating a highlighted image from the defective product image 112, and the highlighted image generating unit 102 may generate a highlighted image by highlighting only the defective parts in the defective product image 112.

[0080] In other words, the emphasized image generating unit 102 may generate an emphasized image by changing only the pixel values ​​of the pixels constituting the defective portion from the original image among the pixels constituting the original image of the composite image. The manner of the change may be any one that improves the detection accuracy of the defective portion, and may be a change to an emphasized color as described above, a change to a pixel value, or the like.

[0081] Furthermore, the highlighted image generating unit 102 may generate a highlighted image by highlighting a part of a defective portion in the composite image or the defective image 112. For example, the highlighted image generating unit 102 may generate a highlighted image by changing the color of pixels constituting the outer edge of the defective portion in the composite image or the defective image 112 to a highlighted color, or by changing the pixel value of the pixel.

[0082] [How to generate a generative model] 6 is a flowchart showing an example of a method for generating the generative model 113 executed by the learning device 1. In the following, an example will be described in which a non-defective product image 111 is stored in the storage unit 11, a composite image and an emphasized image are generated from the non-defective product image 111, and the generated composite image and emphasized image are used as learning data.

[0083] In S11, the composite image generating unit 101 reads out the non-defective image 111 from the storage unit 11. Then, in S12, the composite image generating unit 101 generates a composite image by combining a partial image having an appearance similar to the defective part with a part of the non-defective image 111 read out in S11. The method of generating the composite image is as described with reference to FIG. 5, and therefore will not be described again here.

[0084] In S13, the highlighted image generating unit 102 generates a highlighted image. Specifically, the highlighted image generating unit 102 generates the highlighted image by highlighting the region where the partial image is combined in S12, among the image regions of the non-defective image 111 read in S11. The highlighting of the region is performed, for example, by changing the color of the region to a predetermined highlight color.

[0085] The processes from S11 to S13 are repeated until the number of composite images and highlighted images required for learning are obtained, and the processes from S14 onward are performed after the number of composite images and highlighted images required for learning are obtained.

[0086] In S14, the learning data acquisition unit 103 acquires learning data. Specifically, the learning data acquisition unit 103 acquires the composite image generated in S12, i.e., the image of the object having a defective portion, as an input image for machine learning. The learning data acquisition unit 103 also acquires the highlighted image generated in S13 as teacher data. That is, the learning data acquisition unit 103 acquires a set of the composite image generated in S12 and the highlighted image generated in S13 as learning data.

[0087] The learning data acquired by the learning data acquisition unit 103 is not limited to a pair of a composite image and an emphasized image. If the storage unit 11 has stored a defective product image 112, in S13 the emphasized image generation unit 102 generates an emphasized image from the defective product image 112. Then, in S14, the learning data acquisition unit 103 acquires, as learning data, a pair of the defective product image 112 and an emphasized image generated from the defective product image 112. Of course, the learning data acquisition unit 103 may acquire both the pair of a composite image and an emphasized image and the pair of the defective product image 112 and an emphasized image as learning data.

[0088] In S15, the learning unit 104 constructs a generative model by learning to generate an enhanced image when a composite image is input, using the learning data acquired in S14. The constructed generative model is stored in the storage unit 11 as the generative model 113.

[0089] Furthermore, in S14, the learning data acquisition unit 103 may also acquire the non-defective image 111 as learning data. In this case, in S15, the learning unit 104 performs learning so as to generate a generated image having similar characteristics to the input non-defective image 111 when the non-defective image 111 is input, in addition to the above-mentioned learning.

[0090] As described above, the method for generating a generative model according to this embodiment is a method for generating a generative model executed by the learning device 1, and includes a step (S14) of acquiring an image of an object having a defect as an input image for machine learning, and a step (S15) of constructing a generative model 113 by learning to generate a generated image in which the defect is emphasized when the input image is input. By using the generative model 113 generated by this generation method, it becomes possible to detect a defect part whose appearance is similar to a normal part.

[0091] [Testing method] 7 is a flowchart showing an example of an inspection method executed by the inspection device 2. In the following, a process will be described after an object for which the presence or absence of a defect is photographed and the photographed image is stored in the storage unit 21 as the inspection image 211.

[0092] In S21, the image generation unit 201 reads out the inspection image 211 from the storage unit 21. In the following S22, the image generation unit 201 generates a generated image from the inspection image 211 using the generative model 113. Specifically, the image generation unit 201 inputs the inspection image 211 to the generative model 113 to generate the generated image.

[0093] In S23, the quality determination unit 202 generates a difference image between the inspection image 211 read out in S21 and the generated image generated in S22. Specifically, the quality determination unit 202 calculates the difference between the pixel value of each pixel in the inspection image 211 and the pixel value of each corresponding pixel in the generated image.

[0094] In S24, the quality determination unit 202 determines the presence or absence of a defective portion using the difference image generated in S23. The quality determination unit 202 stores this determination result in the storage unit 21 as determination result data 213, and the process in FIG.

[0095] As described above, the inspection method according to this embodiment is an inspection method executed by the inspection device 2, and includes the steps of: generating a generated image from an inspection image 211 obtained by photographing an object for which the presence or absence of a defect is unknown, using a generative model 113 constructed by learning to generate a generated image in which an image of an object having a defect is used as an input image (S22), and judging the presence or absence of a defect in the object captured in the inspection image 211, using the generated image generated in S22. This configuration has the effect of making it possible to detect a defective portion whose appearance is similar to a normal portion.

[0096] [Modifications] The execution entity of each process described in each of the above-mentioned embodiments is arbitrary and is not limited to the above-mentioned examples. In other words, as long as each process described in each of the above-mentioned embodiments can be executed, the device configuration is arbitrary. For example, among the processes executed by the learning device 1, the process of generating learning data and the process of learning using the generated learning data may be assigned to different information processing devices and executed separately. The same applies to the inspection device 2, and the processes executed by the inspection device 2 may be assigned to multiple information processing devices and executed separately.

[0097] [Software implementation example] The functions of the learning device 1 and the inspection device 2 (hereinafter referred to as the "devices") can be realized by a program (learning program / inspection program) for causing a computer to function as the device, and a program for causing a computer to function as each control block of the device (particularly each part included in the control unit 10 or the control unit 20).

[0098] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory) as hardware for executing the program. The control device and storage device execute the program to realize each function described in each of the above embodiments.

[0099] The program may be non-transitory and may be recorded in one or more computer-readable recording media. The recording media may or may not be included in the device. In the latter case, the program may be provided to the device via any wired or wireless transmission medium.

[0100] In addition, some or all of the functions of each of the control blocks can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each of the control blocks is formed is also included in the scope of the present invention. In addition, the functions of each of the control blocks can be realized by, for example, a quantum computer.

[0101] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0102] 1 Learning device 101 Composite image generation unit 102 Enhanced image generation unit 103 Learning data acquisition unit 104 Learning Department 113 Generative Model 2. Inspection equipment 201 Image Generation Unit 202 Good / bad judgement section 211 Inspection images

Claims

1. an image generation unit that generates a generated image from an inspection image of an object for which the presence or absence of a defect is unknown, using a generation model constructed by learning so that, when an image of an object having a defect is used as an input image, a generated image that emphasizes the defect is generated, and, when an image of an object having no defect is used as the input image, a generated image having similar features to the input image is generated; An inspection apparatus comprising: a pass / fail judgment unit that generates a difference image between the generated image generated by the image generation unit and the inspection image, and uses the generated difference image to judge whether or not there are any defective parts in the object shown in the inspection image.

2. The inspection device of claim 1, wherein the generative model is constructed by learning to generate the generated image in which defective areas are highlighted using a color component that appears relatively infrequently among the color components of the input image used for learning.

3. a learning data acquisition unit that acquires an image of an object having a defect and an image of an object having no defect as input images for machine learning; and a learning unit that constructs a generative model by learning to generate a generated image that emphasizes the defective part when the input image of an object having a defective part is input, and to generate a generated image that has similar characteristics to the input image when the input image of an object having no defective part is input.

4. a composite image generating unit that generates a composite image by combining a partial image having an appearance similar to the defective portion with a portion of an image of an object having no defective portion; an emphasis image generating unit that generates an emphasis image by emphasizing the portion of the composite image, The learning device according to claim 3 , wherein the learning data acquisition unit acquires the composite image as the input image of an object having a defect, and acquires the highlighted image as teacher data.

5. An inspection method performed by an inspection device, comprising: a step of generating a generated image from an inspection image of an object for which the presence or absence of a defect is unknown, using a generative model constructed by learning to generate a generated image that emphasizes the defect when an image of an object having a defect is used as an input image, and to generate a generated image having similar features to the input image when an image of an object having no defect is used as the input image; generating a difference image between the generated image and the inspection image, and using the generated difference image to determine whether or not there are any defective portions in the object shown in the inspection image.

6. A method for generating a generative model executed by a learning device, comprising: A step of acquiring an image of an object having a defect and an image of an object not having a defect as input images for machine learning; and constructing a generative model by learning to generate a generated image that emphasizes the defective part when the input image of an object having a defective part is input, and to generate a generated image that has similar features to the input image when the input image of an object having no defective part is input.

7. 2. An inspection program for causing a computer to function as the inspection apparatus according to claim 1, the inspection program causing a computer to function as the image generating section and the quality determining section.

8. 4. A learning program for causing a computer to function as the learning device according to claim 3, the learning program causing a computer to function as the learning data acquisition unit and the learning unit.

Images