Image generation device, image generation method, and program

Abstract

Provided is an image generation device for generating a defective product image, the image generation device comprising: a non-defective product image acquisition unit that acquires a non-defective product image; a defective product information acquisition unit that acquires defective product information; a property designation unit that designates a defective product property for designating a state of a defective portion in a defective product image to be generated and a range of a property value of the defective product property; and a generation unit that generates a defective product image having a defective portion according to the range of the property value on the basis of the non-defective product image and the defective product information.

Description

Image generation device, image generation method, and program

[0001] The present invention relates to an image generating device, an image generating method, and a program.

[0002] A data generating device that generates learning data from a composite image that includes a detection target portion that may actually occur in an inspection object is known (see, for example, Patent Document 1). Problem to be solved

[0003] An image generating device for generating defective product images more efficiently is provided. General disclosure

[0004] In a first aspect of the present invention, there is provided an image generation device for generating defective product images, comprising: a good product image acquisition unit that acquires good product images; a defective product information acquisition unit that acquires defective product information; a defective product property that specifies the state of defective parts in the defective product image to be generated; a property specification unit that specifies a range of property values ​​for the defective product property; and a generation unit that generates a defective product image having defective parts according to the range of property values ​​based on the good product image and the defective product information.

[0005] The image generating device may further include a property selecting unit that selects a plurality of property values ​​from the range of property values, and the generating unit may generate a plurality of defective product images by generating a defective product image for each of the plurality of property values ​​selected by the property selecting unit for one of the non-defective product images.

[0006] In any of the image generating devices described above, the property selecting unit may select a combination of property values ​​by selecting property values ​​from a plurality of defective product properties, and the generating unit may generate the defective product image according to the combination of property values.

[0007] Any of the image generating devices described above may include a number specifying unit that specifies the number of defective product images to be generated by the generating unit. The property selecting unit may repeat the selection of the combinations of property values ​​a number of times corresponding to the number of images. The generating unit may generate the number of defective product images based on the number of combinations of property values ​​corresponding to the number of images.

[0008] In any of the image generating devices described above, the property specifying section may include a defect position specifying section that specifies a range of property values ​​of the defective product property that indicates the position of the defect portion to be generated.

[0009] In any of the image generating devices described above, the defect position designation section may have an edge designation section that designates an edge of the non-defective image as the range of the property value.

[0010] In any of the image generating devices described above, the property specifying section may include a defect type specifying section that specifies a range of property values ​​of the defective product property that indicates the type of the defect location to be generated.

[0011] In any of the image generating devices described above, the property specifying section may specify a maximum value and a minimum value of the property value as the range of the property value.

[0012] In any of the image generating devices described above, the property specifying section may specify a range of the property value of the defective product property by setting a predetermined number of divisions or division width for the property value.

[0013] Any of the image generating devices described above may include a model storage unit for storing a plurality of predetermined image generators, and an image generator designation unit for designating an image generator from the plurality of image generators to generate the defective product image.

[0014] In any of the image generating devices described above, the model storage unit may include a first image generation model that has not been individually trained for the defective portion of the defective product information, and a second image generation model that has been individually trained for the defective portion of the defective product information.

[0015] In any of the image generating devices described above, the non-defective image acquisition unit may include a non-defective data set storage unit that stores a plurality of non-defective data sets each including the non-defective image, and a non-defective data set selection unit that selects an arbitrary non-defective data set from the plurality of non-defective data sets.

[0016] In any of the image generating devices described above, the defective product information acquisition unit may include a defective product dataset storage unit that stores a plurality of defective product datasets each containing the defective product information, and a defective product dataset selection unit that selects an arbitrary defective product dataset from the plurality of defective product datasets.

[0017] Any of the image generating devices may include a failure mode selection unit that selects a failure mode to be generated from the defective product information, and the generation unit may generate the defective product image having the failure mode selected by the failure mode selection unit.

[0018] Any of the image generating devices described above may include a display unit that displays the defective product image generated by the generation unit, and the display unit may display image generation properties that are properties used to generate the defective product image.

[0019] Any of the image generating devices described above may further include an evaluation unit for evaluating the defective product image generated by the generation unit, and a property update unit for changing the range of the property value based on the evaluation result by the evaluation unit.

[0020] Any of the image generating devices described above may include a determining unit that determines the reliability of the defective product image generated by the generating unit.

[0021] Any of the image generating devices described above may include a generative model construction unit that causes the second image generation model to learn defective parts of the defective product information.

[0022] In a second aspect of the present invention, there is provided an image generation method for generating a defective product image, the image generation method comprising the steps of: a computer acquiring a good product image; a computer acquiring defective product information; a computer specifying a defective product property for specifying the state of a defective part in the defective product image to be generated and a range of property values ​​for the defective product property; and a computer generating a defective product image having a defective part according to the range of property values ​​based on the good product image and the defective product information.

[0023] In a third aspect of the present invention, there is provided a program for causing a computer to execute the image generating method described above.

[0024] The above summary of the invention does not list all of the features of the present invention, and subcombinations of these features may also be inventions.

[0025] 1 shows an overview of the configuration of the image generating device 100. An example of a non-defective product image acquisition unit 10 and a defective product information acquisition unit 20 is shown. An example of the configuration of a property designation unit 30 is shown. A modified example of the image generating device 100 is shown. An example of an image generating method by the image generating device 100 is shown. A diagram for explaining a defective product image generated by the image generating device 100. A diagram for explaining a defective product image synthesized by an image generating device of a comparative example. An example of a selection screen for an image generator and a dataset. An example of a property designation screen for designating a defective product property Pd. An example of a property designation screen for designating a defective product property Pd. An example of a number designation screen for designating the number of defective product images to be generated. An example of an annotation screen for annotating a defective point Rd. An example of a defect position designation screen for designating the position of the defective point Rd. An example of a defect position designation screen for designating the position of the defective point Rd. An example of a defect position designation screen for designating the position of the defective point Rd. An example of a dataset management screen. An example of a confirmation screen for generated defective product images. 22 illustrates an example computer 2200 in which aspects of the present invention may be embodied in whole or in part.

[0026] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0027] 1A shows an overview of the configuration of an image generating device 100. The image generating device 100 of this example includes a non-defective product image acquiring unit 10, a defective product information acquiring unit 20, a property specifying unit 30, a generating unit 40, and a model storing unit 50. The image generating device 100 may also include a defect mode selecting unit 60, a display unit 70, an evaluating unit 80, and a determining unit 85.

[0028] The image generating device 100 generates a defective product image in which defective areas Rd are assigned to the product to be inspected based on a predetermined non-defective product image Ig and defective product information Id of the product to be inspected. The product to be inspected may be an industrial product, a food product, or a household item. However, the types of the product to be inspected are not limited to these. In one example, the product to be inspected is a metal product, an automobile part, or a ceramic product.

[0029] The defective portion Rd may be a defect such as a flaw, a scratch, a dent, a distortion, noise, a crack, or an attachment. The image generating device 100 may generate a defective product image having one defective portion Rd, or may generate a defective product image having multiple defective portions Rd.

[0030] The non-defective product image acquisition unit 10 acquires a non-defective product image Ig. The non-defective product image acquisition unit 10 may acquire a non-defective product data set consisting of a plurality of non-defective product images as the non-defective product images Ig. The non-defective product image acquisition unit 10 may store the acquired non-defective product images Ig. The non-defective product image acquisition unit 10 may output the non-defective product images Ig to the generation unit 40.

[0031] The defective product information acquisition unit 20 acquires the defective product information Id. The defective product information acquisition unit 20 may store the acquired defective product information Id. The defective product information acquisition unit 20 may output the defective product information Id to the generation unit 40.

[0032] The defective product information Id may include a dataset of defective product images, text indicating the name or characteristics of the defective portion Rd, and information regarding the failure mode. The text of the defective product information Id may include text describing the defective portion Rd, such as a "deep scratch," "distorted hole," or "dull stain." The defective product information Id may also be a mapping image obtained by image conversion of the defective image.

[0033] The property specification unit 30 specifies a defective product property Pd for specifying the state of the defective portion Rd in the defective product image to be generated. The property specification unit 30 specifies a range of the property value Vp of the defective product property Pd. There are no particular limitations on the method used by the property specification unit 30 to specify the range of the property value Vp of the defective product property Pd. Note that the term "range of the defective product property Pd" may refer to the range of the property value Vp of the defective product property Pd. Similarly, the term "range of the property value Vp" may refer to the range of the property value Vp of the defective product property Pd.

[0034] The state of the defective portion Rd may be the position or shape of the defective portion Rd. The position of the defective portion Rd may be the position where the defective portion Rd is provided in the non-defective image Ig, and the method of specifying the position may be arbitrary. The shape of the defective portion Rd may include at least one of the area, major axis, angle, degree of distortion, shape, scratch depth, color, brightness, contrast, or texture information of the defective portion Rd. The shape of the defective portion Rd may be specified by the magnification of the defective portion Rd, the number of area pixels, the number of major axis pixels, or the like when applying the defective portion Rd to the non-defective image Ig.

[0035] The range of the property value Vp may be a range in which the value of the property value Vp changes continuously. The property specification unit 30 may specify a maximum value and a minimum value of the property value Vp as the range of the property value Vp. For example, the range of the property value Vp is a continuous range, such as from a minimum value of 10 to a maximum value of 30.

[0036] The range of the property value Vp may be a range in which the value of the property value Vp changes discretely. The property specification unit 30 may specify the range of the property value Vp by setting a predetermined number of divisions or division width for the property value Vp of the defective property Pd. The property specification unit 30 may specify the range of the property value Vp by setting the number of divisions or division width after setting maximum and minimum values ​​for the property value Vp. The property specification unit 30 may specify the range of the property value Vp using a grid. The property specification unit 30 may equally divide the predetermined range of the property value Vp by an arbitrary number of divisions and set discrete values ​​corresponding to the number of divisions as the range of the property value Vp. The property specification unit 30 may also equally divide the predetermined range of the property value Vp by an arbitrary division width and set discrete values ​​corresponding to the division width as the range of the property value Vp. For example, the range of the property value Vp is a discontinuous range such as 10, 20, and 30. The values ​​of the property value Vp may or may not be equally spaced.

[0037] The property specification unit 30 may specify the range of the property values ​​Vp using a list of the property values ​​Vp. The property specification unit 30 may specify the property values ​​Vp of the major axis using a list, or may specify the coordinates using a list.

[0038] The property specification unit 30 may display a specification screen for specifying the range of the defective property Pd and allow the user to select it.The property specification unit 30 may also display a slide bar or the like for specifying the range of the defective property Pd and allow the user to select it.

[0039] The generating unit 40 generates a defective product image having a defective location Rd according to a range of the property value Vp based on the non-defective product image Ig and the defective product information Id. The generating unit 40 may select an arbitrary property value Vp from the range of the defective product property Pd specified by the property specifying unit 30, and generate a defective product image according to the selected property value Vp. The generating unit 40 may generate the defective product image using an image generator stored in the model storage unit 50.

[0040] The model storage unit 50 stores one or more predetermined image generators. The generation unit 40 may select any image generator stored in the model storage unit 50 to generate a defective product image. The model storage unit 50 may store a pre-trained image generator such as a generative adversarial network (GAN) or a diffusion model.

[0041] The failure mode selection unit 60 selects a failure mode to be generated from the defective product information Id. The failure mode selection unit 60 may select one failure mode or multiple failure modes. The failure mode of the defective portion Rd may refer to at least one of defects, scratches, dents, distortions, noise, cracks, and deposits in the defective portion Rd. By selecting the failure mode of the defective portion Rd, it is possible to preferentially generate a failure portion Rd of a desired failure mode and avoid generating a failure portion Rd of an unnecessary failure mode. The generation unit 40 may generate a defective product image having the failure mode selected by the failure mode selection unit 60.

[0042] The display unit 70 displays the defective product image generated by the generation unit 40. The display unit 70 may display image generation properties, which are properties used to generate the defective product image. The display unit 70 may display a property specification screen for specifying the range of the defective product property Pd, or a defect mode selection screen for selecting a defect mode. The display unit 70 may display various display screens, which will be described later.

[0043] The evaluation unit 80 evaluates the defective product images generated by the generation unit 40. The evaluation unit 80 may display the evaluation results of the defective product images to the user. The evaluation unit 80 may feed back the evaluation results of the defective product images to the generation unit 40 to generate defective product images. The evaluation unit 80 may omit defective product images with poor evaluation results from the next generated image, or may learn defective product images with weak defective product properties Pd from the evaluation results to overcome the weak points. The evaluation unit 80 may evaluate the defective product images generated by the generation unit 40 using an evaluation model. The evaluation model of the evaluation unit 80 may be an identification model using machine learning or an identification model for rule-based identification.

[0044] The evaluation unit 80 may take as the evaluation result information input by the user in response to the defective product image generated by the generation unit 40. The evaluation unit 80 may include as the evaluation result a comment added by the user.

[0045] The evaluation unit 80 may visualize and display the evaluation results of the generated defective product images in the form of a graph or the like. The evaluation unit 80 may visualize and display the defective product properties Pd used when generating the defective product images. The evaluation unit 80 may visualize the distribution of each defective product property Pd of the generated multiple defective product images. The evaluation unit 80 may visualize the statistical values ​​of the defective product properties Pd in ​​a histogram. For example, the evaluation unit 80 may visualize the frequency of each defective product property Pd when the horizontal axis represents the size of the defective portion Rd. The evaluation unit 80 may display a scatter plot with the size of the defective portion Rd on the horizontal axis and the color of the defective portion Rd on the vertical axis.

[0046] The evaluation unit 80 may train an evaluation model for evaluating defective product images based on the good product images Ig acquired by the good product image acquisition unit 10 and the defective product information Id acquired by the defective product information acquisition unit 20. An evaluation model for evaluating defective product images may be trained using the evaluation results of the generated defective product images. The evaluation results may include a false detection rate or a non-defective product oversight rate of defective parts Rd as an identification rate.

[0047] The evaluation unit 80 may use the evaluation results to cause the image generator to learn. The evaluation unit 80 may cause the generation unit 40 to generate defective product images to reinforce defective product properties Pd that are weak points in the evaluation model. That is, when the classification rate of a specific defective product property Pd is low, the evaluation unit 80 may cause the generation unit 40 to generate defective product images having defective product properties Pd with a low classification rate. When the classification rate of a specific color is low, the evaluation unit 80 may generate defective product images having that color and cause the image generator to perform additional learning. The image generation device 100 may automatically perform this evaluation and learning loop.

[0048] The determination unit 85 determines the reliability of the defective product image generated by the generation unit 40. The determination unit 85 may determine the reliability by determining whether the generated defective product image is close to the actual defect location Rd. By determining the reliability of the defective product image, the naturalness of the defective product image determined by the image classifier can be indicated.

[0049] The image generation device 100 of this example generates diverse and comprehensive defective product images, thereby improving the efficiency of generating defective product images. The image generation device 100 of this example can freely generate defective product images that are difficult to collect at the manufacturing site. As a result, the image generation device 100 can achieve high-quality defective product detection early, even for new products, allowing for early start-up of the production line. The image generation device 100 of this example can generate defective product images based on defective product images of products different from the product being inspected. As a result, the image generation device 100 can generate defective product images based on data on newly developed products and similar products from the past. The image generation device 100 of this example can improve the accuracy of defective product detection and contribute to reducing false positives.

[0050] 1B shows an example of a good product image acquisition unit 10 and a defective product information acquisition unit 20. The good product image acquisition unit 10 of this example has a good product dataset storage unit 12 and a good product dataset selection unit 14. The defective product information acquisition unit 20 of this example has a defective product dataset storage unit 22 and a defective product dataset selection unit 24.

[0051] The good-quality data set storage unit 12 stores a plurality of good-quality data sets, each of which contains a good-quality image Ig. The good-quality data set selection unit 14 selects an arbitrary good-quality data set from the plurality of good-quality data sets. The good-quality data set selection unit 14 may cause the display unit 70 to display a screen for selecting a good-quality data set, allowing the user to select an arbitrary good-quality data set from the plurality of good-quality data sets.

[0052] The defective product dataset storage unit 22 stores a plurality of defective product datasets, each containing defective product information Id. The defective product dataset storage unit 22 may store a defective product image generated by the generation unit 40 as the defective product information Id. The defective product dataset storage unit 22 may store a defect location Rd of a product different from the product to be inspected as the defective product information Id. The defective product dataset selection unit 24 selects an arbitrary defective product dataset from the plurality of defective product datasets. The defective product dataset selection unit 24 may display a screen for selecting a defective product dataset on the display unit 70, allowing the user to select an arbitrary defective product dataset from the plurality of defective product datasets.

[0053] 1C shows an example of the configuration of the property specification unit 30. The property specification unit 30 of this example has a property selection unit 31, a defect position specification unit 32, a defect type specification unit 36, and a property update unit 38. The defect position specification unit 32 of this example has an edge specification unit 34.

[0054] The property selection unit 31 selects an arbitrary property value Vp from the range of the defective product property Pd specified by the property specification unit 30. The property selection unit 31 may randomly select the property value Vp from the range of the defective product property Pd. The property selection unit 31 may provide the selected property value Vp to the generation unit 40. The generation unit 40 may generate a defective product image according to the property value Vp.

[0055] The property selection unit 31 may select multiple property values ​​Vp from a range of property values ​​Vp. The property selection unit 31 may provide the selected multiple property values ​​Vp to the generation unit 40. The generation unit 40 may generate multiple defective product images according to the multiple property values ​​Vp. This allows the image generation device 100 to generate multiple defective product images from a single defective product image.

[0056] The property selection unit 31 may select a combination of property values ​​Vp by selecting property values ​​Vp from each of a plurality of defective product properties Pd. The property selection unit 31 selects a combination of a first property value Vp and a second property value Vp by selecting an arbitrary first property value Vp from a range of property values ​​Vp for a first defective product property Pd (e.g., the size of a scratch) and selecting an arbitrary second property value Vp from a range of property values ​​Vp for a second defective product property Pd (e.g., the angle of a scratch). In this example, a combination of two property values ​​Vp is described, but a combination of three or more property values ​​Vp may also be selected. The property selection unit 31 may provide the selected combination of property values ​​Vp to the generation unit 40. The generation unit 40 may generate a defective product image according to the combination of property values ​​Vp. This allows the image generation device 100 to generate a defective product image by specifying a combination of multiple property values ​​Vp, such as the size and angle of a scratch.

[0057] The defect position designation unit 32 designates a range of property values ​​Vp of the defective product property Pd indicating the position of the defect point Rd to be generated. The defect position designation unit 32 may designate the position of the defect point Rd in a paint area, may designate points manually, or may designate the position of the defect point Rd using a grid with any intervals. The designation of points may be performed by designating the position of the defect point Rd on an image, or may designate the position of the defect point Rd using coordinates. The defect position designation unit 32 may designate the position of the defect point Rd in two dimensions or three dimensions. The defect position designation unit 32 may include an edge designation unit 34.

[0058] The edge designation unit 34 designates the edges of the non-defective product image Ig as the range of the property value Vp. By designating the position of the defect point Rd by the edge, the edge designation unit 34 can prevent the defect point Rd from extending beyond the edge in the generated defective product image. This allows the image generating device 100 to generate a defective product image having a natural defect point Rd along the edge. The edge designation unit 34 may designate the edges of the non-defective product image Ig automatically, or may allow the user to designate them. The edge designation unit 34 may designate the edges by acquiring segment information of the non-defective product image Ig.

[0059] The defect form designation unit 36 ​​designates a range of property values ​​Vp of the defective product property Pd indicating the form of the defect point Rd to be generated. The defect form designation unit 36 ​​may designate the size, shape, angle, etc. of the defect point Rd as the form of the defect point Rd. If the defect point Rd is a scratch, the defect form designation unit 36 ​​may designate the depth of the scratch as the form of the defect point Rd. The defect form designation unit 36 ​​may designate the form of the defect point Rd as a range using a numerical value or as a list of numerical values.

[0060] The property update unit 38 changes the range of the property value Vp based on the evaluation result by the evaluation unit 80. By updating the range of the property value Vp, the property update unit 38 can cause the generation unit 40 to generate a more appropriate defective product image. The property update unit 38 may suggest to the user to change the range of the defective product property Pd, thereby causing the user to change the range of the defective product property Pd.

[0061] 2 shows a modified example of the image generating device 100. In this example, differences from the image generating device 100 of FIG. 1A will be described. Other points may be the same as those of the image generating device 100 of FIG. 1A. The generating unit 40 of this example has an image generator specifying unit 42 and a number specifying unit 45. The image generating device 100 of this example also includes a generative model building unit 90.

[0062] The image generator designation unit 42 designates an image generator for generating a defective product image from among a plurality of image generators. The image generator designation unit 42 may display a selection screen for a plurality of image generators on the display unit 70 to allow the user to select an image generator. In this example, the image generator designation unit 42 designates the first image generation model 51 or the second image generation model 52 from the model storage unit 50.

[0063] The number designation unit 45 designates the number of defective product images to be generated by the generation unit 40. The generation unit 40 generates the number of defective product images by using a number of property values ​​Vp corresponding to the number designated by the number designation unit 45 from the range of the defective product property Pd. For example, when the number designation unit 45 designates n images (n is a natural number) as the number of images, the generation unit 40 generates n defective product images by using the n property values ​​Vp designated from the range of the defective product property Pd.

[0064] The generation unit 40 may generate the number of defective product images specified by the number-of-images specifying unit 45 based on the number of combinations of property values ​​Vp corresponding to the number specified by the number-of-images specifying unit 45. In this case, the property selection unit 31 may repeat the selection of combinations of property values ​​Vp a number of times corresponding to the number of images. That is, the property selection unit 31 may repeat the process of selecting a first property value Vp from the range of the first defective product property Pd, selecting a second property value Vp from the range of the second defective product property Pd, and selecting a combination of the first property value Vp and the second property value Vp n times. In this example, a combination of two property values ​​Vp has been described, but a combination of three or more property values ​​Vp may also be selected.

[0065] The model storage unit 50 of this example stores a first image generation model 51 and a second image generation model 52. However, the types and number of image generators stored in the model storage unit 50 are not limited to this.

[0066] The first image generation model 51 is an image generator that can be used universally for any product to be inspected. The first image generation model 51 may be an image generator that has not been individually trained on the defective portion Rd of the defective product information Id. Use of the first image generation model 51 makes it easier to quickly mass-produce data without individual training. The first image generation model 51 is an example of a standard model, which will be described later. For example, if the defective portion Rd is a scratch on a round washer, the first image generation model 51 is an image generator that has not been trained using the same scratch on the round washer as the product to be inspected.

[0067] The second image generation model 52 is an image generator that has been individually trained on the defective portion Rd of the defective product information Id. For example, if the defective portion Rd is a scratch on a round washer, the second image generation model 52 is an image generator that has been trained using the defective product information Id related to the scratch on the round washer. The second image generation model 52 may be an image generator that is the same as the first image generation model 51 but that has been individually trained on the defective portion Rd. The second image generation model 52 is more likely to generate higher quality data than the first image generation model 51. The second image generation model 52 is an example of a custom model, which will be described later.

[0068] The generative model construction unit 90 causes the second image generation model 52 to learn the defective location Rd of the defective product information Id. The generative model construction unit 90 may select a dataset stored in the good product dataset storage unit 12 or the defective product dataset storage unit 22 and cause the second image generation model 52 to learn the defective location Rd. The generative model construction unit 90 may cause the second image generation model 52 to learn a dataset that corresponds to the range of the property value Vp specified by the property specification unit 30, in particular. Learning the range of the property value Vp improves the quality of the defective product image. The generative model construction unit 90 may cause the second image generation model 52 to learn a defective product dataset that falls within a range of the property value Vp for which learning is insufficient, depending on the evaluation results of the generated defective product image.

[0069] 3 shows an example of an image generation method using image generation device 100. In step S100, a computer acquires a non-defective product image Ig. In step S102, the computer acquires defective product information Id. In this example, the computer acquires the non-defective product image Ig before the defective product information Id, but the non-defective product image Ig may be acquired after acquiring the defective product information Id.

[0070] In step S104, the computer specifies a defective product property Pd for specifying the state of the defective portion Rd in the defective product image to be generated, and a range of the property value Vp of the defective product property Pd. In step S106, the computer generates a defective product image having a defective portion Rd according to the range of the property value Vp, based on the good product image Ig and the defective product information Id.

[0071] FIG. 4A is a diagram illustrating a defective product image generated by the image generating device 100. In this example, the generating unit 40 generates multiple defective product images based on a non-defective product image Ig and defective product information Id. The generating unit 40 may generate multiple defective product images by generating a defective product image for each of multiple property values ​​Vp selected by the property selecting unit 31 for one non-defective product image Ig. This allows the image generating device 100 to generate multiple defective product images even when only one or a small number of defective product images exist as the defective product information Id. Therefore, the image generating device 100 can easily generate high-quality defective product images.

[0072] FIG. 4B is a diagram illustrating a defective product image synthesized by the image generating device of the comparative example. The image generating device of the comparative example generates a defective product image by synthesizing one good product image and one defective product image. With the image generating device of the comparative example, a large number of defective product images may be required to synthesize a large number of defective product images. Even with the image generating device of the comparative example, it is possible to increase the amount of data, but it requires a lot of effort to generate the defective product images and it is difficult to improve the quality. The defective product images synthesized by the image generating device of the comparative example may have an unnatural appearance, with defects simply pasted on.

[0073] 5 shows an example of a screen for selecting an image generator and a data set. The image generating device 100 of this example displays a display screen 110 and a display screen 112.

[0074] The display screen 110 is a screen for selecting an image generator. The image generating device 100 may use the display screen 110 to allow a user to select an image generator to be used by the generation unit 40. The display screen 110 in this example displays a first image generation model 51 and a second image generation model 52 in a selectable manner. The display screen 110 may display which data set was used to train the second image generation model 52. The display screen 110 may allow a user to select a second image generation model 52 that is suitable for the defective product image to be generated.

[0075] The display screen 112 is a screen for selecting a dataset to be used to generate a defective product image. The image generation device 100 may use the display screen 112 to allow a user to select a dataset. In this example, the display screen 112 displays both a good product dataset and a defective product dataset. The image generation device 100 may allow a user to select only one of the good product dataset and the defective product dataset, depending on the type of image generator selected on the display screen 110. When the first image generation model 51 is selected on the display screen 110, the user may be prompted to select both the good product dataset and the defective product dataset, whereas when the second image generation model 52 is selected, the user may be prompted to select only the good product dataset.

[0076] The display screen 112 may display the defective data sets in descending order of priority. The priority of the defective data sets may be determined based on whether or not they have been used in training the second image generation model 52. The priority of the defective data sets may be determined based on how frequently the defective data sets are used. The display screen 112 may display only the data sets used in training the second image generation model 52. The display screen 112 may highlight or otherwise emphasize the data sets used in training the second image generation model 52.

[0077] 6A shows an example of a property specification screen for specifying defective property Pd. Image generation device 100 of this example displays a property specification screen for specifying the position of defect point Rd and the defect mode. Image generation device 100 of this example displays display screen 120 and display screen 122. Image generation device 100 may display display screen 120 and display screen 122 simultaneously or on separate screens.

[0078] The display screen 120 shows a method for specifying the position of the defective portion Rd in the non-defective image Ig. The display screen 120 may display, as a method for specifying the position of the defective portion Rd, specification by a painted area of ​​the non-defective image Ig, specification by an edge, specification by a point, specification by a grid, specification by coordinate data, or the like. The display screen 120 may allow the user to select a method for specifying the position of the defective portion Rd depending on the defective image to be generated.

[0079] For example, when the method for specifying the position of the defect point Rd is by grid specification, the display screen 120 displays a screen for specifying the range of the horizontal and vertical widths of the grid. The display screen 120 may display a screen for inputting the number of divisions in the horizontal direction and the number of divisions in the vertical direction, and display a grid corresponding to each number of divisions. The display screen 120 in this example displays a screen for inputting the number of divisions, but may also display a screen for inputting the division width. The display screen 120 in this example displays a preview screen for specification by grid.

[0080] The image generating device 100 of this example can specify a range of positions of the defective points Rd in the non-defective image Ig in accordance with the method for specifying the positions of the defective points Rd input to the display screen 120, and generate a defective product image in which the defective points Rd are generated within the specified range of positions. The image generating device 100 may generate a defective product image in which the defective points Rd are generated at random positions within the range specified by the selected method for specifying the positions.

[0081] The display screen 122 is a screen for selecting a failure mode. When there are multiple failure modes, the display screen 122 may allow the user to select all of the failure modes, or may allow the user to select one or more arbitrary failure modes. The display screen 122 may display the types of failure modes as images or as text.

[0082] 6B shows an example of a property specification screen for specifying defective property Pd. Image generation device 100 of this example displays a property specification screen for specifying the shape of defect location Rd. The property specification screen of this example is displayed after the property specification screen of FIG. 6A , but the display order is not limited to this. Image generation device 100 of this example displays display screen 130, display screen 132, display screen 134, and display screen 136. Image generation device 100 may display display screen 130, display screen 132, display screen 134, and display screen 136 simultaneously, or may display each of them on a separate screen.

[0083] The display screen 130 displays a screen for specifying the size of the defect location Rd. The display screen 130 may display a method for specifying the size of the defect location Rd, or may display a range of sizes of the defect location Rd. The display screen 130 of this example displays a screen for selecting one of the following methods for specifying the size of the defect location Rd: specifying by magnification, specifying by major axis (number of pixels), or specifying by area (number of pixels). The display screen 130 of this example displays a screen for inputting minimum and maximum values ​​of the number of pixels [px] for specifying the size of the defect location Rd. The display screen 130 may display images of the defect location Rd with the original number of pixels, the defect location Rd with the minimum number of pixels, and the defect location Rd with the maximum number of pixels.

[0084] The display screen 132 displays a screen for specifying the angle of the defect point Rd. The display screen 132 may display the range of the angle of the defect point Rd. In this example, the display screen 132 displays a screen for inputting the minimum and maximum values ​​of the angle [°] of the defect point Rd. The display screen 132 may display images of the defect point Rd at the original angle, the defect point Rd at the minimum angle, and the defect point Rd at the maximum angle.

[0085] The display screen 134 displays a screen for specifying the degree of distortion of the defective portion Rd. The display screen 134 may display a range of the degree of distortion of the defective portion Rd. In this example, the display screen 134 displays a screen for inputting the minimum and maximum values ​​of the degree of distortion [°] of the defective portion Rd. The display screen 134 may display images of the defective portion Rd with the original degree of distortion, the defective portion Rd with the minimum degree of distortion, and the defective portion Rd with the maximum degree of distortion.

[0086] The display screen 136 displays a screen for specifying the shape of the defective portion Rd. In this example, the display screen 136 displays a screen for selecting manual specification or random specification as a method for specifying the shape of the defective portion Rd. The display screen 136 may display candidate shapes of the defective portion Rd based on the defective portion Rd present in the defective product information Id. The candidate shapes of the defective portion Rd may be prepared in advance as templates. The candidate shapes of the defective portion Rd may be added by the user.

[0087] The image generating device 100 of this example displays the defect points Rd that can be generated according to the range of the defect property Pd each time, so that the range of the defect property Pd can be adjusted while checking the defect points Rd that can be generated. This makes it possible to know how the generated defect points Rd will change according to a change in the property value Vp of the defect property Pd.

[0088] 7 shows an example of a number specification screen for specifying the number of defective product images to be generated. Image generation device 100 in this example displays display screen 140 and display screen 142. The number specification screen in this example is displayed after the property specification screen in FIG. 6B, but the display order is not limited to this.

[0089] The display screen 140 may display a preview of the defective product image generated according to the specified conditions. The user may check the preview on the display screen 140 and change the specified range of the defective product property Pd.

[0090] The display screen 142 displays a screen for specifying the number of defective product images to be generated. The display screen 142 may display a name input field for inputting the name of a data set including the specified number of generated defective product images.

[0091] 8 shows an example of an annotation screen for annotating the defect location Rd. The annotation of the defect location Rd may be performed after selecting the defect data set to be used for generation, or may be performed in advance when the defect data set is stored. The image generating device 100 of this example displays a display screen 150.

[0092] The display screen 150 displays an annotation screen for annotating a defective portion Rd in a defective product image. The display screen 150 allows the user to select a defect mode for the defective portion Rd while annotating the defective portion Rd. That is, the user may associate the defect mode with the image of the defective portion Rd by selecting the defect mode for the defective portion Rd and then annotating the defective portion Rd. While the display screen 150 in this example displays scratches, dents, or distortions as defect modes, additional defect modes may be added as appropriate. The user may annotate the defective portion Rd in the defective product image by painting, etc. The image generating device 100 may store the annotated defect portion Rd as defective product information Id. The display screen 150 may display a comment input field that allows the user to input a comment indicating the characteristics of the scratches.

[0093] The image generating device 100 may annotate the defect points Rd by color-coding them for each defect mode. The image generating device 100 may acquire information on the position or shape of the defect points Rd for each defect mode. The image generating device 100 may automatically annotate the defect points Rd in the defective product image.

[0094] 9A shows an example of a defect position designation screen for designating the position of the defect point Rd. The image generating device 100 of this example displays the display screen 160.

[0095] The display screen 160 displays a non-defective product image Ig in which an area where a defect point Rd is to be generated is painted. The area where a defect point Rd is to be generated may be selected using an input device such as a mouse. The area where a defect point Rd is to be generated may be specified one by one by the user using a paint tool, or may be specified automatically based on past history or the like. The area where a defect point Rd is to be generated may be specified semi-automatically, such as by having the user specify a rough position using text information and then having the computer automatically specify the area.

[0096] 9B shows an example of a defect position designation screen for designating the position of the defect point Rd. The image generating device 100 of this example displays the display screen 162.

[0097] The display screen 162 displays a non-defective product image Ig with the periphery of the edge painted as the region where the defective portion Rd is to be generated. The region where the defective portion Rd is to be generated may be selected using an input device such as a mouse. The image generating device 100 may automatically specify the edge of the non-defective product image Ig.

[0098] 9C shows an example of a defect position designation screen for designating the position of the defect point Rd. Image generating device 100 of this example displays display screen 164.

[0099] The display screen 164 displays the non-defective image Ig with designated points as the area where the defect point Rd is to be generated. The area where the defect point Rd is to be generated may be selected by an input device such as a mouse.

[0100] 10 shows an example of a data set management screen. Image generating device 100 of this example displays display screen 170.

[0101] The display screen 170 displays multiple data sets and details of each data set. The display screen 170 may display a screen for specifying conditions to search for a specific data set from the multiple data sets. The display screen 170 may display a good product data set and a defective product data set. Managing the data sets makes it easier to apply more appropriate conditions to the defect location Rd, and makes it easier to generate high-quality defective product images.

[0102] 11 shows an example of a confirmation screen for the generated defective product image. The image generating device 100 of this example displays a display screen 180.

[0103] The display screen 180 may display which property value Vp within the range of the defective property Pd was used to generate the defective image. The display screen 180 may display multiple property values ​​Vp, or may display combinations of the property values ​​Vp.

[0104] The display screen 180 may display a screen for inputting the evaluation results of the defective product image. The display screen 180 may display a screen for inputting whether the generated defective product image is natural or unnatural. By inputting the evaluation results of the generated defective product image by the user, unnatural defective product images can be omitted the next time they are generated. The display screen 180 may display a field for selecting whether to learn the defective product property Pd that is a weakness. The display screen 180 may display a screen for inputting a user name. The display screen 180 may display the reliability as a judgment result of the generated defective product image.

[0105] 12 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0106] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0107] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0108] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0109] ROM 2230 stores therein a boot program or the like that is executed by computer 2200 upon activation, and / or programs that depend on the hardware of computer 2200. I / O chip 2240 may also connect various I / O units to I / O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0110] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0111] For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0112] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and may perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0113] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0114] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0115] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0116] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0117] 10... Good product image acquisition unit, 12... Good product data set storage unit, 14... Good product data set selection unit, 20... Defective product information acquisition unit, 22... Defective product data set storage unit, 24... Defective product data set selection unit, 30... Property designation unit, 31... Property selection unit, 32... Defect position designation unit, 34... Edge designation unit, 36... Defect form designation unit, 38... Property update unit, 40... Generation unit, 42... Image generator designation unit, 45... Number designation unit, 50... Model storage unit, 51... First image generation model, 52... Second image generation model, 60... Defect mode selection unit, 70... Display unit, 80... Evaluation unit, 85... Judgment unit, 90... Generative model construction unit, 100... Image generation device, 110... Display screen, 112... Display screen, 120... Display screen, 122 ...Display screen, 130...Display screen, 132...Display screen, 134...Display screen, 136...Display screen, 140...Display screen, 142...Display screen, 150...Display screen, 160...Display screen, 162...Display screen, 164...Display screen, 170...Display screen, 180...Display screen, 2200...Computer, 2201...DVD-ROM, 2210...Host controller, 2212...CPU, 2214...RAM, 2216...Graphics controller, 2218...Display device, 2220...Input / output controller, 2222...Communication interface, 2224...Hard disk drive, 2226...DVD-ROM drive, 2230...ROM, 2240...Input / output chip, 2242...Keyboard

Claims

1. An image generating device for generating defective product images, comprising: a good product image acquisition unit that acquires good product images; a defective product information acquisition unit that acquires defective product information; a property designation unit that designates a defective product property for designating the state of defective points in the defective product image to be generated and a property value range of the defective product property; and a generation unit that generates a defective product image having defective points according to the property value range based on the good product image and the defective product information.

2. The image generating device according to claim 1, further comprising a property selection unit that selects a plurality of property values ​​from the range of property values, and the generation unit generates a plurality of defective images for one of the good product images by generating a defective product image for each of the plurality of property values ​​selected by the property selection unit.

3. The image generating device according to claim 2, wherein the property selection unit selects a combination of property values ​​by selecting property values ​​from a plurality of defective product properties, and the generation unit generates the defective product image according to the combination of property values.

4. The image generating device according to claim 3, further comprising a number designation unit that designates the number of defective product images to be generated by the generating unit, wherein the property selection unit repeats the selection of the combination of property values ​​a number of times corresponding to the number of images, and the generating unit generates the number of defective product images based on the number of combinations of property values ​​corresponding to the number of images.

5. The image generating device according to claim 1, wherein said property designation section has a defect position designation section that designates a range of property values ​​of said defective product property indicating the position of said defective portion to be generated.

6. The image generating device according to claim 5, wherein said defect position designation section has an edge designation section that designates an edge of said non-defective image as the range of said property value.

7. The image generating device according to claim 1, wherein said property designation section has a defect form designation section that designates a range of property values ​​of said defective product property indicating the form of said defective portion to be generated.

8. The image generating device according to claim 1, wherein the property specification section specifies a maximum value and a minimum value of the property value as the range of the property value.

9. The image generating device according to claim 1, wherein the property designation section sets a predetermined number of divisions or a division width for the property value of the defective property to designate a range for the property value.

10. An image generating device as described in claim 1, comprising: a model storage unit for storing a plurality of predetermined image generators; and an image generator designation unit for designating an image generator for generating the defective product image from the plurality of image generators.

11. The image generation device according to claim 10, wherein the model storage unit includes: a first image generation model that has not been individually trained on the defective parts of the defective product information; and a second image generation model that has been individually trained on the defective parts of the defective product information.

12. The image generating device according to claim 1, wherein the good product image acquisition unit has: a good product dataset storage unit that stores a plurality of good product datasets in which the good product images are stored; and a good product dataset selection unit that selects an arbitrary good product dataset from the plurality of good product datasets.

13. The image generating device according to claim 1, wherein the defective product information acquisition unit has: a defective product dataset storage unit that stores a plurality of defective product datasets each including the defective product information; and a defective product dataset selection unit that selects an arbitrary defective product dataset from the plurality of defective product datasets.

14. The image generating device according to claim 1, further comprising a failure mode selection unit that selects a failure mode to be generated from the defective product information, and the generation unit generates the defective product image having the failure mode selected by the failure mode selection unit.

15. The image generating device according to claim 1, further comprising a display unit for displaying the defective product image generated by the generation unit, the display unit displaying image generation properties which are properties used to generate the defective product image.

16. The image generating device according to claim 1, further comprising: an evaluation unit for evaluating the defective image generated by the generation unit; and a property update unit for changing the range of the property value based on a result of the evaluation by the evaluation unit.

17. The image generating device according to claim 1, further comprising a determining unit for determining the reliability of the defective product image generated by the generating unit.

18. The image generating device according to claim 11, further comprising a generation model construction unit for causing the second image generating model to learn the defective parts of the defective product information.

19. An image generation method for generating a defective product image, comprising the steps of: a computer acquiring a good product image; a computer acquiring defective product information; a computer specifying a defective product property for specifying the state of a defective part in the defective product image to be generated and a range of property values ​​of the defective product property; and a computer generating a defective product image having a defective part according to the range of property values ​​based on the good product image and the defective product information.

20. A program for causing a computer to execute the image generating method according to claim 19.

Images