Method, program, information processing device, and learned model

Abstract

To improve determination accuracy in product inspection while reducing man-hours for model generation when a defective product image generated in a pseudo manner is used when a plurality of types of products are inspected.SOLUTION: With respect to a quality determination model 10122 that is learned using teacher data whose input is a learning image set including a defective product image I11 of a product of a type A which is a defective product, a non-defective product image I20 of the product of a type B which is a non-defective product, and a pseudo-defective product image I21 of the product of the type B, in which a pseudo-abnormal image indicating a pseudo-abnormality different from the abnormality that has occurred in the product of the type B is composited into the non-defective product image I20, and whose output is information indicating whether the product of the type B is the non-defective product or the defective product, a quality determination device 101 inputs an inspection image I22 of the product of the type B and acquires information indicating whether the product of the type B is the non-defective product or the defective product from the quality determination model 10122 as an output.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method, a program, an information processing device, and a trained model. [Background technology]

[0002] Appearance inspection is one method for inspecting the quality of a product. Appearance inspection may be performed automatically by a computer, for example, by inputting a photographed image of the product into a trained model to obtain a result. For a certain product, for example, due to a small number of products manufactured, it may not be possible to prepare a sufficient number of defective product images necessary for learning to generate a trained model. As a technology for compensating for the shortage of defective product images, Patent Document 1 describes an image data generation system capable of generating image data of various types of defective products. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-136583 A Summary of the Invention [Problem to be solved by the invention]

[0004] Generating a trained model requires many man-hours, such as collection and annotation of defective product images, learning to generate a trained model, and accuracy verification of the trained model. In particular, since collection and annotation of defective product images requires many man-hours, in addition to the fact that defective product images cannot be sufficiently prepared, collection and annotation of defective product images may not be performed from the viewpoint of reducing man-hours. In the inspection of multiple different types of products contained in a certain product, a trained model needs to be generated for each of the different types of products. Therefore, it is necessary to collect and annotate defective product images for each different type, and the man-hours further increase as the number of types increases.

[0005] Furthermore, the defective product images generated by the image data generation system described in Patent Document 1 are merely pseudo defective product images. Therefore, if pseudo defective product images are generated for each of different types of products, the generated images may include parts that differ from images containing defects that actually occurred in each of the different types of products. If learning is performed to generate a trained model using pseudo defective product images generated in this way, there is a possibility that the detection accuracy of images containing defects that actually occurred will decrease.

[0006] Therefore, the present invention aims to improve the accuracy of judgment while reducing the labor required to generate a trained model when using pseudo-generated defective product images when inspecting multiple types of products using a trained model. [Means for solving the problem]

[0007] A method according to one embodiment of the present invention includes: inputting an inspection image of a second variety of product to be inspected into a first trained model trained using teacher data, the first trained model being trained using training data that includes a training image set including a first defective image of a first variety of product that is defective, a second good image of a second variety of product that is good, and a pseudo-defective image of the second variety of product in which a pseudo-abnormal image showing a pseudo-abnormality different from an abnormality occurring in the second variety of product is superimposed on the second good image, and outputting information indicating whether the second variety of product included in the training image set is a good product or a defective product; and obtaining information indicating whether the second variety of product to be inspected is a good product or a defective product as an output from the first trained model.

[0008] A program according to another aspect of the present invention causes a computer to execute the following steps: input an inspection image of a second variety of product to be inspected into a first trained model trained using training data that receives as input a training image set including a first defective image of a first variety of product that is defective, a second good image of a second variety of product that is good, and a pseudo-defective image of the second variety of product in which a pseudo-abnormal image showing a pseudo-abnormality different from an abnormality occurring in the second variety of product is superimposed on the second good image, and outputs information indicating whether the second variety of product included in the training image set is a good product or a defective product; and obtain as output from the first trained model information indicating whether the second variety of product to be inspected is a good product or a defective product.

[0009] An information processing device according to another aspect of the present invention includes a quality assessment unit that inputs an inspection image of a second variety of product to be inspected into a first trained model trained using teacher data that receives as input a training image set including a first defective image of a first variety of product that is defective, a second good image of a second variety of product that is good, and a pseudo-defective image of the second variety of product in which a pseudo-abnormal image indicating a pseudo-abnormality different from an abnormality occurring in the second variety of product is superimposed on the second good image, and outputs information indicating whether the second variety of product included in the training image set is a good product or a defective product, and obtains as output from the first trained model information indicating whether the second variety of product to be inspected is a good product or a defective product.

[0010] A trained model according to another aspect of the present invention is trained using training data that receives as input a training image set including a first defective image of a first variety of product, which is a defective product, a second good image of a second variety of product, which is a good product, and a pseudo-defective image of the second variety of product, in which a pseudo-abnormal image showing a pseudo-abnormality different from an abnormality occurring in the second variety of product is superimposed on the second good image, and outputs information indicating whether the second variety of product included in the training image set is good or defective, and causes a computer to function in such a way that, in response to an input of an inspection image of the second variety of product to be inspected, the computer outputs information indicating whether the second variety of product to be inspected is good or defective. Effect of the Invention

[0011] According to the present invention, when artificially generated defective product images are used to inspect multiple types of products using a trained model, it is possible to improve the accuracy of judgment in product inspection while reducing the labor required to generate the trained model. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram of a quality determination device according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a diagram illustrating a concept of processing using a quality determination model according to the present embodiment. [Diagram 3] 4 is a flowchart illustrating an outline of a process performed by the quality assessment device according to the present embodiment. [Figure 4] 11 is a flowchart illustrating a quality determination model generation process according to the present embodiment. [Diagram 5] 10 is a flowchart illustrating a pseudo-abnormal image generating process according to the present embodiment. [Figure 6] FIG. 4 is a diagram illustrating anomaly information according to the embodiment. [Figure 7] 1A to 1C are diagrams illustrating an example of a pseudo-abnormal image generating method according to the present embodiment. [Figure 8] 1A to 1C are diagrams illustrating an example of a pseudo-abnormal image generating method according to the present embodiment. [Figure 9]1A to 1C are diagrams illustrating an example of a pseudo-abnormal image generating method according to the present embodiment. [Figure 10] 10A to 10C are diagrams illustrating generation of a pseudo-defective product image according to the present embodiment. [Figure 11] 11 is a flowchart illustrating a process of generating pseudo-defective segment information according to the present embodiment. [Figure 12] 10 is a flowchart illustrating a quality determination process according to the embodiment. [Figure 13] 11A to 11C are diagrams illustrating another example of the pseudo abnormal image generating method according to the embodiment. [Figure 14] FIG. 11 is a diagram for explaining the concept of processing by a quality determination model as another example according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described with reference to the accompanying drawings, in which like reference numerals denote like or similar structures.

[0014] 1 shows a block diagram of a quality assessment device 101 according to this embodiment. The quality assessment device 101 has a display unit 1011, a storage unit 1012, an image acquisition unit 1013, a pseudo-defective product image generation unit 1014, a quality assessment model generation unit 1015, and a quality assessment unit 1016. Each unit of the quality assessment device 101 can be realized by executing a program stored in a storage device by a processor in an information processing device such as a personal computer.

[0015] The quality assessment device 101 is a device used for visual inspection of products. Products targeted by the quality assessment device 101 include subtype products in which the components constituting the product are made of the same material but have different component shapes or component arrangement patterns, and these similar products are distinguished as being of different "varieties." One example of a product is a printed circuit board, and products with different wiring on the printed circuit board are distinguished as different varieties. Note that the products and varieties targeted by the quality assessment device 101 are not limited to this, and may include mechanical parts such as screws, for example.

[0016] For example, a certain product may include a product of type A (first type) and a product of type B (second type). When performing visual inspection of a product, it is preferable to judge whether the product is good or defective for each type. Here, a defective product means a product that has an abnormality, such as a product with a foreign object attached, a product with a chip, a product with a flaw, or a product with dirt or stains. In visual inspection by the quality judgment device 101, a judgment of whether the product is good or defective is performed by image processing using a trained model.

[0017] In this case, it is assumed that for products of type A, sufficient good product images (first good product images) and defective product images (first defective product images) are collected to enable model learning, while for products of type B, the number of defective product images is insufficient for learning. This situation occurs, for example, when the number of defective product images is small to begin with, or when collection of defective product images is omitted to reduce the man-hours required for generating a trained model. In the quality assessment device 101, the number of defective product images of type B is supplemented using pseudo-generated defective product images of products of type B (pseudo-defective product images). The quality assessment device 101 is a device that assesses whether products of type B are good or defective using a trained model trained using an image set including pseudo-defective product images.

[0018] The display unit 1011 is a display of the quality determination device 101. The storage unit 1012 stores various information used in processing by the quality determination device 101. The storage unit 1012 stores an image generation model 10121 and a quality determination model 10122 used in processing by the quality determination device 101. The storage unit 1012 also stores data generated by the quality determination device 101.

[0019] The image generation model 10121 (second trained model) is a trained model that is trained to generate an image based on segment information, which is an input of an image labeled by region or pixel unit by a segmentation means such as semantic segmentation or instance segmentation. As the image generation model 10121, for example, an image generation model such as label2image is used as a base model. The image generation model 10121 is trained using segment information obtained by segmenting images of good products and images of defective products of a product type A, for example.

[0020] The quality determination model 10122 (first trained model) is a trained model trained to output information indicating whether the product of type B to be inspected is a non-defective product or a defective product in response to an input of an inspection image of the product of type B to be inspected. The quality determination model 10122 will be described later.

[0021] The image acquisition unit 1013 acquires, for example, an external image of a product captured by an imaging device installed in a production line of the product from an external device. The image acquisition unit 1013 communicates with, for example, an information processing device that captures the image, and acquires the external image of the product. The image acquisition unit 1013 also acquires a defective product image of a type A product and a non-defective product image of a type B product (a second non-defective product image) from the external information processing device, and stores them in the memory unit 1012 of the quality assessment device 101.

[0022] The pseudo-defective product image generating unit 1014 generates a pseudo-defective product image of the product of type B by combining a pseudo-abnormality image showing a pseudo-abnormality different from an abnormality that actually occurred in the product of type B with a second non-defective product image of the product of type B. The generation of the pseudo-defective product image will be described later.

[0023] The quality determination model generation unit 1015 generates a quality determination model 10122 that outputs a determination result of whether a product of type B is pass or fail, based on the pseudo-defective product image generated by the pseudo-defective product image generation unit 1014. Specifically, the quality determination model generation unit 1015 generates the quality determination model 10122 based on the defective product image of product type A, the pass product image of product type B, and the pseudo-defective product image.

[0024] The quality judgment unit 1016 inputs an inspection image of the product of type B to be inspected to the quality judgment model 10122, and obtains information from the quality judgment model 10122 indicating whether the product of type B to be inspected is a pass / fail product.

[0025] In the description of the quality assessment device 101, the quality assessment device 101 performs both the generation of the quality assessment model 10122 by the quality assessment model generation unit 1015 and the judgment of whether the product is good or defective by the quality assessment unit 1016. However, the generation of the quality assessment model 10122 may be performed, for example, by another information processing device, and the generated quality assessment model 10122 may be stored in the memory unit 1012.

[0026] With reference to FIG. 2, the quality judgment device 101 will be described for determining whether a product of type B is good or bad. In step (A) of FIG. 2, the quality judgment model generation unit 1015 uses a learning image set including a defective product image I11 of a product of type A, a good product image I20 of a product of type B, and a pseudo-defective product image I21 of a product of type B to generate a quality judgment model 10122 that is trained to output information indicating whether a product of type B included in the learning image set is good or defective. Here, the defective product image I11 of the product of type A is an image that includes an actual foreign object O1. The pseudo-defective product image I21 of the product of type B is an image generated by the pseudo-defective product image generation unit 1014 described later so as to include a pseudo-generated foreign object O2.

[0027] In step (B), the quality judgment unit 1016 inputs the inspection image I22 of the product of type B to the quality judgment model 10122. In step (C), the quality judgment unit 1016 obtains a judgment result indicating whether the product of type B shown in the inspection image is a good product or a defective product from the quality judgment model 10122. In the example of Fig. 2, since the inspection image I22 of the product of type B contains a foreign object O3, the judgment result is "defective product."

[0028] As another example, the quality judgment model 10122 may be trained to output, for each pixel of the inspection image, which type of defect (also called abnormality) (for example, foreign matter, chipping, flaw, etc.) the defective product belongs to. In this case, the judgment result includes information indicating the position of the defect in pixel units. The judgment result may also include the judgment result of the defective product in pixel units and the type of defect. The information indicating whether the product of type B to be inspected is a good product or a defective product may be information indicating the position of an abnormal area in the inspection image. The information indicating whether the product of type B to be inspected is a good product or a defective product may include information indicating the position of a non-abnormal area. The information indicating whether the product of type B to be inspected is a good product or a defective product may include an inspection image binarized into an abnormal area indicating an abnormality in the inspection image and a non-abnormal area that is not an abnormal area. The inspection image binarized into an abnormal area and a non-abnormal area that is not an abnormal area has values ​​that change discontinuously, but may be an inspection image in which the numerical values ​​of the abnormal area and the non-abnormal area change continuously. In this case, the abnormal and non-abnormal regions are depicted by shading due to continuous changes in the numerical values. Furthermore, the information indicating whether the product of type B being inspected is a pass / fail product may include a determination result indicating whether the product of type B shown in the inspection image is a pass / fail product.

[0029] The quality judgment model 10122 can also perform other classifications to indicate whether the product of type B is a good product or a defective product. For example, the quality judgment model 10122 may be trained to output information indicating which type of defect (e.g., foreign matter, chipping, or flaw) the product of type B belongs to. The quality judgment model 10122 may also be trained to output information indicating which pixel of the inspection image is defective. The quality judgment model generation unit 1015 can appropriately label the learning image set according to the result output by the quality judgment model 10122 to train the quality judgment model 10122.

[0030] 3 shows a flowchart of the processing by the quality judging device 101. In step S301, the quality judging device 101 performs a quality judgment model generation process. In step S302, the quality judging unit 1016 performs a quality judgment process.

[0031] FIG. 4 shows a flowchart of the quality assessment model generation process.

[0032] In step S401, the image acquisition unit 1013 acquires a non-defective product image of type B product.

[0033] In step S402, the pseudo-defective image generating unit 1014 executes a pseudo-abnormal image generating process, which will be described with reference to FIG.

[0034] 5, the pseudo-defective image generating unit 1014 acquires the abnormality type (e.g., flaw, foreign matter, dirt, etc.) indicated by the pseudo-abnormal image to be synthesized. Here, the pseudo-defective image generating unit 1014 may acquire the abnormality type by, for example, an input from a user who operates the quality assessment device 101, or may acquire the abnormality type by randomly selecting it by the pseudo-defective image generating unit 1014.

[0035] In step S502, the pseudo-defective image generating unit 1014 refers to the anomaly information and selects a pseudo-abnormal image generating method associated with the anomaly type. The anomaly information is stored, for example, in the storage unit 1012. An example of the anomaly information is shown in FIG.

[0036] The abnormality information has the following items: "Anomaly type," "Pseudo abnormal image generating method," "Location of abnormality," and "Abnormality condition." The abnormality information is information that has been generated in advance, for example, based on defective product images of type A products for which a sufficient number of defective product images have been collected. The "Anomaly type" item stores information indicating the type of abnormality represented by the pseudo-abnormality. The "Pseudo abnormal image generating method" item stores a method for generating a pseudo-abnormal image. The "Location of abnormality" item stores information indicating the location where an abnormality may occur. The "Abnormal condition" item stores the size conditions that are set for the pseudo-abnormal image.

[0037] For example, when the type of abnormality is a flaw, the pseudo-abnormal image generation method is the random walk method 1 or use of a defective product image of type A, and the method is selected by the pseudo-defective product image generation unit 1014 according to a setting from the user. Also, it is stored that the flaw abnormality occurs in a gold-plated portion of the product. Also, it is stored that the abnormal condition for the flaw is that the width is 3 pixels or less, and the length is in the range of 10 to 30 pixels.

[0038] When the abnormality type is a foreign body, the pseudo-abnormal image generation method is the random walk method 2 or use of a defective product image of type A, and the method is selected by the pseudo-defective product image generation unit 1014 according to a setting from the user. Also, it is stored that a foreign body abnormality occurs in a solder resist portion of a product. Also, it is stored that the abnormal condition for a foreign body is an area of ​​1200 pixels or less.

[0039] When the abnormality type is dirt or stains, the pseudo-abnormal image generation method is to use a noise image or a defective product image of type A, and the method is selected by the pseudo-defective product image generation unit 1014 according to a setting from the user. Also, it is stored that the dirt or stain abnormality occurs in the solder resist portion of the product. Also, it is stored that the abnormal condition for dirt or stains is an area of ​​3000 pixels or less.

[0040] In step S503, the pseudo-defective image generating unit 1014 generates a pseudo-abnormal image based on the pseudo-abnormal image generating method selected in step S502.

[0041] Returning to FIG. 4, in step S403, the pseudo-defective image generating unit 1014 generates pseudo-abnormal segment information by segmenting an area showing a pseudo-abnormality based on the pseudo-abnormal image.

[0042] An example of the pseudo abnormal image generating process and the generation of pseudo abnormal segment information will be described with reference to FIGS.

[0043] 7 shows an example in which the type of anomaly is a flaw and the pseudo-abnormal image generating method is the random walk method 1. In the random walk method 1, a graph G70 is generated by arranging a sequence of random walk points on a two-dimensional coordinate system and continuously connecting each point with a spline curve. Then, the pseudo-defective image generating unit 1014 generates (7A) an image I70 including a curve D that connects six points including the point P1 in the graph G70. The pseudo-defective image generating unit 1014 generates (7B) a pseudo-abnormal segment image S70 in which an area showing a pseudo-abnormality of a flaw is segmented based on the image I70.

[0044] 8 shows an example in which the type of anomaly is a foreign object and the pseudo-abnormal image generating method is the random walk method 2. In the random walk method 2, a random walk point sequence including a point P2 whose angle and distance from the center C are randomly determined is arranged on a two-dimensional coordinate system, and (8A) an image I81 including a foreign object O indicated by a closed curve (closed line) formed by connecting the points is generated. After that, the pseudo-defective image generating unit 1014 generates a pseudo-abnormal segment image S80 in which a region showing a pseudo-abnormality of a foreign object is segmented based on the image I70 (8B).

[0045] FIG. 9 shows an example in which the type of anomaly is dirt or stain, and the pseudo-abnormal image generation method uses a noise image. The pseudo-defective image generation unit 1014 generates a noise image I90 using, for example, a generation method of Perlin noise or fBM (fractional Brownian Motion) noise. The pseudo-defective image generation unit 1014 (9A) converts the generated noise image into a multi-value image I91 having two or more pixel values ​​based on at least one predetermined threshold, and (9B) extracts an image I92 corresponding to a pseudo-abnormality. Here, the extraction of the image I92 in (9B) extracts an image I92 formed by surrounding an area A1 formed by successive pixels of a first value in the multi-value image I91 with pixels of a second value different from the first value as shown in an area A2. The pseudo-defective image generation unit 1014 (C) generates pseudo-abnormal segment information S90 in which an area showing a pseudo-abnormality is segmented based on the image I92.

[0046] 9, the case of binarization has been described as an example of multi-value conversion, but the pseudo-defective image generating unit 1014 may convert the noise image to have three or more pixel values. For example, the noise image may be converted to an image having four pixel values, such as a pixel indicating a non-defective product, a pixel indicating a flaw, a pixel indicating a chip, and a pixel indicating a dent. This makes it possible to generate pseudo-abnormal segment information while distinguishing between types of defects.

[0047] Returning to Fig. 4, in step S404, the pseudo-defective product image generating unit 1014 performs a segmentation process on the non-defective product image of the product type B to generate non-defective product segment information. An example of this process is shown in step (10A) of Fig. 10. From the non-defective product image I20 of the product type B, non-defective product segment information S20 is generated. The non-defective product segment information S20 includes, for example, a segment S201 corresponding to gold plating and a segment S202 corresponding to soldering resist depending on the material of the product.

[0048] In step S405, the pseudo-defective image generating unit 1014 performs a pseudo-defective segment information generating process. The pseudo-defective segment information generating process will be described with reference to Fig. 11. In step S1101, the pseudo-defective image generating unit 1014 acquires the generated pseudo-abnormal image and the abnormality type indicated by the pseudo-abnormal segment.

[0049] In step S1101, the pseudo-defective image generating unit 1014 converts the pseudo-abnormal segment so as to satisfy the abnormality condition associated with the anomaly type. For example, if the anomaly type is a flaw, the pseudo-defective image generating unit 1014 modifies the size of the pseudo-abnormal segment so that the width of the pseudo-abnormal segment is 3 pixels or less and the length is in the range of 10 to 30 pixels. For example, if the anomaly type is a foreign object, the pseudo-defective image generating unit 1014 modifies the size of the pseudo-abnormal segment so that the area of ​​the pseudo-abnormal segment is 1200 pixels or less. Also, if the anomaly type is a stain or stain, the pseudo-defective image generating unit 1014 modifies the size of the pseudo-abnormal segment so that the area of ​​the pseudo-abnormal segment is 3000 pixels or less.

[0050] In step S1101, the pseudo-defective product image generating unit 1014 acquires an abnormality location associated with the abnormality type. For example, if the abnormality type is a flaw, "gold plating" is acquired as the abnormality location. Also, if the abnormality location is a foreign object, dirt, or stain, "soldering resist" is acquired as the abnormality location.

[0051] In step S1101, the pseudo-defective image generating unit 1014 generates pseudo-defective segment information by combining a pseudo-abnormal segment with a segment corresponding to an abnormality occurrence location in the non-defective segment information. For example, when the pseudo-abnormal segment indicates a flaw, the pseudo-defective image generating unit 1014 combines the pseudo-abnormal segment with the segment corresponding to the gold plating since the gold plating is the abnormality occurrence location.

[0052] 10, when the pseudo abnormal segment indicates a foreign body, the pseudo defective image generating unit 1014 combines the pseudo abnormal segment with the segment S202 corresponding to the solder resist since the solder resist is the location where the abnormality occurs. By this process, pseudo defective segment information S21 is generated in which the pseudo abnormal segment information S90 is combined with the non-defective segment information S20.

[0053] In step S406, the pseudo-defective image generating unit 1014 inputs the pseudo-defective segment information to the image generation model 10121 to generate a pseudo-defective image of the product type B. An example of this process is shown in step (10C) of FIG. 10. By this process, a pseudo-defective image I21 including a foreign object O2 (pseudo-abnormal image) is generated from the pseudo-defective segment information S21. The pseudo-defective image generating unit 1014 can generate a plurality of types of pseudo-defective images by repeating the generation of pseudo-defective images. The pseudo-defective image generating unit 1014 can generate pseudo-defective images including anomalies at anomaly occurrence locations where anomalies are highly likely to occur, depending on the anomaly type. This makes it possible to generate pseudo-defective images that are closer to actual defects.

[0054] In step S407, the quality determination model generation unit 1015 trains a quality determination model based on a training image set including a defective product image of type A product, a non-defective product image of type B product, and a pseudo-defective product image of type B product.

[0055] The quality judgment process will be described with reference to Fig. 12. In step S1201, the image acquisition unit 1013 acquires an inspection image of the product of type B. In step S1202, the quality judgment unit 1016 inputs the inspection image of the product of type B to the quality judgment model 10122. In step S1203, the quality judgment unit 1016 acquires a judgment result from the quality judgment model 10122. As a result, an appearance inspection of the product can be performed using the quality judgment model 10122.

[0056] In determining whether a product is good or bad using normal machine learning, learning is often performed using a learning image set that includes good product images of product type B and pseudo-defective product images of product type B. The pseudo-defective product images of product type B are not images of abnormalities occurring in actual products, but are pseudo-generated images. Therefore, for example, if a quality determination model is trained using only good product images and pseudo-defective product images of product type B, it may be trained to determine whether a product is defective based on the features generated by the pseudo-defective product images. As a result, the model may make false positives or over-detections, resulting in low accuracy.

[0057] On the other hand, the quality judgment model 10122 in the quality judgment device 101 includes in the learning image set not only good product images and pseudo-defective product images of product type B, but also defective product images of product type A. In the quality judgment model 10122, the defective product images of product type A are images showing actual abnormalities that have occurred in the actual product. Therefore, the quality judgment model 10122 can also use feature amounts based on actual abnormalities that occur in the actual product, and is trained to judge defective products while reflecting the features of actual defective products. As a result, the quality judgment device 101 can use the quality judgment model 10122 to improve the judgment accuracy in product inspection.

[0058] In the above-described embodiment, an example of generating a pseudo-defective product image based on a random walk method or a noise image has been described, but the method of generating a pseudo-defective product image is not limited to this. For example, a pseudo-defective product image may be generated by cutting out a pseudo-abnormal image, which is a region corresponding to an abnormality, from a defective product image of a product of type A, and synthesizing it with a non-defective product image of a product of type B. For example, as shown in FIG. 13, (13A) a pseudo-abnormal image, which is a region corresponding to a foreign body O1, may be cut out from a defective product image of a product of type A, and (13B) an image I23 may be generated in which a foreign body O4 based on the foreign body O1 is placed in a segment S202 where a foreign body should be placed in the non-defective product segment information of product B. When synthesizing the images, an image processing method such as Poisson blending may be used. In addition, synthesis may be performed so that a weighted average is taken according to the distance from the center of the defective region or the color tone so that the cut-out image blends well with the non-defective product image. In addition, a defective product image of a product of type A is segmented, and a region corresponding to a defect is generated. A pseudo-defective product image may be generated by synthesizing a non-defective product image of the product type B with the segment information that has been subjected to the segmentation process. In this case, the pseudo-defective product image is generated by the image generation model 10121 based on the synthesized segment information.

[0059] Another example of the quality assessment device 101 according to this embodiment will be described. A process for assessing whether a product of type B is good or bad by the quality assessment device 101 according to this embodiment will be described with reference to FIG. 14. This example differs from the example shown in FIG. 2 in that a good product image I10 (first good product image) of a product of type A is used for learning the quality assessment model 10122A. In this example, the image acquisition unit 1013 acquires a good product image of the product of type A from an external information processing device in addition to an inspection image, a defective product image of the product of type A, and a good product image of the product of type B, and stores them in the memory unit 1012 of the quality assessment device 101.

[0060] In step (A) of FIG. 14, the quality judgment model generation unit 1015 uses a learning image set including a good product image I10 of a product of type A, a defective product image I11 of a product of type A, a good product image I20 of a product of type B, and a pseudo-defective product image I21 of a product of type B to generate a quality judgment model 10122 trained to output information indicating whether a product of type B included in the learning image set is good or defective. The process of step (C) of FIG. 14 is similar to that of step (C) of FIG. 2, but the process of step (B) of FIG. 14 is different from that of step (B) of FIG. 2. Specifically, the quality judgment unit 1016 inputs the good product image I20 and the inspection image I22 of the product of type B to the quality judgment model 10122. As a result, even if an image of a product of type A is used in learning the quality judgment model 10122 that judges the quality of a product of type B, the quality judgment model 10122 is less susceptible to the influence of differences in product structure, that is, differences in product types, and the judgment accuracy in product inspection is improved. Also, by performing a comparison process such as aligning the image coordinates of the good product image and the inspection image so that the same parts of the product in the good product image and the inspection image have the same coordinates and taking the difference between the good product image and the inspection image, it becomes easier to identify defective parts present in the inspection image. Note that in step (B) of FIG. 14, unlike the process in step (B) of FIG. 2, in the process in step (C) of FIG. 14, the quality judgment unit 1016 inputs the good product image I20 of the product of type B in addition to the inspection image I22 to the quality judgment model 10122. This is because it is expected that the image format when the quality judgment unit 1016 inputs the image to the quality judgment model 10122 and the image format when the quality judgment model generation unit 1015 generates the quality judgment model 10122 are the same image format.

[0061] The present embodiment has been described above. The quality judgment device 101 is a quality judgment model 10122 that is trained using teacher data that receives as input a learning image set including a defective product image I11 of a defective product of type A, a good product image I20 of a good product of type B, and a pseudo-defective product image I21 of a product of type B in which a pseudo-abnormal image showing a pseudo-abnormality different from an abnormality occurring in the product of type B is synthesized with the good product image I20, and outputs information indicating whether the product of type B included in the learning image set is a good product or a defective product. The quality judgment device 101 receives as output information indicating whether the product of type B is a good product or a defective product from the quality judgment model 10122.

[0062] This allows the quality assessment device 101 to assess the inspection image using the quality assessment model 10122 that has been trained to include defective product images of product type A, which are images showing actual abnormalities that have occurred in the actual product. The quality assessment device 101 can also use features based on actual abnormalities that have occurred in the actual product, and is trained to assess defective products while reflecting the features of actual defective products, thereby improving the assessment accuracy in product inspection compared to when assessing defective products based on features generated by pseudo defective product images. In addition, the use of pseudo defective product images reduces the amount of work required to collect defective product images, etc.

[0063] In the above embodiment, the pseudo-defective product image generating unit 1014 may generate a pseudo-defective product image by synthesizing a pseudo-abnormal image at a position in the second good product image corresponding to the location of the abnormality, based on the type of abnormality in the pseudo-abnormal image and the location of the abnormality associated with the type of abnormality. This allows the pseudo-defective product image generating unit 1014 to generate a pseudo-defective product image including an abnormality at the location of the abnormality where the abnormality is likely to occur, according to the type of abnormality, and to generate a pseudo-defective product image that is closer to an actual defect.

[0064] In the above embodiment, the pseudo-defective image generating unit 1014 may generate pseudo-abnormal segment information by segmenting the pseudo-abnormal image, generate good segment information S20 by segmenting the good image I20, and generate pseudo-defective segment information S21 based on the good segment information S20 and the pseudo-abnormal segment information S50. The pseudo-defective image generating unit 1014 may also generate the pseudo-defective image I21 based on the pseudo-defective segment information S21.

[0065] By generating pseudo-defective product images via segment information, it is possible to generate noise and reproduce a large number of abnormalities, so that pseudo-defective product images can be generated more efficiently.

[0066] In the above embodiment, the pseudo-defective product image generation unit 1014 inputs defective product segment information obtained by segmenting a defective product image, inputs pseudo-defective product segment information S21 to an image generation model 10121 that has been trained to output a defective product image, and generates a pseudo-defective product image I21 as an output.

[0067] This makes it possible to generate pseudo-defective product images more efficiently.

[0068] In the above embodiment, the pseudo-defective image generating unit 1014 acquires a noise image I90, converts the noise image I90 into a multi-value image having two or more pixel values ​​based on at least one predetermined threshold, and extracts an image I92 from the converted noise image I91, in which an area A1 formed by successive pixels of a first value in the converted noise image I91 is surrounded by pixels of a second value different from the first value. The pseudo-defective image generating unit 1014 may generate pseudo-abnormal segment information S90 in which the area A1 is segmented as an area indicating a pseudo-abnormality.

[0069] This makes it possible to extract a large number of noise images when one noise image contains a plurality of closed regions, thereby enabling pseudo-defective product images to be generated more efficiently.

[0070] In the above embodiment, the image acquisition unit 1013 may acquire a non-defective product image I10 of a product of type A that is a non-defective product, and the learning image set may further include non-defective product images of the first product type.

[0071] This makes it possible to more accurately distinguish whether or not a product of type B is non-defective, thereby improving the accuracy of product inspection.

[0072] In the above aspect, the quality judgment unit 1016 may input the non-defective product image I10 of the product type A, which is a non-defective product, and the inspection image to the quality judgment model 10122.

[0073] The above-described embodiments are intended to facilitate understanding of the present invention, and are not intended to limit the present invention. The elements of the embodiments, as well as their arrangements, conditions, shapes, sizes, etc., are not limited to those illustrated, and can be changed as appropriate. In addition, configurations shown in different embodiments can be partially substituted or combined with each other. [Explanation of symbols]

[0074] 101...quality judgment device, 1011...display unit, 1012...storage unit, 1013...image acquisition unit, 1014...pseudo-defective product image generation unit, 1015...quality judgment model generation unit, 1016...quality judgment unit, 10121...image generation model, 10122, 10122A...quality judgment model

Claims

1. The computer A trained model, A first defective product image of a first type of product that is a defective product; A second quality image of the second type of the product, which is a quality product; and a pseudo-defective product image of the second type of the product, the pseudo-abnormal product image being synthesized with the second non-defective product image, the pseudo-abnormal product image being a pseudo-abnormal product different from the abnormality occurring in the second type of the product; The input is a training image set containing A first trained model trained using training data that outputs information indicating whether the second product type included in the training image set is a good product or a defective product, inputting an inspection image of the second product type that is an inspection target; and obtaining, as output, information from the first trained model indicating whether the product of the second variety being inspected is good or defective.

2. 2. The method of claim 1 , The computer, The method further includes synthesizing the pseudo-abnormal image at a position in the second good image that corresponds to the location of the abnormality based on the type of abnormality in the pseudo-abnormal image and the location of the abnormality associated with the type of abnormality, thereby generating the pseudo-defective image.

3. 3. The method according to claim 1 or 2, The computer, generating non-defective product segment information by segmenting the second non-defective product image; generating pseudo abnormal segment information by segmenting the pseudo abnormal image; generating pseudo-defective segment information based on the non-defective segment information and the pseudo-abnormal segment information; The method, wherein generating the pseudo-defective image includes generating the pseudo-defective image based on the pseudo-defective segment information.

4. 4. The method of claim 3, The generating of the pseudo-defective product image includes: A method comprising: inputting defective segment information obtained by segmenting the first defective image; inputting the pseudo-defective segment information into a second trained model trained to output the first defective image; and generating the pseudo-defective image as an output.

5. 4. The method of claim 3, The computer, acquiring a noise image; The generating of the pseudo abnormal segment information includes: converting the noise image into a multi-value image having two or more pixel values ​​based on at least one predetermined threshold; extracting from the converted noise image an image in which a first region formed by successive pixels having a first value in the converted noise image is surrounded by pixels having a second value different from the first value; generating the pseudo anomaly segment information in which the first region is segmented as a region indicating the pseudo anomaly.

6. 2. The method of claim 1 , The computer, acquiring a first quality image of a first variety of products that are quality products; The training image set further includes a first quality image of the first variety of product.

7. 7. The method of claim 6, Inputting the inspection image includes: inputting the first good product image and the inspection image to the trained model.

8. On the computer, A trained model, A first defective product image of a first type of product that is a defective product; A second quality image of the second type of the product, which is a quality product; and a pseudo-defective product image of the second type of the product, the pseudo-abnormal product image being synthesized with the second non-defective product image, the pseudo-abnormal product image being a pseudo-abnormal product different from the abnormality occurring in the second type of the product; The input is a training image set containing A first trained model trained using training data that outputs information indicating whether the second product type included in the training image set is a good product or a defective product, inputting an inspection image of the second product type that is an inspection target; and obtaining, as output, information from the first trained model indicating whether the second product type to be inspected is a good product or a defective product.

9. A trained model, A first defective product image of a first type of product that is a defective product; A second quality image of the second type of the product, which is a quality product; and a pseudo-defective product image of the second type of the product, the pseudo-abnormal product image being synthesized with the second non-defective product image, the pseudo-abnormal product image being a pseudo-abnormal product different from the abnormality occurring in the second type of the product; The input is a training image set containing A first trained model trained using training data that outputs information indicating whether the second product type included in the training image set is a good product or a defective product, an information processing device comprising: a quality determination unit that inputs an inspection image of the second product type that is an inspection target, and obtains, as an output, information indicating whether the second product type that is an inspection target is a good product or a defective product from the first trained model.

10. A first defective product image of a first type of product that is a defective product; A second quality image of the second type of the product, which is a quality product; and a pseudo-defective product image of the second type of the product, the pseudo-abnormal product image being synthesized with the second non-defective product image, the pseudo-abnormal product image being a pseudo-abnormal product different from the abnormality occurring in the second type of the product; The input is a training image set containing learning is performed using teacher data that outputs information indicating whether the second product type included in the learning image set is a non-defective product or a defective product; A trained model that causes a computer to function in such a way that, in response to input of an inspection image of the second variety of the product being inspected, the computer outputs information indicating whether the second variety of the product being inspected is a pass or fail product.

Images