Learning data generation device, learning data generation method, program, and inspection device

Abstract

To simply realize an accurate annotation when generating learning data to be applied to an inspection device capable of detecting a defective article from among a plurality of inspection objects.SOLUTION: A background image Ibg obtained by extracting only a background bg, a non-defective article extracted image Ige obtained by extracting only images of a plurality of non-defective articles Pg, and a defective article extracted image Ide obtained by extracting only an image of a defective article Pd are generated, and a learning image IL is generated by synthesizing these images (step S10 to S16). Consequently, an arrangement of the defective article extracted image Ide in the learning image IL is easily specified. Then an annotation is automatically applied on the basis of the specified arrangement of the defective article extracted image Ide. Thus, an accurate annotation can be simply realized when generating learning data to be applied to an inspection device 1 capable of detecting the defective article Pd from among a plurality of inspection objects 90.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present invention relates to a training data generation device and a training data generation method capable of generating training data to be applied to an inspection device capable of detecting defective products from among multiple inspection objects, a program used therefor, and an inspection device equipped with a learning model trained using training data generated by the training data generation device. [Background technology]

[0002] Japanese Patent Laid-Open Publication No. 2019-211288 (Patent Document 1) describes an inspection device that arranges multiple inspection objects in a line and transports them while capturing images of each object using a camera, extracts features from each captured image using a trained learning device generated by a convolutional neural network, and determines whether the transported inspection object is a non-defective product based on the extracted features. This inspection device can automatically determine the quality of the inspection objects in real time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-211288 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, the learning device is trained in advance based on a plurality of images to which teacher signals indicating whether the product is good or defective are assigned. However, in order to improve the accuracy of determining whether a product is good or defective, it is important to assign accurate annotations to the images. Here, in a configuration in which inspection objects conveyed one by one are determined to be good or defective, as in the inspection device described in the above-mentioned publication, if the determination result is a defective product, the location of defects, etc., is identified in the image of the inspection object determined to be defective, and annotations are assigned. In this case, since it is known that the inspection object is a defective product, it is sufficient to identify the location of defects, etc., in the image of the inspection object. Therefore, annotations can be assigned relatively easily.

[0005] On the other hand, in a configuration where defective products are detected from multiple inspection targets conveyed at the same time, there are cases where the inspection targets overlap with each other, making it difficult to identify the position of the defective products from the captured image. This not only increases the workload required for annotation, but also increases the probability of assigning incorrect annotations.

[0006] The present invention has been made in view of the above, and has an object to provide a technology that can easily realize accurate annotation in generating training data to be applied to an inspection device that can detect defective products from among multiple inspection objects. Another object of the present invention is to provide a technology that contributes to improving the efficiency of training and the accuracy of inspection in an inspection device that can detect defective products from among multiple inspection objects. [Means for solving the problem]

[0007] A preferred embodiment of the training data generation device according to the present invention provides a training data generation device capable of generating training data for application to an inspection device capable of detecting defective products from among multiple inspection targets. The training data generation device includes an image acquisition unit, a defective product extraction image generation unit, a training image generation unit, and an annotation assignment unit. The image acquisition unit is capable of acquiring good product images, which are images of only good products including a background, and a defective product image, which is an image of only at least one defective product including a background. The defective product extraction image generation unit generates a defective product extraction image, which is an image obtained by extracting only the image of the defective product excluding the background from the defective product image. The training image generation unit generates a training image by combining the good product image and the defective product extraction image. The annotation assignment unit then assigns annotations to the training image based on the arrangement of the defective product extraction image in the training image. Here, in this invention, a "normal product" typically corresponds to one that has no defects (has the desired quality), and a "defective product" corresponds to one that has defects (does not have the desired quality), but in the case of an embodiment in which an item other than the desired type and specifications is detected from a plurality of inspection objects that include different types and specifications, a "normal product" corresponds to one that has the desired type and specifications, and a "defective product" corresponds to one that does not have the desired type and specifications.

[0008] According to the present invention, a training image is generated by combining multiple images of good products, including the background, and a defective product extraction image, which is an extracted image of only defective products. This allows for easy identification of the arrangement of the defective product extraction image in the training image. Furthermore, annotations are automatically added based on the identified arrangement of the defective product extraction image. This allows for easy addition of accurate annotations when generating training data for application to an inspection device capable of detecting defective products from multiple inspection targets. As a result, training efficiency and inspection accuracy can be improved in an inspection device capable of detecting defective products from multiple inspection targets.

[0009] According to a further aspect of the training data generation device of the present invention, the device further includes a good-item extracted image generation unit that generates a good-item extracted image, which is an image obtained by extracting only images of multiple good items from good-item images and excluding the background. The image acquisition unit is capable of acquiring a background image, which is an image of only the background. The training image generation unit generates a training image by combining the background image, the good-item extracted image, and the defective-item extracted image, and is capable of changing the order in which the good-item extracted image and the defective-item extracted image are superimposed when generating the training image.

[0010] According to this embodiment, it is possible to generate multiple learning images by changing the order in which the non-defective product extracted image and the defective product extracted image are superimposed, thereby making it possible to improve the efficiency of learning and the accuracy of inspection in an inspection device that can detect defective products from multiple inspection targets.

[0011] According to a further aspect of the training data generation device of the present invention, the device further includes a good-item extracted image generation unit that generates a good-item extracted image, which is an image obtained by extracting only images of multiple good items from good-item images and excluding the background. The image acquisition unit is capable of acquiring a background image, which is an image of only the background. The training image generation unit generates a training image by combining the background image, the good-item extracted image, and the defective-item extracted image, and is capable of changing the degree of overlap between the good-item extracted image and the defective-item extracted image when generating the training image.

[0012] According to this embodiment, it is possible to generate a plurality of learning images by changing the degree of overlap between the good product extraction image and the defective product extraction image, thereby making it possible to improve the efficiency of learning and the inspection accuracy in an inspection device that can detect defective products from among a plurality of inspection targets.

[0013] According to a further aspect of the training data generation device of the present invention, the device further includes a good-item extracted image generation unit that generates a good-item extracted image, which is an image obtained by extracting only images of multiple good items from good-item images and excluding the background. The image acquisition unit is capable of acquiring a background image, which is an image of only the background. The training image generation unit generates a training image by combining the background image, the good-item extracted image, and the defective-item extracted image, and is capable of positioning the defective-item extracted image at any position relative to the good-item extracted image when generating the training image.

[0014] According to this embodiment, it is possible to generate multiple learning images by changing the position of the defective product extraction image relative to the non-defective product extraction image, thereby making it possible to improve the efficiency of learning and the inspection accuracy in an inspection device that can detect defective products from multiple inspection targets.

[0015] According to a further aspect of the present invention, the learning data generating device further includes a timing determination unit capable of determining the timing of capturing at least a defective product image, wherein the timing determination unit compares the captured image area with a background image when capturing at least the defective product image, and determines that it is the right time to capture the defective product image if there is no difference between the captured image and the background image at least in the outer periphery of the captured image area.

[0016] According to this embodiment, it is possible to prevent the capture of an image of a defective product in a state where an unnecessary object such as a worker's hand is captured in the image capture area, thereby realizing the generation of an appropriate learning image.

[0017] According to a further aspect of the learning data generation device of the present invention, which includes a timing determination unit, when there is a difference between the image capture area and the background image at least in the outer periphery of the image capture area, the timing determination unit determines whether the difference changes over a predetermined time period. If the difference does not change over the predetermined time period, the timing determination unit determines that it is time to capture an image, and if the difference changes over the predetermined time period, the timing determination unit determines that it is not time to capture an image.

[0018] According to this embodiment, when there is a difference between the image and the background image at least in the outer periphery of the image capture area, it is possible to accurately determine whether the difference is due to an unwanted object such as a worker's hand being captured in the image capture area, thereby generating a more appropriate learning image.

[0019] According to a further aspect of the learning data generation device of the present invention, the image acquisition unit can acquire changed-orientation defective product images, which are images of defective products captured in different orientations. The defective-product extraction image generation unit can generate changed-orientation defective product extraction images, which are images obtained by extracting only the defective product images excluding the background from the changed-orientation defective product images. The learning image generation unit can generate learning images by combining the background image, the good-product extraction image, and the changed-orientation defective product extraction image.

[0020] According to this aspect, it is possible to generate a plurality of learning images in which the posture of a defective product is changed, thereby making it possible to improve the efficiency of learning and the accuracy of inspection in an inspection device that can detect defective products from among a plurality of inspection targets.

[0021] According to a further aspect of the learning data generation device of the present invention, the image acquisition unit can acquire an imaging mode-altered background image, an imaging mode-altered good product image, and an imaging mode-altered defective product image, which are images of a background, a good product, and a defective product captured in different imaging modes. The defective product extraction image generation unit can generate an imaging mode-altered defective product extraction image, which is an image obtained by extracting only images of defective products from the imaging mode-altered defective product image, excluding the imaging mode-altered background image. The good product extraction image generation unit can generate an imaging mode-altered good product extraction image, which is an image obtained by extracting only images of good products from the imaging mode-altered good product image, excluding the imaging mode-altered background image. The learning image generation unit can generate learning images by combining the imaging mode-altered background image, the imaging mode-altered good product extraction image, and the imaging mode-altered defective product extraction image. Here, "different imaging modes" in the present invention include modes of imaging using a combination of various cameras capable of imaging with light of different wavelengths and lighting capable of irradiating light of different wavelengths (e.g., multispectral images), modes of imaging with different camera positions or different lighting positions, modes of imaging in different imaging modes with different camera exposure times or light irradiation times, modes of imaging by irradiating the object to be inspected with light of different patterns, and modes of measuring the reflection time of light, sound waves, etc. to image the shape of the object to be inspected.

[0022] According to this embodiment, multiple learning images can be generated using multiple images of a background, a non-defective product, and a defective product captured by changing the imaging mode. That is, multiple learning images can be generated without preparing multiple defective products. This makes it possible to improve the efficiency of learning and the accuracy of inspection in an inspection device that can detect defective products from multiple inspection targets.

[0023] According to a preferred embodiment of the inspection device of the present invention, an inspection device capable of detecting defective products from among multiple inspection objects is configured. The inspection device includes an imaging unit, a trained model, a feature extractor, and a classifier. The imaging unit is capable of capturing inspection images, which are images of the multiple inspection objects. The trained model is trained in advance using training data generated by the training data generation device of the present invention according to any of the above-described aspects. The feature extractor extracts features from the inspection images using the trained model. The classifier determines whether or not there are defective products among the multiple inspection objects based on the features extracted by the feature extractor.

[0024] According to the present invention, since the training data generation device according to any one of the above-described aspects of the present invention is provided, it is possible to achieve the same effects as those achieved by the training data generation device of the present invention, such as the effect of being able to easily add accurate annotations, thereby improving the efficiency of training and the accuracy of inspection.

[0025] According to a further aspect of the present invention, the inspection device further includes a transport unit capable of transporting at least defective products to an imaging area, and a vibration unit capable of applying vibrations to at least the defective products in the imaging area via the transport unit. The imaging unit is capable of capturing images of defective products whose posture has been changed by vibration from the vibration unit. The image acquisition unit is capable of acquiring changed-posture defective product images, which are images of defective products captured in different postures. The defective product extraction image generation unit is capable of generating changed-posture defective product extraction images, which are images of only defective products excluding the background from the changed-posture defective product images. The learning image generation unit is capable of generating learning images by combining the background image, the good product extraction image, and the changed-posture defective product extraction image.

[0026] According to this embodiment, since the conveying unit is simply vibrated by the vibration unit, the posture of the defective product can be easily changed. Furthermore, multiple learning images can be generated with the posture of the defective product changed. This makes it possible to improve the efficiency of learning and the inspection accuracy in an inspection device that can detect defective products from multiple inspection targets.

[0027] According to a further aspect of the inspection device of the present invention, the imaging unit is capable of imaging the background, the non-defective products, and the defective products in different imaging modes. The image acquisition unit is capable of acquiring an imaging mode-altered background image, an imaging mode-altered non-defective product image, and an imaging mode-altered defective product image, which are images of the background, the non-defective products, and the defective products, respectively, imaged in different imaging modes. The defective product extraction image generation unit is capable of generating an imaging mode-altered defective product extraction image, which is an image obtained by extracting only images of the defective products, excluding the imaging mode-altered background image, from the imaging mode-altered defective product image. The non-defective product extraction image generation unit is capable of generating an imaging mode-altered non-defective product extraction image, which is an image obtained by extracting only images of the non-defective products, excluding the imaging mode-altered background image, from the imaging mode-altered non-defective product image. The learning image generation unit is capable of generating a learning image by combining the imaging mode-altered background image, the imaging mode-altered non-defective product extraction image, and the imaging mode-altered defective product extraction image. Here, "different imaging modes" in the present invention include modes of imaging using a combination of various cameras capable of imaging with light of different wavelengths and lighting capable of irradiating light of different wavelengths (e.g., multispectral images), modes of imaging with different camera positions or different lighting positions, modes of imaging in different imaging modes with different camera exposure times or light irradiation times, modes of imaging by irradiating the object to be inspected with light of different patterns, and modes of measuring the reflection time of light, sound waves, etc. to image the shape of the object to be inspected.

[0028] According to this embodiment, multiple learning images can be generated using multiple images of a background, a non-defective product, and a defective product captured by changing the imaging mode. That is, multiple learning images can be generated without preparing multiple defective products. This makes it possible to improve the efficiency of learning and the accuracy of inspection in an inspection device that can detect defective products from multiple inspection targets.

[0029] A preferred embodiment of the inspection method according to the present invention provides a training data generation method for generating training data for application to an inspection device capable of detecting defective products from among multiple inspection targets. The training data generation method (a) acquires good-product images, which are images of only good products including a background, and at least one defective-product image, which is an image of only defective products including a background, (b) generates defective-product extracted images, which are images of only the defective products excluding the background, from the acquired defective-product images, (c) generates training images by combining the good-product images and the defective-product extracted image, and (d) generates training data by annotating the generated training images.

[0030] According to the present invention, a training image is generated by combining a background image from which only the background is extracted, a good-product extracted image from which only images of multiple good products are extracted, and a defective-product extracted image from which only images of defective products are extracted. This allows for easy identification of the arrangement of the defective-product extracted images in the training image. Annotations are then automatically added based on the identified arrangement of the defective-product extracted images. This allows for easy and accurate annotations to be added when generating training data for application to an inspection device capable of detecting defective products from multiple inspection targets. As a result, training efficiency and inspection accuracy can be improved in an inspection device capable of detecting defective products from multiple inspection targets.

[0031] According to a further aspect of the inspection method of the present invention, the method further includes a step of generating a non-defective product extracted image, which is an image obtained by extracting only images of a plurality of non-defective products from the non-defective product images and excluding the background. Step (a) also includes a step of acquiring a background image, which is an image of only the background. Step (c) also includes a step of generating a learning image by combining the background image, the non-defective product extracted image, and the defective product extracted image, and changing the order of superimposition of the non-defective product extracted image and the defective product extracted image when generating the learning image.

[0032] According to this embodiment, it is possible to generate multiple learning images by changing the order in which the non-defective product extracted image and the defective product extracted image are superimposed, thereby making it possible to improve the efficiency of learning and the accuracy of inspection in an inspection device that can detect defective products from multiple inspection targets.

[0033] According to a further aspect of the inspection method of the present invention, the method further includes a step of generating a non-defective product extracted image, which is an image obtained by extracting only images of a plurality of non-defective products from the non-defective product images and excluding the background. Step (a) also includes a step of acquiring a background image, which is an image of only the background. Step (c) also includes a step of generating a learning image by combining the background image, the non-defective product extracted image, and the defective product extracted image, and changing the degree of overlap between the non-defective product extracted image and the defective product extracted image when generating the learning image.

[0034] According to this embodiment, it is possible to generate a plurality of learning images by changing the degree of overlap between the good product extraction image and the defective product extraction image, thereby making it possible to improve the efficiency of learning and the inspection accuracy in an inspection device that can detect defective products from among a plurality of inspection targets.

[0035] According to a preferred embodiment of the program of the present invention, there is provided a program for generating training data to be applied to an inspection device capable of detecting defective products from among a plurality of inspection targets. The program is configured to cause one or more computers to execute each step of the training data generation method of the present invention according to any of the above-described aspects. The program may be recorded on a computer-readable recording medium, such as a hard disk, ROM, SSD, flash memory (such as a USB memory or SD card), floppy disk, CD, or DVD, or may be distributed from one computer to another via a transmission medium, such as a communication network such as the Internet or a LAN, or may be transmitted in any other manner.

[0036] According to this embodiment, each step of the learning data generation method of the present invention in any of the above-mentioned aspects is executed by having a program executed on one computer or by having each step shared and executed by multiple computers, so that it is possible to obtain the same effects as the learning data generation method of the present invention described above, such as the effect of being able to easily add accurate annotations, and the effect of being able to improve learning efficiency and inspection accuracy in an inspection device that can detect defective products from multiple inspection objects. [Effects of the Invention]

[0037] According to the present invention, accurate annotation can be easily achieved in generating training data to be applied to an inspection device capable of detecting defective products from among multiple inspection objects. Furthermore, according to the present invention, it is possible to improve the efficiency of learning and the inspection accuracy of an inspection device capable of detecting defective products from among multiple inspection objects. [Brief explanation of the drawings]

[0038] [Figure 1] 1 is a diagram showing an outline of the configuration of an inspection device 1 according to an embodiment of the present invention. [Figure 2] 1 is a functional block diagram showing the functional configuration of a computer 10 that functions as a training data generation device according to an embodiment of the present invention. [Figure 3] FIG. 2 is an explanatory diagram showing an example of a background image Ibg. [Figure 4] FIG. 10 is an explanatory diagram showing an example of a non-defective image Igp. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a defective product image Idp. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a non-defective product extracted image Ige. [Figure 7] FIG. 10 is an explanatory diagram showing an example of a defective product extraction image Ide. [Figure 8] FIG. 2 is an explanatory diagram showing an example of a learning image IL. [Figure 9] FIG. 10 is an explanatory diagram showing the order in which non-defective products Pg and defective products Pd are superimposed. [Figure 10] FIG. 10 is an explanatory diagram showing the degree of overlap between a non-defective product Pg and a defective product Pd. [Figure 11] 10 is an explanatory diagram showing how defective product extraction images Ide are arbitrarily arranged in a learning image IL. FIG. [Figure 12] FIG. 10 is an explanatory diagram showing how a user (operator) places a defective product Pd in ​​an imaging area Ica. [Figure 13] 10 is a flowchart illustrating an example of a learning data generation process. [Figure 14] 10 is a flowchart illustrating an example of an imaging process. [Figure 15] 10 is a flowchart illustrating an example of a defective product detection process. [Figure 16] FIG. 2 is an explanatory diagram showing an example of an inspection image Ii. [Figure 17] FIG. 10 is a diagram showing an outline of the configuration of an inspection device 100 according to a modified example. [Figure 18] FIG. 10 is an explanatory diagram showing an example of a defective product image Idp obtained by capturing an image of a plurality of defective products Pd. [Figure 19] 10 is an explanatory diagram showing an example of a learning image IL in which a plurality of defective products Pd are arranged. FIG. [Figure 20] FIG. 10 is an explanatory diagram showing an example of a defective product image Idp obtained by capturing an image of a plurality of defective products Pd. [Figure 21] 10 is an explanatory diagram showing an example of a learning image IL generated by combining defective product extraction images Ide in an arbitrary arrangement. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0039] Next, the best mode for carrying out the present invention will be described using examples. [Example]

[0040] As shown in FIG. 1 , the inspection device 1 according to this embodiment includes a belt conveyor BC capable of transporting multiple inspection targets 90, a camera 70 capable of capturing images of the multiple inspection targets 90 transported by the belt conveyor BC, an illumination 72 that irradiates an imaging area Ica of the camera 70 with predetermined light (e.g., white light) from a predetermined direction, and a computer 10. The inspection device 1 captures images of the multiple inspection targets 90 transported by the belt conveyor BC using the camera 70 while irradiated with light from the illumination 72, and detects defective products Pd from among the multiple inspection targets 90 using the captured images of the multiple inspection targets 90. Here, the belt conveyor BC corresponds to a “transport unit” in the present invention, and the camera 70 and the illumination 72 are an example of an embodiment corresponding to an “imaging unit” in the present invention. The computer 10 is an example of an embodiment corresponding to a “learning data generation device” in the present invention.

[0041] As shown in FIG. 1, the computer 10 includes a microprocessor centered on a CPU 2, a ROM 4 for storing processing programs, a RAM 6 for temporarily storing data, a GPU 7 for performing calculations and matrix operations required for image processing, a hard disk (HDD) 8 which is a large-capacity memory for storing various application programs (simply referred to as programs) and various data including images, and an input / output interface (I / F) 9 for inputting and outputting data to and from external devices such as a camera 70.

[0042] The computer 10 is equipped with an input device 14 such as a keyboard and a mouse through which a user inputs various commands, and a display 60 that displays various information. The CPU 2, ROM 4, RAM 6, GPU 7, HDD 8, I / F 9, etc. are electrically connected by a bus 80, and are configured to be able to exchange various control signals and data with each other.

[0043] The computer 10 has a function of executing an operation corresponding to an input operation performed by a user via the input device 14 using a cursor or the like displayed on the display 60. The computer 10 also functions as the inspection device 1 according to this embodiment or as a learning data generation device by executing various processes using programs stored in the HDD 8, specifically, a learning data generation program that executes a learning data generation process for generating learning data used in training a trained model, and a defective product detection program that executes a defective product detection process for detecting defective products Pd from among multiple inspection targets 90. In this embodiment, the inspection device 1 or the learning data generation device is configured to be achievable by the computer 10, but may also be realized as a dedicated device.

[0044] As shown in FIG. 2, the computer 10 is configured with functional blocks such as an image acquisition unit 20, an image display control unit 22, an extracted image generation unit 24, a learning image generation unit 26, an annotation assignment unit 28, a timing determination unit 30, a feature extractor 32, a determiner 34, and a memory unit 36, through cooperation of one or both of the hardware resources mentioned above, such as a CPU 2, a ROM 4, a RAM 6, a GPU 7, a HDD 8, an I / F 9, an input device 14, a display 60, a camera 70, lighting 72, and a belt conveyor BC (see FIG. 1), and software such as a learning data generation program and a defective product detection program according to this embodiment. 1 (CPU 2, ROM 4, RAM 6, GPU 7, HDD 8, I / F 9, input device 14, display 60, camera 70, lighting 72, belt conveyor BC, etc.) operate independently or in cooperation with each other in response to instructions from CPU 2, which executes a program loaded from HDD 8 onto RAM 6. The image acquisition unit 20, image display control unit 22, extracted image generation unit 24, learning image generation unit 26, annotation assignment unit 28, timing determination unit 30, feature extractor 32, determiner 34, memory unit 36, etc. are electrically connected by bus lines 82, such as an address bus or a data bus.

[0045] The image acquisition unit 20 outputs an imaging instruction signal to the camera 70 and the lighting unit 72 to capture a background image Ibg (an image of only the background when the inspection object 90 is not present; see FIG. 3), a good-product image Igp (see FIG. 4) which is an image of only a plurality of good products Pg including the background bg, and a defective-product image Idp (see FIG. 5) which is an image of only at least one defective product Pd including the background bg, and acquires these images (background image Ibg, good-product image Igp, defective-product image Idp) captured by the camera 70, and supplies these acquired images (background image Ibg, good-product image Igp, defective-product image Idp) to the extracted image generation unit 24 and the storage unit 36. The image acquisition unit 20 also acquires a good-product extracted image Ige (FIG. 6) and a defective-product extracted image Ide (FIG. 7) generated by the extracted image generation unit 24, and supplies them to the learning image generation unit 26 and the storage unit 36. Furthermore, the image acquisition unit 20 acquires the learning images IL ( FIG. 8 ) generated by the learning image generation unit 26, and supplies them to the annotation assignment unit 28, the feature extractor 32, the determiner 34, and the storage unit 36. The image acquisition unit 20 also acquires inspection images Ii, which are images for detecting defective products Pd from among multiple inspection targets 90, and supplies them to the feature extractor 32, the determiner 34, and the storage unit 36. The image acquisition unit 20 also acquires the background image Ibg, the good-product image Igp, the defective-product image Idp, the extracted good-product image Ige, the extracted defective-product image Ide, the learning images IL, and the inspection images Ii from the storage unit 36 ​​as necessary.

[0046] The image display control unit 22 displays the determination result by the determiner on the display 60, and in response to a user operation, displays various images (background image Ibg, good-product image Igp, defective-product image Idp, extracted good-product image Ige, extracted defective-product image Ide, learning image IL, and inspection image Ii) on the display 60. When displaying the learning image IL on the display 60, the image display control unit 22 can also simultaneously display the annotations assigned by the annotation assigning unit 28 on the display 60.

[0047] The extracted image generation unit 24 processes the good product images Igp and defective product images Idp acquired by the image acquisition unit 20 to generate good product extracted images Ige and defective product extracted images Ide, which are images obtained by extracting only good products Pg and only defective products Pd, which are images obtained by removing the background from the good product images Igp and defective product images Idp, and supplies these to the memory unit 36.

[0048] The learning image generation unit 26 generates a learning image IL by combining the background image Ibg with the non-defective product extraction image Ige and the defective product extraction image Ide generated by the extraction image generation unit 24, and supplies the generated learning image IL to the storage unit 36. When generating the learning image IL, the learning image generation unit 26 may change the order of superimposition of the non-defective product extraction image Ige and the defective product extraction image Ide as shown in FIG. 9 (FIG. 9(a) is a diagram in which the non-defective product extraction image Ige is arranged above the defective product extraction image Ide, and FIG. 9(b) is a diagram in which the non-defective product extraction image Ige is arranged below the defective product extraction image Ide), or may change the degree of superimposition of the non-defective product extraction image Ige and the defective product extraction image Ide as shown in FIG. 10 (FIG. 10(a) is a diagram in which the non-defective product extraction image Ige is arranged below the defective product extraction image Ide). 10(a) is a diagram showing a state in which the degree of overlap between the good-product extraction image Ige and the defective-product extraction image Ide is increased, and FIG. 10(b) is a diagram showing a state in which the degree of overlap between the good-product extraction image Ige and the defective-product extraction image Ide is decreased), or the arrangement of the defective-product extraction image Ide in the learning image IL can be changed as shown in FIG. 11 (FIG. 11(a) is a diagram showing a state in which the defective-product extraction image Ide is arranged to the left in the learning image IL, and FIG. 11(b) is a diagram showing a state in which the defective-product extraction image Ide is arranged to the right in the learning image IL). The degree of overlap and the order of overlap between the good-product extraction image Ige and the defective-product extraction image Ide can also be automatically selected by random numbers.

[0049] The annotation assigning unit 28 assigns annotations to the learning images IL generated by the learning image generation unit 26. Specifically, when synthesizing the learning images IL from the background image Ibg, the good-product extraction images Ige, and the defective-product extraction images Ide, the annotation assigning unit 28 automatically assigns annotations based on the positions of the defective-product extraction images Ide in the background image Ibg and the good-product extraction images Ige. More specifically, the annotation assigning unit 28 assigns annotations by automatically identifying the positions of the defective-product extraction images Ide in the learning images IL and automatically inputting an indication that the defective products Pd are defective (for example, a "mark") into the learning images IL. The annotations assigned by the annotation assigning unit 28 are stored together with the learning images IL in the storage unit 36.

[0050] When the camera 70 captures an image of the defective product Pd, the timing determination unit 30 determines whether it is time to capture an image, and if it determines that it is time to capture an image, outputs an image capture instruction signal to the camera 70. In this embodiment, as shown in FIGS. 3 and 12 , the determination of whether it is time to capture an image is performed by comparing the image capture area Ica of the camera 70 with the background image Ibg and determining whether there is a difference at least in the outer periphery Pe. More specifically, the timing determination unit determines that it is time to capture an image when there is no difference between at least the outer periphery Pe of the image capture area Ica and at least the outer periphery Pe of the background image Ibg. On the other hand, if there is a difference between at least the outer periphery Pe of the image capture area Ica and at least the outer periphery Pe of the background image Ibg, the timing determination unit determines whether the state of the difference changes over a predetermined period of time. If the state of the difference does not change over a predetermined period of time, it determines that it is time to capture an image. If the state of the difference changes over a predetermined period of time, it determines that it is not time to capture an image, assuming that an unwanted object, such as a worker's hand, is captured in the image. When the timing determination unit 30 determines that it is time to capture an image, it outputs an image capture instruction signal to the camera 70. Here, in this embodiment, the predetermined time is set to be equal to or longer than the time required for the worker to place the defective product Pd.

[0051] The feature extractor 32 extracts features from the training images IL and the inspection images Ii. In this embodiment, machine learning, more specifically, deep metric learning, is used to modify calculation parameters and perform learning so that features capable of accurately detecting defective products Pd from among multiple inspection targets 90 can be extracted. That is, the feature extractor 32 applies deep metric learning to a series of processes that calculates the distance between the extracted features and modifies parameters using backpropagation and gradient descent based on the calculated distance. The feature extractor 32 performs learning in advance using multiple training images IL to which annotations have been added (with teacher signals), and the resulting learning model is stored in the storage unit 36 ​​as a single trained model ML.

[0052] The determiner 34 determines whether or not there is a defective product Pd among the multiple inspection targets 90, based on the features extracted by the feature extractor 32. The determination result is displayed on the display 60. The determiner 34 can use so-called statistical machine learning, which is machine learning other than machine learning using a neural network including deep learning, such as gradient boosting, support vector machine, random forest, neural network, Gaussian normalization, or ensemble testing.

[0053] The storage unit 36 ​​is secured in at least one of the RAM 6 and the HDD 8, and stores the background image Ibg, the good-product image Igp, the defective-product image Idp, and the inspection image Ii captured by the camera 70, the good-product extraction image Ige and the defective-product extraction image Ide generated by the extraction image generation unit 24, the training images IL generated by the training image generation unit 26, and the trained model ML. The training images IL are stored together with annotations assigned by the annotation assignment unit.

[0054] Next, a description will be given of a learning data generation process performed in the computer 10 as the learning data generation device according to this embodiment when a learning data generation program is launched. FIG. 13 is a flowchart showing an example of the learning data generation process. The learning data generation process is executed, for example, when a user presses a "Generate Learning Data" button from among various command buttons displayed on the display 60. The learning data generation process is mainly executed by the image acquisition unit 20, image display control unit 22, extracted image generation unit 24, learning image generation unit 26, annotation assignment unit 28, timing determination unit 30, and storage unit 36 ​​of the computer 10. Note that this embodiment will be described taking as an example a case where an undesired item (foreign object) is detected from among a plurality of inspection targets 90. Therefore, hereinafter, a "good item" refers to a desired item, and a "defective item" refers to an undesired item (foreign object).

[0055] When the learning data generation process is executed, as shown in FIG. 13, first, the image acquisition unit 20 executes a process of capturing images of a background bg, a good product Pg, and a defective product Pd (step S10). The capturing is performed based on the capturing process shown in FIG. 14. When this process is executed, the image acquisition unit 20 first executes a process of determining whether the captured image object is the background bg, a good product Pg, or a defective product Pd (step S20). The determination of the captured image object can be made based on, for example, whether the user (operator) selects "capture background," "capture good product," or "capture defective product" from various command buttons.

[0056] That is, when the user (operator) selects "imaging background" from the various command buttons, the image acquisition unit 20 determines that the image target is the background bg, when the user (operator) selects "imaging good products" from the various command buttons, the image acquisition unit 20 determines that the image target is a good product Pg, and when the user selects "image of defective products" from the various command buttons, the image acquisition unit 20 determines that the image target is a defective product Pd. Note that in this embodiment, in the case of "imaging good products," multiple good products Pg that have been pre-selected from multiple inspection targets 90 are placed on the belt conveyor BC upstream of the imaging area Ica in the conveying direction and are transported to the imaging area Ica by the belt conveyor BC, and in the case of "imaging defective products," defective products Pd that have been pre-selected from multiple inspection targets 90 are placed one by one in the imaging area Ica of the belt conveyor BC for each imaging.

[0057] If it is determined in step S20 that the imaging target is the background bg or a non-defective product Pg, an imaging instruction signal is output to the camera 70 to image the background bg or the non-defective product Pg, and the captured background image Ibg (FIG. 3) or non-defective product image Igp (FIG. 4) is stored (step S22), and this process ends. Here, in this embodiment, imaging of the background bg is performed immediately after the user (operator) presses the "image background" button. Also, imaging of the non-defective product Pg is performed when the non-defective product Pg is transported to the imaging area Ica after the user (operator) presses the "image non-defective product" button.

[0058] On the other hand, if it is determined in step S20 that the image target is a defective product Pd, a process is executed to determine whether it is time to take an image (step S24). The determination of whether it is time to take an image can be made based on whether there is an output from the timing determination unit 30. In this way, by determining whether it is time to take an image using the timing determination unit 30 and taking an image only when it is determined that it is time to take an image, it is possible to effectively prevent the hands of a user (worker) from being captured when placing the defective product Pd on the belt conveyor BC. This makes it possible to capture an appropriate defective product image Idp.

[0059] If it is determined in step S24 that it is time to take an image, an image capture instruction signal is output to the camera 70 to capture an image of the defective product Pd, and the captured defective product image Idp (FIG. 5) is stored (step S22), and this process ends. Note that if it is determined in step S24 that it is not time to take an image, the determination in step S24 is repeatedly executed until it is time to take an image.

[0060] When the imaging process is completed in this manner, the image acquisition unit 20 executes a process of reading the background image Ibg, the good product image Igp, and the defective product image Idp captured by the imaging process (step S12). Subsequently, the extracted image generation unit 24 executes a process of generating a good product extracted image Ige (FIG. 6) and a defective product extracted image Ide (FIG. 7) from the read good product image Igp (FIG. 4) and defective product image Idp (FIG. 5) by image processing (step S14).

[0061] Then, the learning image generation unit 26 executes a process of generating a learning image IL (FIG. 8) by combining the loaded background image Ibg (FIG. 3) with the good product extraction image Ige (FIG. 6) and the defective product extraction image Ide (FIG. 7) generated by the extraction image generation unit 24 (step S16), and the annotation assignment unit 28 assigns an annotation to the learning image IL (step S18), thereby terminating this process.

[0062] As described above, the training data generation device according to this embodiment is configured to generate training images IL by combining the background image Ibg, the non-defective product extraction images Ige, and the defective product extraction images Ide, making it easy to identify the arrangement of the defective product extraction images Ide in the training images IL. Furthermore, because annotations are automatically added based on the identified arrangement of the defective product extraction images Ide, accurate annotations can be easily achieved in generating training data to be applied to an inspection device capable of detecting defective products Pd from multiple inspection targets 90.

[0063] In this way, the feature extractor 32 can be trained using the training images IL with accurate annotations, and a trained model ML can be constructed that can accurately detect defective products Pd from multiple inspection objects 90 (learning efficiency can be improved).

[0064] Next, the operation of the inspection device 1 according to this embodiment, particularly the operation when detecting a defective product Pd from among a plurality of inspection targets 90 using the trained model ML, will be described. Fig. 15 is a main flowchart showing an example of the defective product detection process. The defective product detection process is performed when the defective product detection program is started, and is executed mainly by the image acquisition unit 20, image display control unit 22, feature extractor 32, determiner 34, and storage unit 36 ​​of the computer 10.

[0065] 15, when the defective product detection process is executed, first, the image acquisition unit 20 outputs an imaging instruction signal to the camera 70 to capture images of the plurality of inspection objects 90 (step S30), and executes a process of reading the captured inspection image Ii (step S32). Here, the inspection image Ii is an image obtained by capturing a background bg and the plurality of inspection objects 90 including a plurality of non-defective products Pg and defective products Pd (there may be cases where the defective products Pd are not included), as shown in FIG.

[0066] When the inspection image Ii is read, the feature extractor 32 executes a process of extracting features from the inspection image Ii using the trained model ML (step S34). Next, the determiner 34 determines whether or not a defective product Pd exists among the multiple inspection objects 90 based on the extracted features (step S36), and the image display control unit 22 executes a process of outputting the determination result (step S38), thereby terminating this process.

[0067] In this way, according to the inspection device 1 of this embodiment, the trained model ML, which has been properly trained using training data with accurate annotations, is used to extract features of the inspection image Ii, and judgment is made based on the extracted features, so that it is possible to accurately detect whether or not there are defective products Pd among multiple inspection objects 90.

[0068] In this embodiment, when generating the learning image IL, the defective product Pd is imaged only once while the defective product Pd maintains a constant orientation (i.e., the orientation is not changed). However, this is not limiting. For example, the defective product Pd may be imaged multiple times while the orientation of the defective product Pd is changed. In this case, as shown in the modified inspection device 100 of FIG. 17, a vibration device 92 such as a vibration motor may be mechanically connected to the belt conveyor BC, and the vibration device 92 may vibrate the defective product Pd on the belt conveyor BC. This configuration allows multiple defective product images Idp with different orientations to be generated for the same defective product Pd, making it easy to obtain multiple learning images IL annotated (with teacher signals). This allows for efficient improvement of the detection accuracy during training of the trained model ML. It is sufficient that the posture of the defective product Pd is changed at least in the imaging area Ica. For example, a configuration may be adopted in which the vibration device 92 vibrates the defective product Pd on the belt conveyor BC only when the defective product Pd reaches the imaging area Ica. Furthermore, imaging of not only the defective product Pd but also the non-defective products Pg may be performed multiple times while changing the posture of the non-defective products Pg. Here, the multiple defective product images Idp with different postures correspond to the "defective product image of changed posture" in this invention, and an image of only the defective product Pd with the background bg removed from the defective product image Idp (defective product extraction image Ide) is an example of an embodiment corresponding to the "defective product extraction image of changed posture" in this invention.

[0069] In the present embodiment and the modified example described above, a defective product image Idp in which only one defective product Pd is captured is used to generate a learning image IL, but this is not limiting. For example, as shown in FIG. 18, a defective product image Idp in which multiple defective products Pd are captured may be used. In this case, as shown in FIG. 19, a learning image IL in which multiple defective products Pd are arranged is generated, and annotations are added to the locations of the defective products Pd. Note that, as shown in FIG. 20, multiple defective product extraction images Ide are generated from a defective product image Idp in which multiple defective products Pd are captured. When generating the learning image IL, the multiple defective product extraction images Ide may be synthesized in the same arrangement as in the defective product image Idp (see FIG. 19), or may be synthesized in an arbitrary arrangement as shown in FIG. 21.

[0070] In the present embodiment, a predetermined imaging mode is used, that is, an imaging area Ica illuminated with predetermined light (e.g., white light) from a predetermined direction by an illumination 72 fixed at a predetermined position is imaged by a camera 70 fixed at a predetermined position in a predetermined imaging mode. However, the imaging mode may be changed and an image may be captured each time the imaging mode is changed. Possible different imaging modes include, for example, a configuration in which the illumination 72 sequentially switches between irradiating with a plurality of patterned lights (e.g., irradiation from a plurality of angles, or a plurality of grid-like or striped lights) and the camera 70 captures an image each time the patterned light is switched, a configuration in which the illumination 72 sequentially switches between irradiating with light of different wavelengths and the camera 70 captures an image each time the wavelength is switched, a configuration in which the illumination direction of the illumination 72 is changed and the camera 70 captures an image each time the illumination direction is changed, a configuration in which the imaging direction of the camera 70 is changed and the camera 70 captures an image each time the imaging direction is changed, or a configuration in which the imaging mode (e.g., exposure time) of the camera 70 is changed and the camera 70 captures an image each time the imaging mode is changed. Possible ways to vary the illumination direction include changing the angle of the illumination 72, changing the position of the illumination 72 itself, or using multiple illuminations 72. Possible ways to vary the imaging direction include changing the angle of the camera 70, changing the position of the camera 70 itself, or using multiple cameras 70. In this case, when generating the training images IL, it is of course necessary to combine the background image Ibg, the good-product image Igp, and the defective-product image captured in the same imaging mode. This configuration allows multiple training images to be generated without preparing multiple defective products, thereby improving learning efficiency. Furthermore, compared to preparing and imaging multiple defective products, the labor and time required for imaging can be reduced, thereby reducing costs. By comparing learning results using training images generated for different imaging modes, it is easy to determine which imaging mode contributes to improving inspection accuracy. This allows training images to be generated by selecting an imaging mode that contributes to improving inspection accuracy, thereby improving learning efficiency and inspection accuracy.Here, the background image Ibg, the good product image Igp and the defective product image Idp captured in different imaging modes correspond to the "background image with changed imaging mode", the "good product image with changed imaging mode" and the "defective product image with changed imaging mode" in the present invention, respectively, and an image of only the good product Pg obtained by removing the background bg from the good product image Igp captured in a different imaging mode (good product extracted image Ige), and an image of only the defective product Pd obtained by removing the background bg from the defective product image Idp captured in a different imaging mode (defective product extracted image Ide) are examples of implementation configurations corresponding to the "good product extracted image with changed imaging mode" and the "defective product extracted image with changed imaging mode" in the present invention, respectively.

[0071] In this embodiment, the present invention is applied to a case where an undesired item (foreign matter) is detected from among a plurality of inspection objects 90, but the present invention is not limited to this. For example, the present invention can be applied to a case where a defective item is detected from among a plurality of inspection objects 90.

[0072] In this embodiment, various programs including the defective product detection program and the learning data generation program are configured to be stored in the HDD 8, but this is not limiting. For example, various programs including the defective product detection program and the learning data generation program may be distributed from another computer to the computer 10 serving as the inspection device according to this embodiment via a transmission medium, for example, a communication network such as the Internet or a LAN.

[0073] The present embodiment shows an example of a mode for carrying out the present invention, and therefore the present invention is not limited to the configuration of the present embodiment. [Explanation of symbols]

[0074] 1 Inspection equipment (inspection equipment) 2 CPU 4 ROM 6 RAM 7 GPU 8 HDD 9 Input / Output Interface 10 Computer (learning data generation device) 14 Input Devices 20 Image acquisition unit (image acquisition unit) 22 Image display control unit 24 Extraction image generation unit (good product extraction image generation unit, defective product extraction image generation unit) 26 Learning image generation unit (Learning image generation unit) 28 Annotation section (annotation section) 30 Timing judgment unit (timing judgment unit) 32 Feature Extractor (Feature Extractor) 34 Determiner (determiner) 36 Memory section 60 displays 70 Camera (imaging unit) 72 Lighting (imaging unit) 80 Bus 82 Bus Line 90 Inspection subject (inspection subject) 92 Vibration device 100 Inspection equipment (inspection equipment) BC Belt Conveyor ML trained model (trained model) bg background (background) Pg Good product (Good product) Pd Defective product (defective product) Ibg background image (background image) Igp Good Product Image (Good Product Image) IDP defective product image (defective product image) IL Learning Images (Learning Images) Ige Good product extracted image (Good product extracted image) Ide Defective Product Extraction Image (Defective Product Extraction Image) Ii Inspection image (inspection image) Ica imaging area (imaging area) Pe outer periphery (outer periphery)

Claims

1. A learning data generation device capable of generating learning data to be applied to an inspection device capable of detecting defective products from among a plurality of inspection objects, an image acquisition unit capable of acquiring a plurality of non-defective product images, which are images of only non-defective products including a background, and a defective product image, which is an image of only at least one of the defective products including the background; a defective product extraction image generation unit that generates a defective product extraction image, which is an image obtained by extracting only the image of the defective product from the defective product image and excluding the background; a learning image generating unit that generates a learning image by combining the non-defective product image and the defective product extraction image; an annotation adding unit that adds annotations to the learning images based on the arrangement of the defective product extracted images in the learning images; A learning data generation device comprising:

2. a non-defective product extracted image generating unit that generates a non-defective product extracted image, which is an image obtained by extracting only the images of the non-defective products from the non-defective product images and excluding the background; the image acquisition unit is capable of acquiring a background image that is an image of only the background, The learning image generation unit generates the learning image by synthesizing the background image, the good product extraction image, and the defective product extraction image, and is capable of changing the order in which the good product extraction image and the defective product extraction image are superimposed when generating the learning image. The learning data generating device according to claim 1 .

3. a non-defective product extracted image generating unit that generates a non-defective product extracted image, which is an image obtained by extracting only the images of the non-defective products from the non-defective product images and excluding the background; the image acquisition unit is capable of acquiring a background image that is an image of only the background, The learning image generating unit generates the learning image by synthesizing the background image, the good product extracted image, and the defective product extracted image, and is capable of changing the degree of overlapping between the good product extracted image and the defective product extracted image when generating the learning image. The learning data generating device according to claim 1 .

4. The learning image generating unit is capable of changing the degree of overlap between the non-defective product extracted image and the defective product extracted image when generating the learning image. The learning data generating device according to claim 2 .

5. a non-defective product extracted image generating unit that generates a non-defective product extracted image, which is an image obtained by extracting only the images of the non-defective products from the non-defective product images and excluding the background; the image acquisition unit is capable of acquiring a background image that is an image of only the background, The learning image generating unit generates the learning image by synthesizing the background image, the good product extracted image, and the defective product extracted image, and when generating the learning image, the defective product extracted image can be positioned at an arbitrary position relative to the good product extracted image. The learning data generating device according to claim 1 .

6. The learning image generating unit can arrange the defective product extracted image at an arbitrary position relative to the non-defective product extracted image when generating the learning image. The learning data generating device according to any one of claims 2 to 4.

7. Further, a timing determination unit is provided which can determine the timing of capturing at least the defective product image, The timing determination unit compares an imaging area with the background image at least when imaging the defective product image, and determines that it is the imaging timing when there is no difference between the imaging area and the background image at least in the outer periphery of the imaging area. The learning data generating device according to any one of claims 2 to 5.

8. When there is a difference between the image and the background image at least in the outer periphery of the imaging area, the timing determination unit determines whether the difference changes over a predetermined time period, and if there is no change, determines that it is the imaging timing, and if there is a change, determines that it is not the imaging timing. The learning data generating device according to claim 7 .

9. the image acquisition unit is capable of acquiring a changed posture defective product image, which is an image of the defective product captured in a different posture, The defective product extracted image generating unit is capable of generating a changed-posture defective product extracted image, which is an image obtained by extracting only an image of the defective product excluding the background from the changed-posture defective product image. The learning image generating unit is capable of generating a learning image by combining the background image, the non-defective product extracted image, and the changed-posture defective product extracted image. The learning data generating device according to any one of claims 2 to 5.

10. the image acquisition unit is capable of acquiring an imaging mode-changed background image, an imaging mode-changed non-defective product image, and an imaging mode-changed defective product image, which are images of the background, the non-defective product, and the defective product captured in different imaging modes; the defective product extraction image generation unit is capable of generating an imaging mode-changed defective product extraction image, which is an image obtained by extracting only the image of the defective product excluding the imaging mode-changed background image from the imaging mode-changed defective product image, the non-defective product extracted image generation unit is capable of generating an imaging mode-changed non-defective product extracted image, which is an image obtained by extracting only the image of the non-defective product excluding the imaging mode-changed background image from the imaging mode-changed non-defective product image, The learning image generating unit is capable of generating a learning image by combining the imaging mode-changed background image, the imaging mode-changed non-defective product extraction image, and the imaging mode-changed defective product extraction image. The learning data generating device according to any one of claims 2 to 5.

11. An inspection device capable of detecting defective products from among a plurality of inspection objects, an imaging unit capable of capturing test images, which are images of the plurality of test objects; a trained model that has been trained in advance using training data generated by the training data generation device according to any one of claims 1 to 5; a feature extractor that extracts features from the inspection image using the trained model; a determiner that determines whether or not the defective product is present among the plurality of inspection objects based on the feature amount; An inspection device comprising:

12. The inspection system further includes a transport unit capable of transporting at least the defective product to an imaging area, and a vibration unit capable of applying vibration to at least the defective product in the imaging area via the transport unit, the imaging unit is capable of imaging the defective product whose posture has been changed by the vibration of the vibration unit, the image acquisition unit is capable of acquiring a changed posture defective product image, which is an image of the defective product captured in a different posture, The defective product extracted image generating unit is capable of generating a changed-posture defective product extracted image, which is an image obtained by extracting only an image of the defective product excluding the background from the changed-posture defective product image. The learning image generating unit is capable of generating a learning image by combining the background image, the non-defective product extracted image, and the changed-posture defective product extracted image.

12. The inspection device according to claim 11, dependent on claim 2.

13. The inspection system further includes a transport unit capable of transporting at least the defective product to an imaging area, and a vibration unit capable of applying vibration to at least the defective product in the imaging area via the transport unit, the imaging unit is capable of imaging the defective product whose posture has been changed by the vibration of the vibration unit, the image acquisition unit is capable of acquiring a changed posture defective product image, which is an image of the defective product captured in a different posture, The defective product extracted image generating unit is capable of generating a changed-posture defective product extracted image, which is an image obtained by extracting only an image of the defective product excluding the background from the changed-posture defective product image. The learning image generating unit is capable of generating a learning image by combining the background image, the non-defective product extracted image, and the changed-posture defective product extracted image.

12. The inspection device according to claim 11, when dependent on claim 3.

14. the imaging unit is capable of imaging the background, the non-defective product, and the defective product in different imaging modes; the image acquisition unit is capable of acquiring an imaging mode-changed background image, an imaging mode-changed non-defective product image, and an imaging mode-changed defective product image, which are images of the background, the non-defective product, and the defective product, respectively, captured in different imaging modes; the defective product extraction image generation unit is capable of generating an imaging mode-changed defective product extraction image, which is an image obtained by extracting only the image of the defective product excluding the imaging mode-changed background image from the imaging mode-changed defective product image, the non-defective product extracted image generation unit is capable of generating an imaging mode-changed non-defective product extracted image, which is an image obtained by extracting only the image of the non-defective product excluding the imaging mode-changed background image from the imaging mode-changed non-defective product image, The learning image generating unit is capable of generating a learning image by combining the imaging mode-changed background image, the imaging mode-changed non-defective product extraction image, and the imaging mode-changed defective product extraction image.

14. The inspection device according to claim 12 or 13.

15. A learning data generation method capable of generating learning data to be applied to an inspection device capable of detecting defective products from among a plurality of inspection objects, comprising: (a) acquiring a plurality of non-defective product images that are images of only the non-defective products including a background, and at least one defective product image that is an image of only the defective product including the background; (b) generating a defective product extracted image, which is an image obtained by extracting only the image of the defective product excluding the background from the acquired defective product image; (c) generating a learning image by combining the non-defective product image and the defective product extraction image; (d) generating the learning data by adding annotations to the generated learning images; Method for generating training data.

16. generating a non-defective product extracted image, which is an image obtained by extracting only the images of the non-defective products excluding the background from the non-defective product image; The step (a) includes a step of obtaining a background image that is an image of only the background; The step (c) includes a step of generating the learning image by combining the background image, the good product extraction image, and the defective product extraction image, and changing the order of superimposition of the good product extraction image and the defective product extraction image when generating the learning image. The learning data generation method according to claim 15.

17. generating a non-defective product extracted image, which is an image obtained by extracting only the images of the non-defective products excluding the background from the non-defective product image; The step (a) includes a step of obtaining a background image that is an image of only the background; The step (c) includes a step of generating the learning image by synthesizing the background image, the non-defective product extracted image, and the defective product extracted image, and a step of changing the degree of overlap between the non-defective product extracted image and the defective product extracted image when generating the learning image. The learning data generation method according to claim 15.

18. A program for generating learning data to be applied to an inspection device capable of detecting defective products from among a plurality of inspection objects, A program for causing one or more computers to execute each step of the learning data generation method according to any one of claims 15 to 17.

Images