Component inspection device

Abstract

A component inspection apparatus includes: a storage unit that stores at least images of good products; a generation unit that generates images of defective products using a machine learning model; a setting unit that allows a user to set parameters for component inspection; and an output unit that uses the parameters to inspect the images of good products stored in the storage unit and the images of defective products generated by the generation unit, and outputs inspection results.

Description

Technical Field

[0001] This invention relates to a component inspection device. Background Technology

[0002] A component inspection device is used to check the mounting condition of mounting components soldered to a printed circuit board. In such a component inspection device, threshold tuning of the inspection logic is generally implemented to suppress false detections (over-detection, under-detection) (Patent Document 1, etc.).

[0003] The adjustment process is conducted as follows: Good and defective product images actually generated during the inspection process are registered as model images. Inspection logic is virtually applied to these model images (a process called model testing). Thresholds are manually adjusted to ensure accurate detection of both good and defective product images. Defective product images are indispensable in the adjustment process. To accumulate defective product images while repeatedly performing subtle threshold adjustments to refine the inspection logic, a significant amount of time and skilled techniques are required.

[0004] However, with recent technological innovations, the performance of surface-mount manufacturing equipment has improved dramatically, and the defect rate has decreased to the level of a few ppm. Therefore, collecting actual images of defective products is not easy. Especially in the case of newly set up production lines, defective product images are almost non-existent, making it difficult to adjust thresholds appropriately and increasing the risk of false detections.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2018-77147 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] The purpose of this invention is to provide a technique for assisting in the effective performance of threshold adjustment in a component inspection device.

[0010] Methods for solving problems

[0011] This invention generates defective product images using a machine learning model and performs teaching processing using these images, enabling effective adjustment of thresholds for component inspection. More specifically, one aspect of the component inspection apparatus of this invention includes: a storage unit storing at least good product images; a generation unit generating defective product images using a machine learning model; a setting unit capable of user-defined parameters for component inspection; and an output unit using the parameters to inspect the good product images stored in the storage unit and the defective product images generated by the generation unit, and outputting inspection results.

[0012] In this way, by using machine learning techniques to generate defective product images, the number of defective product images that can be used for teaching processing can be increased, and the threshold can be adjusted appropriately.

[0013] In this invention, the generation component can also generate defective images by applying a machine learning model to at least one of the good product images stored in the storage component. The machine learning model can, for example, utilize a generative model based on a generative adversarial network (GAN).

[0014] In this way, by generating defective product images based on good product images, it is possible to generate defective product images that are similar to good product images. That is, it is possible to generate images with the same defects that may actually occur, thus enabling appropriate teaching processing.

[0015] In this invention, a designation unit may also be included, which allows a user to specify the type of defect in the defective product image generated by the generation unit. The generation unit generates a defective product image having the type of defect specified by the designation unit. The designation unit may display a graphical user interface (GUI) such as a designation screen that allows the user to specify multiple feature quantities representing the defect, using the GUI to allow the user to specify the type of defect to be generated. This designation screen may be configured to allow the user to specify a region within a feature space composed of multiple feature quantities, and then display an image representing the defect corresponding to the feature space in at least one of the regions within the feature space. Alternatively, it may be configured so that the user can independently specify the values ​​of multiple feature quantities instead of specifying a region within the feature quantity space.

[0016] Based on this structure, it is possible to generate images of defective products that the user deems necessary for the teaching process. This allows for threshold adjustments that enable more appropriate inspection.

[0017] In this invention, a suggestion unit may also be included, which suggests the types of defective product images that should be generated. For example, the suggestion unit suggests generating defective product images of a specific type when the number of registered defective product images of that type is small. Specifically, the suggestion unit can generate defective product images for multiple types of defects through the generation unit, perform clustering processing to classify the generated defective product images into multiple clusters, determine which cluster each defective product image stored in the storage unit belongs to, and suggest that defects corresponding to clusters with a predetermined number of existing defective product images be considered as the types of defective product images that should be generated. Furthermore, when generating defective product images corresponding to a certain cluster, random noise can be added to generate images containing unspecified defects within the cluster.

[0018] This structure enables users to generate insufficient images of defective products, allowing for more appropriate threshold adjustments.

[0019] In this invention, the output unit may output an image of the object to be inspected, information indicating whether the image of the object to be inspected was generated by the generation unit, and information indicating whether the object was determined to be a good product or a defective product using the parameters as the inspection result.

[0020] By displaying such inspection results, users can make appropriate threshold adjustments or generate images of insufficient defective products.

[0021] Another aspect of the present invention is an auxiliary method for assisting in setting parameters for component inspection in a component inspection apparatus, comprising: a generation step, using a machine learning model to generate defective product images from stored good product images; a setting step, receiving parameter settings for component inspection from a user; and an output step, using the parameters to perform inspections on the stored good product images and the generated defective product images, and outputting inspection results.

[0022] The present invention can also be understood as a method comprising at least a portion of the above-described processes, or a program for implementing the method, or a recording medium for non-transitory recording of the program. Furthermore, the aforementioned units and processes can be combined with each other as much as possible to constitute the present invention.

[0023] Invention Effects

[0024] According to the present invention, it is possible to assist in the efficient performance of threshold adjustment operations in a component inspection device. Attached Figure Description

[0025] [ Figure 1 ] Figure 1 This is a diagram showing the functional blocks of the component inspection device according to the first embodiment.

[0026] [ Figure 2 ] Figure 2 This is a diagram showing the hardware structure of the component inspection device.

[0027] [ Figure 3 ] Figure 3 This is a diagram showing the overall flow of the teaching process (threshold adjustment process) performed by the component inspection device.

[0028] [ Figure 4 ] Figure 4 This is a diagram illustrating the process of generating and processing images of defective products.

[0029] [ Figure 5 ] Figure 5 This is an example of the output of results for model testing and the screen for threshold adjustment used in the teaching process.

[0030] [ Figure 6 ] Figure 6 This is an example of a screen used to specify the image of a defective product generated in the defective product image generation process.

[0031] [ Figure 7 ] Figure 7 This is an example of a screen used to specify the image of a defective product generated in the defective product image generation process.

[0032] [ Figure 8 ] Figure 8 This is an example of a screen used to specify the image of a defective product generated in the defective product image generation process.

[0033] [ Figure 9 ] Figure 9 (A) is a diagram showing the functional blocks of the component inspection device according to the second embodiment. Figure 9 (B) is an example of a specified screen used to specify the generated defective product image.

[0034] [ Figure 10 ] Figure 10 (A) and Figure 10 (B) is a diagram illustrating the process of making suggestions for the generated defects in the second embodiment. Detailed Implementation

[0035] <Application Examples>

[0036] Reference Figure 1 One example of the application of the present invention will be described below. Figure 1 The component inspection apparatus 1, which incorporates the present invention, is characterized by an auxiliary technique for appropriately adjusting parameters (adjustment work) for component inspection. Good product image data and defective product image data generated on the actual production line and captured by the imaging unit 10 are stored in the storage unit 40. With advancements in manufacturing technology in recent years, defect rates have decreased, making it difficult to collect necessary defective product images. Therefore, in this invention, the defective product image generation unit 20 uses machine learning technology to generate defective product images based on the good product image data 41. The user specifies the type of defect to be generated via the designation unit 21. Thus, it is possible to generate defective product images that resemble actual good product images and contain the types of defects specified by the user.

[0037] By using the generated defective product images, along with the actual captured images of good and defective products, and adjusting the threshold using the teaching unit 30, the user can make appropriate adjustments.

[0038] <First Embodiment>

[0039] Reference Figures 1-5 The component inspection device 1 according to the embodiments of the present invention will be described in detail. Figure 1 This is a diagram showing the functional blocks of the component inspection device 1. Figure 2 This is a diagram showing the hardware structure of the component inspection device 1. Figure 3 This is a diagram showing the overall flow of the teaching process (threshold adjustment process) performed by the component inspection device 1. Figure 4 This is a diagram illustrating the process of generating and processing images of defective products. Figures 5-8 This is an example of a display screen used in teach processing. Figure 5 The main focus is on the results of model testing and the visuals used for threshold adjustment. Figures 6-8 It is used to specify the screen for generating defective product images in the defective product image generation process.

[0040] The component inspection device 1 also has the function of actually inspecting components based on the teaching results, but the following mainly describes the functions related to the teaching process of the component inspection device 1.

[0041] (structure)

[0042] like Figure 1 As shown, the component inspection device 1 includes an imaging unit 10, a defective product image generation unit 20, a teaching unit 30, and a storage unit 40.

[0043] The imaging unit 10 photographs components on the production line (e.g., components mounted on printed circuit boards). The images selected by the user from the captured images are stored as model images in the storage unit 40, labeled as good or bad product images.

[0044] The defective product image generation unit 20 is a functional unit that generates defective product images using machine learning technology. The defective product image generation unit 20 includes: a specifying unit 21, which is used to specify the type of defect in the defective product image to be generated from the user; and a generation unit 22, which is used to generate the specified defective product image.

[0045] The designation unit 21 displays a GUI screen (user interface) that allows the user to specify the type of defect to be generated, and accepts input from the user.

[0046] The generation unit 22 applies a machine learning model to the good product images stored in the storage unit 40 to generate defective product images with the types of defects specified by the designation unit 21. The machine learning model used by the generation unit 22 is, for example, a generative model based on a Generative Adversarial Network (GAN). In a GAN, a generator that generates images similar to the correct solution image and a recognizer that identifies whether an image is a correct solution image or a generated image are learned. The generator is learned to generate images that are closer to the correct solution, and the recognizer is learned to more accurately identify correct solution images and generated images. The generation unit 22 uses the generator to generate and output defective product images with the specified types of defect features transferred to the actual good product images. In this way, defective product images can be generated as defective product images with the specified types of defects for the same parts as the good product images, i.e., defective product images with a high probability of occurring in reality.

[0047] The teaching unit 30 is a functional unit used by the user to set parameters (thresholds) for component inspection. The teaching unit 30 includes an inspection unit 31, a result output unit 32, and a threshold setting unit 33. The inspection unit 31 uses the currently set logic and thresholds to inspect (test) the model image stored in the storage unit 40. Hereinafter, the test of the model image will also be referred to as model testing. The result output unit 32 displays the inspection results of the inspection unit 31 to the user. The inspection results include an image of the component being inspected, information indicating whether the image of the inspected component was generated by the defective product image generation unit 20, and information indicating whether it was determined to be a good product or a defective product using the current threshold, as detailed later.

[0048] The storage unit 40 stores good product image data 41, defective product image data 42, and inspection threshold 43. Good product image data 41 is image data actually captured by the imaging unit 10. Defective product image data 42 includes image data actually captured by the imaging unit 10 and image data generated by the defective product image generation unit 20, and also stores information indicating whether it is actually captured image data or generated image data. The user specifies which model image to register as a good product image or a defective product image. The inspection threshold 43 is a threshold used for component inspection logic, which the user can change using the teaching unit 30.

[0049] Figure 2 This is a diagram showing the hardware structure of component inspection device 1. For example... Figure 2As shown, the component inspection device 1 has the same structure as a general computer (information processing device), including a CPU (Central Processing Unit) 51, ROM (Read-Only Memory) 52, RAM (Random Access Memory) 53, storage device 54, keyboard 55, mouse 56, monitor 57, and communication interface 58. The various components are connected to each other via a bus 59 in a manner that enables them to communicate with each other.

[0050] In this embodiment, a learning program for performing learning processing of the learning model is stored in ROM 52 or storage device 54. CPU 51 is a central processing unit that executes various programs or controls various structures. That is, CPU 51 reads the program from ROM 52 or storage device 54 and uses RAM 53 as a working area to execute the program. CPU 51 performs control of the aforementioned structures and various arithmetic processing according to the program recorded in ROM 52 or storage device 54. ROM 52 stores various programs and various data. RAM 53 temporarily stores programs or data as a working area. Storage device 54 is composed of HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, storing various programs, including the operating system, and various data. Keyboard 55 and mouse 56 are examples of input devices used for various inputs. Monitor 57 is, for example, a liquid crystal display (LCD) that displays the user interface. Monitor 57 can also be a touch panel, functioning as an input unit. Communication interface 58 is an interface used to communicate with other devices, such as using standards such as Ethernet (registered trademark), FDDI, or Wi-Fi (registered trademark).

[0051] (deal with)

[0052] Figure 3 This is a flowchart illustrating the overall process of the teaching process performed by the component inspection device 1. Figure 4 This is a flowchart illustrating the process of generating and processing images of defective products. Figure 5 This is an example of a screen used in teaching processing.

[0053] First of all, Figure 5The following describes the screen. Screen 100 includes a magnified image display area 110, a result output area 120, a part number specification area 130, a logic / threshold setting area 140, a test execution button 150, and a defective product image generation button 160. The part number specification area 130 is a GUI for specifying the part (part number) to be used as the object of the threshold setting; the user specifies the part as the object by selecting a checkbox. The logic / threshold setting area 140 is a GUI for the user to select the inspection item and input the threshold to be used in that inspection item. In the result output area 120, the image used for testing, the inspection result under the current settings, and information indicating whether the image is an actual good product image, an actual defective product image, or a generated defective product image are displayed. In the magnified image display area 110, the image selected from multiple images as the inspection object is magnified, and the height profile in the X and Y directions at the cursor position is displayed. By pressing the test execution button 150, the inspection in the part number specification area 130 and the logic / threshold setting area 140 is executed. By pressing the defective product image generation button 160, a defective product image can be generated.

[0054] Next, refer to Figure 3 The teaching process is explained. When the teaching process begins, the teaching section 30 displays screen 100. Additionally, in... Figure 3 The processing order shown in the flowchart is just one example. It is not necessary to follow this order, and the processing order can be changed as appropriate.

[0055] In step S1, the teaching unit 30 accepts the designation of the component part number as the object of model testing via the part number designation area 130. Figure 5 In the example, the part number "BBB" was selected in part number specified area 130.

[0056] In step S2, the teaching unit 30 accepts the designation of check items for model testing via the logic / threshold setting area 140. Figure 5 In the example, electrode wetting at the center of the end was selected as the inspection item. When an inspection item is selected, the current threshold related to the selected inspection item is displayed in the logic / threshold setting area 140, and the value can be edited.

[0057] At the specified part number and inspection logic point in time, the teaching unit 30 can also display the model image of the specified part number and whether each model image is a good or defective product for the specified inspection item.

[0058] In step S3, if the user determines that there are not enough defective product images when conducting model testing, the user presses the defective product image generation button 160 to generate defective product images and register them as model images.

[0059] Reference Figure 4 The details of the defective product image generation process in step S3 will be explained. This process is performed by the defective product image generation unit 20.

[0060] In step S31, a screen is displayed for the specifying unit 21 to specify the type of defect to be generated. Figures 6-8 This is an example of an image used to specify the type of defect to be generated. Here, we will describe three defects as solder amount, electrode floating, and electrode non-wetting. These are representative examples of defects that are difficult to set appropriately during teaching, so it is preferable to generate defect images based on such defects, but defect images can also be generated for other defects.

[0061] Figure 6 and Figure 7 This screen allows users to specify the type of defect to be generated by specifying a region within a feature space consisting of three characteristic quantities representing defects. In these examples, users can specify the type of defect to be generated from a region defined by dividing solder amount into three levels, electrode float into two levels, and electrode wetting / non-wetting into two levels. Furthermore, solder amount and electrode float can also be divided into more or fewer levels. Electrode wetting and non-wetting are essentially binary information, but can also be divided into three or more levels. Additionally, since some combinations of characteristic quantities are impossible, they cannot be selected on the screen.

[0062] exist Figure 6 In the example, the various defect types are represented by regions within a space divided into electrode wetting and non-wetting, with the horizontal axis representing the amount of solder and the vertical axis representing the degree of electrode float. Here, a total of 10 regions, ID 11 to ID 15 and ID 21 to ID 25, can be selected. Furthermore, for each region, an image showing an example of a defect corresponding to that feature space is displayed. This allows the user to easily understand what kind of defect will be generated. The user selects which type of defect to generate on this designated screen. Here, box 601 indicates the defect selected by the user.

[0063] Figure 7 The example is with Figure 6 Similarly, the type of defect is specified by defining a region within a feature space, but the user specifies a region within a three-dimensional space based on three axes: solder amount, float degree, and wetting / non-wetting. Figure 7 In, with Figure 6 Unlike other regions, images representing bad examples are not displayed, but they can still be displayed.

[0064] Figure 8This is another example of a designated screen. In this screen, the user can directly input the amount of solder, the degree of electrode float, and the wetting / non-wetting values ​​of the electrodes. Figure 8 In the example, a GUI is used to input the solder amount and float level using a scroll bar, but other GUIs such as rotation descent control or numerical input into text boxes can also be used. Furthermore, since electrode wetting and dewetting are set as binary information, specifying them using checkboxes is convenient.

[0065] In step S32, the designation unit 21 selects the type of defect to be generated from the user's acceptance. Specifically, the designation unit 21 obtains the type of defect from the user's acceptance. Figures 6-8 When the NG generation button 603 is pressed, the type of defect entered is used as the type of defect to be generated.

[0066] In step S33, the generation unit 22 retrieves an image of the object part number from the storage unit 40. The object part number is the part number specified in step S1. The generation unit 22 retrieves data for one or more object part numbers from the good product image data 41 stored in the storage unit 40. If multiple corresponding images exist in the storage unit 40, they can be selected randomly or according to a prescribed rule.

[0067] In step S34, the generation unit 22 generates a defective product image based on the image obtained in step S33. As described above, the generation unit 22 has a generator based on a Generative Adversarial Network (GAN), which can generate a defective product image that actually occurs on-site by inputting the obtained good product image into the generator and adding defective feature quantities specified by the user as noise. The added noise is determined, for example, randomly from the specified feature quantity space. Therefore, even if the specified defective product type is the same, fluctuations will occur in the actually generated defective product image, and defective product images with different details although they are of the same type of defect can be generated.

[0068] In step S35, the generation unit 22 displays the generated defective product image to the user. Specifically, in Figures 6-8 In the display screen, images of defective products are shown on the 3D viewer units 602a and 602b. In the 3D viewer units 602a and 602b, the user can arbitrarily specify the display position, magnification, and viewpoint direction of the generated image to confirm the generated defect.

[0069] In step S36, it is determined whether the user has registered the generated defective product image as a model image. The user confirms the defective product image using the 3D viewer units 602a and 602b. If the defective product image is determined to be suitable for teaching, the user presses the "Add to Library" button 604. If button 604 is pressed, the process proceeds to step S37.

[0070] In step S37, the generated defective product image is stored as a model image in the storage unit 40. At this time, the defective product image is stored in the storage unit 40 in a manner that can be determined to be a defective product image generated by the defective product image generation unit 20.

[0071] In step S38, it is determined whether the user should generate the next defective product image. If generation continues, the process returns to step S32; if generation ends, the process ends.

[0072] Thus, in the defective product image generation process of step S3, a machine learning-based generator generates a simulated defective product image with defects specified by the user, and registers it as a model image. Furthermore, the generated image is not necessarily a defective product image; the same process can also be used to generate a good product image, or a good product image with desired features can be generated and registered as a model image.

[0073] return Figure 3 Continuing the explanation. In step S4, the user presses the test execution button 150, and performs an inspection on the model image using the current inspection content and inspection threshold. Specifically, the inspection unit 31 performs the inspection logic specified in step S2 on each model image stored in the storage unit 40.

[0074] In step S5, the result output unit 32 outputs the inspection result of step S4 to the result output area 120 of the screen 100. The result output area 120 displays the model image that underwent model testing, the model test result, and information related to the model image. The information related to the model image includes information indicating whether the model image is a good product image or a defective product image, and, if the model image is a defective product image, whether it is an actual defective product image captured or a defective product image generated by the defective product image generation unit 20.

[0075] exist Figure 5 In the example, five model images 121a to 121e are displayed in the result output area 120, showing their respective inspection results 122a to 122e and information 123a to 123e related to the model images. Here, model images 121a and 121c are actual good product images, model images 121b and 121d are actual defective product images, and model image 121e is a defective product image generated by the defective product image generation unit 20. Furthermore, it is known that model images 121a and 121b can be appropriately determined as good or defective products. In contrast, model image 121c results in a false detection (over-detection) of a good product being identified as a defective product, and model image 121d results in a false detection (missed detection) of a defective product being identified as a good product.

[0076] In step S6, the user determines whether the check was performed correctly, i.e., whether the threshold needs to be adjusted. If threshold adjustment is required, the user changes the threshold value in the logic / threshold setting area 140. The threshold setting unit 33 stores the user-modified threshold in the storage unit 40. Afterward, the process returns to step S4 to perform the model test again.

[0077] In addition, Figure 3 In the flowchart, if threshold adjustment is required, only the value of the threshold is changed, but the defective product image generation process in step S3 can also be performed to add defective product images.

[0078] The user repeats the above process to adjust the threshold for accurate component inspection. This process can be performed on various part numbers and inspection logic.

[0079] According to this embodiment, simulated defective product images generated using a machine learning model can be used for threshold adjustment processing (teaching operation). Collecting defective product images becomes difficult, especially when a new production line is set up, but appropriate threshold adjustments can be made by automatically generating images to compensate for the lack of good product images. Furthermore, to confirm whether threshold adjustments have been appropriate, sometimes specific defective product images are needed; since the user can specify and generate such defective product images, appropriate threshold adjustment becomes easier from this perspective.

[0080] Furthermore, since the defective product images are generated using a generative adversarial network model, the generated defective product images are close to the actual good product images, and it is possible to generate defective product images similar to those produced in the actual field.

[0081] <Second Implementation>

[0082] In the first embodiment, the user needs to specify what kind of defective product image should be generated. The component inspection device in this embodiment also has the function of suggesting what kind of defective product image should be generated.

[0083] Figure 9 (A) is a diagram showing the functional blocks of the component inspection device 2 in this embodiment. Compared with the first embodiment ( Figure 1 The difference is that the defective product image generation unit 20 also has a suggestion unit 23, otherwise they are the same. Therefore, the suggestion unit 23 will be mainly described below.

[0084] Figure 9 (B) represents an example of a screen where the user specifies the type of defective image to be generated. In this embodiment, the screen includes a button 901 for performing a function that suggests the type of defect to be generated. When button 901 is pressed, the suggestion unit 23 performs... Figure 10 The generated image shown in (A) is proposed for processing.

[0085] In step S101, the suggestion unit 23 uses the generation unit 22 to generate defective product images for each of the multiple defective IDs. Here, for each defect, multiple, for example, about 10 defective product images are generated.

[0086] In step S102, the suggestion unit 23 performs principal component analysis or similar methods on the generated defective product image to extract features and maps them to a feature space. In step S103, the suggestion unit 23 performs clustering processing on the defective product image in the feature space to classify it into multiple clusters. The clustering method is not particularly limited; for example, k-means can be used.

[0087] In step S104, the suggestion unit 23 extracts the model images (library images) stored in the storage unit 40 into feature quantities and maps them onto the feature quantity space in the same way as in step S102, determining which cluster the model image belongs to. In step S105, the suggestion unit 23 determines the clusters in which library images do not exist within the region. In step S106, the suggestion unit 23 displays a reminder on the GUI for the defect types corresponding to the clusters in which model images do not exist within the region, suggesting to the user to generate defect images for that defect type. Alternatively, the suggestion unit 23 may suggest defect types where the number of existing model images is less than a predetermined number, instead of suggesting defect types corresponding to clusters where no model images exist at all.

[0088] Figure 10 (B) is a diagram illustrating the above processing. Figure 10 The region in (B) represents the cluster (region of each defect) obtained by mapping the defective product image generated in step S101 to a two-dimensional space and performing clustering processing. Ten clusters (regions) are obtained corresponding to the 10 defect types. Figure 10 The points in (B) are points that map the model image onto a two-dimensional space. Based on this result, there is no model image contained in the cluster with ID 12.

[0089] Therefore, it is recommended that section 23 specify the type of defect on the designated screen ( Figure 9 In (B)), a reminder 902 is displayed for ID 12 indicating a defect or deficiency, and indicating that an image of the defect should be generated. Furthermore, in the case of multiple types of defects or deficiencies, a reminder can be displayed for each defect individually.

[0090] According to this embodiment, it automatically determines which type of defective product image is insufficient and suggests generating it, thus reducing the trouble for users, and even less experienced users can make appropriate threshold adjustments.

[0091] <Other>

[0092] The above embodiments are merely illustrative examples illustrating the structure of the present invention. The present invention is not limited to the specific embodiments described above, and various modifications can be made within the scope of its technical concept.

[0093] For example, while defective product images are generated as described above, good product images can also be generated in the same way. Although a large number of good product images can be obtained, there are cases where it is impossible to obtain the good product images required for threshold adjustment. Therefore, if good product images with the desired features are generated using the same method as described above, threshold adjustment can be performed more appropriately.

[0094] In the example above, the object of inspection was the component mounting printed circuit board, but the component to be inspected can be any item. Furthermore, examples of defects such as solder quantity, electrode floating, and electrode non-wetting have been given, but other types of defects can also be inspection items.

[0095] Furthermore, while the above example uses a generative adversarial network (GAN)-based machine learning model to generate defective product images, any machine learning model other than GANs, particularly deep generative models, can also be used to generate defective product images. Deep generative models include, for example, generative adversarial networks (GANs), variational autoencoders (VAEs), and flow-based generative models.

[0096] <Postscript>

[0097] 1. A component inspection apparatus (1) comprising: a storage unit (40) storing at least a good product image (41); a generation unit (22) generating a defective product image using a machine learning model; a setting unit (21, 140) capable of user-setting parameters for component inspection; and an output unit (32, 120) using the parameters to inspect the good product image stored in the storage unit and the defective product image generated by the generation unit, and outputting an inspection result.

[0098] 2. An auxiliary method for assisting in setting parameters for component inspection in a component inspection device, comprising: a generation step (S3) of generating defective product images using a machine learning model based on stored good product images; a setting step (S7) of receiving parameter settings for component inspection from a user; and an output step (S5) of inspecting the stored good product images and the generated defective product images using the parameters (S4) and outputting the inspection results.

[0099] Label Explanation

[0100] 1, 2: Component inspection device; 10: Imaging unit; 20: Defective product image generation unit; 21: Designation unit; 22: Generation unit; 23: Suggestion unit; 30: Teaching unit; 31: Inspection unit; 32: Result output unit; 33: Threshold setting unit; 40: Storage unit; 41: Good product image data; 42: Defective product image data; 43: Inspection threshold.

Claims

1. A component inspection device, comprising: The storage unit stores images of good products and images of defective products that were actually captured. The generation unit uses a machine learning model to generate images of defective products. The setting unit allows users to set parameters for component inspection; as well as The output unit uses the parameters to inspect the good product images and defective product images stored in the storage unit, as well as the defective product images generated by the generation unit, and outputs the inspection results as information to assist in adjusting the parameters. The output unit associates the image of the inspected object, information indicating whether the defective image of the inspected object was generated by the generation unit or actually captured, and information indicating whether it was judged as a good product or a defective product using the parameters, and displays them on the same screen as the inspection result.

2. The component inspection device according to claim 1, wherein, The generation unit applies a machine learning model to at least one of the good product images stored in the storage unit to generate a defective product image.

3. The component inspection device according to claim 1, wherein, The machine learning model is a deep generative model.

4. The component inspection device according to claim 1, wherein, The component inspection device also includes a designation unit, which allows the user to specify the type of defect in the defective product image generated by the generation unit. The generation unit generates images of defective products having the types of defects specified by the designated unit.

5. The component inspection device according to claim 4, wherein, The designated unit displays a screen that allows the user to specify multiple characteristic quantities representing defects.

6. The component inspection device according to claim 5, wherein, The specified screen is configured to be a region within a feature space composed of multiple feature quantities, which can be specified by the user.

7. The component inspection device according to claim 6, wherein, In at least one of the regions within the feature space, an image representing a defect corresponding to that feature space is displayed.

8. The component inspection device according to claim 5, wherein, The specified screen is configured such that the user can independently specify the values ​​of multiple feature quantities.

9. The component inspection device according to claim 1, wherein, The component inspection device also includes a suggestion unit that makes suggestions on the types of defective product images that should be generated.

10. The component inspection apparatus according to claim 9, wherein, The suggestion unit generates defective product images for multiple types of defects through the generation unit, performs clustering processing to classify the generated defective product images into multiple clusters, determines which cluster each defective product image stored in the storage unit belongs to, and suggests that the defective product corresponding to the cluster with a specified number of existing defective product images be the type of defective product image to be generated.

11. The component inspection apparatus according to any one of claims 1 to 10, wherein, The output unit outputs an image of the object being inspected, information indicating whether the image of the object being inspected was generated by the generation unit, and information indicating whether the object was judged as a good product or a defective product using the parameters, as the inspection result.

12. An auxiliary method for assisting in setting parameters for component inspection in a component inspection device, the component inspection device having a storage unit for storing actually captured images of good products and actually captured images of defective products, the auxiliary method comprising: The generation step uses a machine learning model to generate images of defective products. The setup process begins with the user accepting the setting of parameters for component inspection; as well as In the output step, the parameters are used to inspect the good product images and defective product images stored in the storage unit, as well as the defective product image generated in the generation step. The inspection results are output as information to assist in adjusting the parameters. In the output step, the image of the inspected object, information indicating whether the defective image of the inspected object was generated in the generation step or actually captured, and information indicating whether it was judged as a good product or a defective product using the parameters are displayed together on the same screen as the inspection result.

13. A storage medium that non-transitory stores a computer program for causing a computer to perform the steps of the method of claim 12.

Images