Additional inspection device, inspection system, program, and method for creating machine learning model

Abstract

To provide an additional inspection device which is added to an inspection device that inspects the appearance of a printed wiring board, in which work efficiency is improved when an operator adds true / false information to a defect image and creates a machine learning model.SOLUTION: An additional inspection device 12 comprises: a reception unit 31 that receives a defect image indicating a defect of a printed wiring board that was detected by an inspection device; and a reinspection unit 33 that reinspects the defect image. The defect image is at least a part of images determined as showing a defect on the basis of whether or not an inspection criterion is satisfied in the inspection device. The reinspection unit 33 reinspects the defect image to determine whether it indicates a defect by using a first machine learning model 7 when inspection information 82, which is information related to an inspection selected by the inspection device, is first inspection information, and reinspects the defect image to determine whether it indicates a defect by using a second machine learning model 7 when the inspection information 82 is second inspection information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a technique for inspecting the appearance of a printed wiring board. [Background technology]

[0002] In the manufacture of printed wiring boards, inspection devices have been used to detect defects by imaging printed wiring boards. Known examples of such visual inspection devices include intermediate inspection devices called AOI (Automated Optical Inspection) and final visual inspection devices called AVI (Automated Final Visual Inspection).

[0003] Visual inspection devices may detect defects that are not actually defects as defects known as "false alarms" or "false defects." Therefore, technology has been proposed for visual inspection that uses machine learning to reduce false alarms. For example, the inspection system described in Patent Document 1 includes a primary inspection unit that performs defect determination without using machine learning based on captured images of objects, and a secondary inspection unit that uses a machine learning model to distinguish between truly defective products and over-determined products based on images of objects determined to be defective by the primary inspection unit. This reduces productivity losses due to over-determined products.

[0004] In the inspection system of Patent Document 2, the inspection unit performs inspection without using machine learning to detect defects, and classifies the defect type of the defect using a trained model (machine learning model) for an image showing the defect. At this time, whether or not the defect needs to be classified is determined based on defect-related information acquired or used by the inspection unit when the defect is detected. This avoids serious misclassification in the trained model and reduces the time required for classification processing. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent Publication No. 2021-177154 [Patent Document 2] Japanese Patent Application Publication No. 2023-141721 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, for example, a final appearance inspection device inspects various parts of a printed wiring board, such as pads, solder resist, silk printing, and base material. To generate a machine learning model such as that described in Patent Document 1 to remove false reports from various defects obtained from images of a printed wiring board, an operator must create training data by adding true / false information to approximately tens of thousands of defect images to classify the defects as false reports or true defects. In this case, because there are a wide variety of defect types, even experienced operators are likely to make mistakes in classifying the defects as true / false.

[0007] Furthermore, if the product number of the printed wiring board to be inspected changes and the design of the printed wiring board changes, the classification accuracy using the machine learning model will decrease, so each time this happens, it will be necessary to input the authenticity information of tens of thousands of defect images and regenerate the machine learning model.

[0008] The present invention has been made in consideration of the above-mentioned problems, and its main purpose is to improve the work efficiency when an operator determines the authenticity of a defect image and generates a machine learning model in an additional inspection device that uses a machine learning model. [Means for solving the problem]

[0009] A first aspect of the present invention is an additional inspection device that is added to an inspection device that inspects the appearance of a printed wiring board, and includes an acceptance unit that accepts defect images that indicate defects in the printed wiring board detected by the inspection device, and a re-inspection unit that re-inspects whether the defect images indicate defects, wherein the defect images are at least a portion of an image that has been determined in the inspection device to indicate a defect based on whether the defect images satisfy inspection criteria in an inspection selected from multiple types of inspection, and wherein the re-inspection unit, when inspection information that is information related to the inspection selected in the inspection device is first inspection information, re-inspects whether the defect images indicate a defect using a first machine learning model, and when the inspection information is second inspection information, re-inspects whether the defect images indicate a defect using a second machine learning model different from the first machine learning model.

[0010] A second aspect of the present invention is an additional inspection device of the first aspect, wherein the reception unit receives the defect image from the inspection device together with the inspection information, which is information related to the inspection performed by the inspection device when the defect image was detected.

[0011] Aspect 3 of the present invention is an additional inspection device of aspect 1 (which may be aspect 1 or 2), further comprising an output unit that outputs the defect image to a defect confirmation device for an operator to confirm the defect image when the re-inspection unit determines that the defect image indicates a defect.

[0012] Aspect 4 of the present invention is an additional inspection device of aspect 1 (which may be any one of aspects 1 to 3), in which at least some of the multiple pieces of inspection information corresponding to multiple types of inspections performed by the inspection device are associated with functional components of wiring on the printed wiring board.

[0013] Aspect 5 of the present invention is an additional inspection device of aspect 1 (which may be any one of aspects 1 to 4), in which at least a portion of the multiple inspection information related to multiple types of inspections performed by the inspection device is associated with the material that constitutes the printed wiring board.

[0014] A sixth aspect of the present invention is the additional inspection device of the first aspect (which may be any one of the first to fifth aspects), wherein when the inspection information is third inspection information, the reinspection unit does not perform a reinspection.

[0015] A seventh aspect of the present invention is the additional inspection device of the sixth aspect, wherein the third inspection information is information indicating an inspection for defects in which copper is exposed in the solder resist region.

[0016] Aspect 8 of the present invention is an additional inspection device of aspect 6 (which may be aspect 6 or 7), wherein the third inspection information is information indicating an inspection of defects where solder resist has adhered to the plating area.

[0017] A ninth aspect of the present invention is an inspection system for inspecting the appearance of a printed wiring board, comprising an additional inspection device described in any one of aspects 1 to 8, and the inspection device that outputs a defect image showing defects in the printed wiring board to the additional inspection device.

[0018] A tenth aspect of the present invention is a program that causes a computer to function as an additional inspection device that is added to an inspection device that inspects the appearance of printed wiring boards, and execution of the program by the computer causes the computer to execute the following steps: a) receiving a defect image that indicates a defect in a printed wiring board detected by the inspection device; and b) re-inspecting whether the defect image indicates a defect; the defect image is an image that has been determined to indicate a defect in the inspection device based on whether it satisfies inspection criteria in an inspection selected from multiple types of inspection, and step b) includes, if the inspection information, which is information related to the type of inspection selected in the inspection device, is first inspection information, re-inspecting whether the defect image indicates a defect using a first machine learning model; and, if the inspection information is second inspection information, re-inspecting whether the defect image indicates a defect using a second machine learning model different from the first machine learning model.

[0019] Aspect 11 of the present invention is a machine learning model generation method for generating a machine learning model to be used for reinspection in an additional inspection device that is added to an inspection device that inspects the appearance of a printed wiring board and reinspects defect images output from the inspection device, comprising the steps of: a) for each of a plurality of inspection information pieces that are information related to a plurality of types of inspections performed by the inspection device, a1) receiving authenticity information from an operator that is an input of the authenticity of the defect image output from the inspection device, a2) associating the defect image with the authenticity information, and a3) repeating steps a1) and a2); and b) for each of the plurality of inspection information pieces, generating a machine learning model by learning using a plurality of combinations of defect images and the authenticity information associated therewith. [Effects of the Invention]

[0020] According to the present invention, in an additional inspection device that uses a machine learning model, it is possible to improve the work efficiency when an operator determines the authenticity of a defect image and generates a machine learning model. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 10 is a diagram showing the configuration of an inspection system including an additional inspection device. [Figure 2] FIG. 10 is a diagram showing the configuration of a computer that is an additional inspection device. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of an additional inspection device. [Figure 4] FIG. 2 is a diagram showing the flow of operations of the inspection device. [Figure 5] FIG. 10 is a diagram showing a defect image. [Figure 6A] FIG. 10 is a diagram showing a defect image. [Figure 6B] FIG. 10 is a diagram showing a part of a reference binary image. [Figure 6C] FIG. [Figure 7A] FIG. 10 is a diagram showing a defect image. [Figure 7B] FIG. 10 is a diagram showing a part of a reference image. [Figure 7C]FIG. [Figure 8] FIG. 10 is a diagram showing a defect image. [Figure 9] FIG. 10 is a diagram showing a defect image. [Figure 10] FIG. 10 is a diagram showing a defect image. [Figure 11] FIG. 10 is a diagram showing the flow of operations of the additional inspection device. [Figure 12] FIG. 2 is a diagram illustrating a functional configuration of a learning model generation device. [Figure 13] FIG. 2 is a diagram showing the flow of operations of the learning model generation device. DETAILED DESCRIPTION OF THE INVENTION

[0022] FIG. 1 is a diagram showing the configuration of an inspection system 1 including an additional inspection device 12 according to one embodiment of the present invention. The inspection system 1 inspects the appearance of a printed wiring board, which is an object. "Inspecting the appearance" means acquiring an image of the printed wiring board and determining whether the printed wiring board has a defect based on the image. The inspection system 1 includes an appearance inspection device (hereinafter simply referred to as "inspection device") 11 that captures an image of the printed wiring board and inspects its appearance, an additional inspection device 12, and a defect confirmation device 13, which are connected via a communication network 14. The inspection device 11 may be an intermediate inspection device called AOI or a final appearance inspection device called AVI. The printed wiring board is the board inspected by these devices, and may be a printed wiring board in the process of being manufactured or a completed printed wiring board.

[0023] The inspection device 11 includes an image acquisition unit 21 and an inspection unit 22. The image acquisition unit 21 includes an imaging unit and a movement mechanism. The imaging unit has a so-called line sensor in which multiple imaging elements are arranged in a line. The movement mechanism moves the printed wiring board relative to the imaging unit. The imaging unit repeatedly acquires line images while the printed wiring board moves relative to the imaging unit, thereby acquiring a two-dimensional image of the printed wiring board. The imaging unit that acquires the image of the printed wiring board can be modified in various ways. For example, the imaging unit may acquire a two-dimensional image of the printed wiring board using an image sensor in which imaging elements are arranged two-dimensionally.

[0024] The inspection unit 22 is realized by, for example, a computer and / or a dedicated electric circuit. The inspection unit 22 detects defects from an image of the printed wiring board and acquires an image including the defect and its vicinity as a defect image. The defect image at this stage is an image showing a defect that the inspection device 11 has determined to be a defect. In this way, "defective" means that the defect has been determined to be a defect according to the determining entity. Defect image data 81, which is data on the defect image, is stored in the memory unit 221 of the inspection unit 22. The inspection unit 22 performs various inspections, i.e., defect detection processes, on the image of the printed wiring board.

[0025] In the storage unit 221, together with the defect image data 81, inspection information 82, which is information related to the inspection performed when the defect image data 81 was acquired, is stored in association with the defect image data 81. An example of "information related to the inspection" is "type of inspection" (hereinafter also referred to as "inspection type"). "Type of inspection" typically refers to the type of inspection algorithm, and in principle corresponds to an individual inspection algorithm. However, a general concept that encompasses multiple inspection algorithms may also be treated as a single inspection type. For example, multiple methods for inspecting specific functional elements of a printed wiring board or multiple methods for inspecting specific materials may be considered as a single inspection type.

[0026] Another example of "information related to inspection" is information associated with functional components of wiring to be inspected on a printed wiring board. That is, information indicating functional component units of wiring is used as inspection information. Examples of functional components of wiring include pads, wiring, and through-holes. Another preferred example is information associated with materials constituting the printed wiring board. That is, information indicating the materials constituting the printed wiring board is used as inspection information. Examples of materials constituting the printed wiring board include solder resist, copper, silk printing, plating, substrate, solder resist on copper (pattern), and solder resist on substrate. Yet another preferred example is the type of defect. That is, information indicating the type of defect is used as inspection information. Of course, the multiple types of "information related to inspection" do not need to be all of the above examples. The multiple types of inspection information are not limited to the above examples, and it is sufficient if at least a portion of the inspection information is one of the above examples.

[0027] The storage unit 221 may be provided outside the inspection unit 22. For example, a computer or NAS (Network Attached Storage) provided outside the housing of the inspection device 11 may function as the storage unit 221.

[0028] The inspection unit 22 is usually arranged inside the housing of the inspection device 11. The inspection unit 22 may be arranged outside the housing of the inspection device 11, or may be connected to the main body of the inspection device 11 via the communication network 14.

[0029] In the following description, defect image data will also be simply referred to as a "defect image." Similarly, data of other images may also be simply referred to by the name of the image. Furthermore, processing that is actually performed on image data may also be simply described as processing that is performed on the image. For example, binarization that is actually performed on defect image data may also be simply described as binarization of the defect image.

[0030] The additional inspection device 12 performs an additional inspection on the defect image acquired by the inspection device 11, and determines whether the defect image indicates a defect or a false report indicating a false defect. In the following description, the additional inspection by the additional inspection device 12 is also referred to as a "re-inspection." As will be described later, the additional inspection device 12 performs the re-inspection using a machine learning model. The machine learning model is a classification model that has undergone machine learning, and is a classifier that classifies the defect image into a defect or a false defect (false report). The re-inspection is also a classification operation.

[0031] The additional inspection device 12 is a device that is added to the inspection device 11, but "addition" here means that the inspection device 11 has the independent function of outputting defect images, and the additional inspection device 12 has the function of performing additional inspection on the defect images. In other words, if the inspection device 11 functions as a device that performs inspection independently, the additional inspection device 12 does not need to be physically added later, and for example, the additional inspection device 12 may be installed at the same time as the inspection device 11 is installed in a predetermined location.

[0032] Furthermore, the additional inspection device 12 may be provided inside the housing of the inspection device 11 without going through the communication network 14, or may be directly connected to the inspection device 11, or conversely, the additional inspection device 12 may be installed remotely from the inspection device 11 via the communication network 14. Furthermore, the additional inspection device 12 may be realized by a computer that realizes the inspection unit 22 in the inspection device 11. In this way, the additional inspection device 12 may be provided in various modes as long as it is added to the inspection device 11 as a function.

[0033] The additional inspection device 12 re-inspects the defect image to determine whether it is a false alarm. The additional inspection device 12 re-inspects the defect image by, for example, having a computer execute a program. Only when the re-inspection determines that the defect image is not a false alarm does the additional inspection device 12 send the defect image to the defect confirmation device 13. The defect image at this stage is an image that has been determined by the inspection device 11 and the additional inspection device 12 to show a defect.

[0034] The defect confirmation device 13 is a device that allows an operator to confirm a defect image when the additional inspection device 12 determines that the defect image indicates a defect. The defect confirmation device 13 is also a device that is realized by a computer executing a program. The defect confirmation device 13 displays a defect image and an image (hereinafter referred to as a "master image") in which no defect corresponding to the defect image exists on a display unit. The operator visually compares the two images to determine whether the defect image indicates a defect or is a false report indicating a spurious defect, and inputs the determination result into the defect confirmation device 13 via an input unit such as a mouse or keyboard. The defect confirmation device 13 is not limited to the above form as long as it is a device that allows an operator to confirm defects.

[0035] Defects in printed wiring board inspection include "functional defects" that affect the function of the printed wiring board, and "visual defects" that do not affect the function but do not meet the appearance standard. Defects in the present invention may be either functional defects or visual defects, but since defect images that are determined to be false reports by the defect confirmation device 13 are mainly images that show defects that do not affect the function, defects in the present invention may be interpreted as being limited to functional defects.

[0036] FIG. 2 is a diagram showing the configuration of a computer serving as the additional inspection device 12. The additional inspection device 12 has the configuration of a typical computer system, including a CPU 301, a GPU 302, a ROM 303, a RAM 304, a fixed disk 305, a display 306, an input unit 307, a reading device 308, a communication unit 309, and a bus 30. The CPU 301 performs various arithmetic operations. The GPU 302 performs various arithmetic operations related to image processing. The ROM 303 stores basic programs. The RAM 304 and the fixed disk 305 store various types of information. The display 306 displays various types of information, such as images. The input unit 307 includes a keyboard 307a and a mouse 307b for receiving input from an operator. The reading device 308 reads information from a computer-readable recording medium 9, such as an optical disk, a magnetic disk, a magneto-optical disk, or a memory card. The communication unit 309 transmits and receives signals to and from other components of the additional inspection device 12 and external devices. The bus 30 is a signal circuit that connects the CPU 301, the GPU 302, the ROM 303, the RAM 304, the fixed disk 305, the display 306, the input unit 307, the reading device 308, and the communication unit 309.

[0037] In the additional inspection device 12, the program 91 is read in advance from the recording medium 9 via the reading device 308 and stored on the fixed disk 305. The program 91 may be stored on the fixed disk 305 via a communication network. The CPU 301 and the GPU 302 execute arithmetic processing using the RAM 304 and the fixed disk 305 in accordance with the program 91. The CPU 301 and the GPU 302 function as a calculation unit in the additional inspection device 12. Other components that function as a calculation unit may be employed in addition to the CPU 301 and the GPU 302.

[0038] FIG. 3 is a diagram showing the functional configuration of the additional inspection device 12, which is realized by a computer executing arithmetic processing and the like according to a program 91. That is, execution of the program by a computer causes the computer to function as the additional inspection device 12. The CPU 301, GPU 302, ROM 303, RAM 304, fixed disk 305, communication unit 309, and their peripheral components realize the reception unit 31, memory unit 32, re-inspection unit 33, and output unit 34 in FIG. 3. The re-inspection unit 33 includes a learning model selection unit 331. All or part of these functions may be realized by dedicated electrical circuits, or these functions may be realized by individual programs. Furthermore, these functions may be realized by multiple computers.

[0039] The receiving unit 31 receives defect images (more precisely, defect image data 81) from the inspection device 11 and stores them in the storage unit 32. For example, the receiving unit 31 is a communication unit 309 such as an interface that communicates with the communication network 14 and has a functional configuration that controls the communication unit 309, and the defect images are stored in the fixed disk 305 in FIG. 2 that functions as the storage unit 32. The receiving unit 31 receives inspection information 82, which is information related to the inspection when the defect was detected, together with the defect image data 81 from the inspection device 11, and stores the inspection information 82 in the storage unit 32 as well.

[0040] The reinspection unit 33 stores multiple machine learning models 7. The machine learning models 7 are stored in, for example, the RAM 304 in FIG. 2. The machine learning models 7 are classifiers that, when a defect image (data) is input, output whether the defect image is a false report indicating a false defect or not. The multiple machine learning models 7 are used depending on the inspection information 82 and have different classification functions. Each machine learning model 7 is generated, for example, by deep learning using a neural network. For example, an AI model using an AI network such as Resnet or Efficentnet is adopted as the machine learning model 7. The machine learning model 7 is an input / output structure that outputs a false report or a non-false report in response to an image input. The parameters in the structure are determined by learning, and in some cases, the structure itself is also determined. A "non-false report" means that the inspection device 11 and the additional inspection device 12 did not determine the image as a "false report" but rather determined it to be an image indicating a defect. The machine learning model 7 may be generated by machine learning other than deep learning.

[0041] In a preferred example, multiple machine learning models 7 are provided for each inspection type in the inspection device 11. In another preferred example, multiple machine learning models 7 are provided for each functional component of wiring on a printed wiring board. Examples of functional components of wiring include pads, wiring, and through-holes. In yet another preferred example, multiple machine learning models 7 are provided for each material constituting the printed wiring board. Examples of materials constituting the printed wiring board include solder resist, copper, silk printing, plating, a substrate, solder resist on copper (pattern), and solder resist on a substrate. In yet another preferred example, multiple machine learning models 7 are provided for each type of defect. Of course, each machine learning model 7 may be provided corresponding to at least one element selected from the inspection type, the material constituting the printed wiring board, the functional component of wiring, and the type of defect.

[0042] The re-examination unit 33 is connected to a machine learning model generation device described below, and the structure and parameter values ​​of the machine learning model generated by the machine learning model generation device are stored in the re-examination unit 33 as the machine learning model 7.

[0043] The learning model selection unit 331 of the reinspection unit 33 receives the inspection information 82 from the storage unit 32 and selects the machine learning model 7 to be used in accordance with the inspection information 82. The reinspection unit 33 uses the selected machine learning model 7 to determine whether the defect image is a false alarm or not. That is, the reinspection unit 33 reinspects the defect image to determine whether the defect image indicates a defect.

[0044] If the reinspection determines that the defect image is an image that indicates a defect, the defect image data 81 and the inspection information 82 are sent to the defect confirmation device 13 via the output unit 34 and the communication network 14. If the reinspection determines that the defect image does not indicate a defect, the defect image data 81 and the inspection information 82 are not sent to the defect confirmation device 13. The output unit 34 is a communication unit 309 such as an interface that communicates with the communication network 14 and has a functional configuration that controls this.

[0045] FIG. 4 is a diagram showing the flow of operations of the inspection device 11. First, a printed wiring board is carried into the inspection device 11, and the image acquisition unit 21 acquires a two-dimensional image of the entire printed wiring board (hereinafter referred to as a "board image") (step S11). The board image may be a color image or a monochrome image, or both images may be acquired. Next, the inspection unit 22 performs processing to detect defects on the printed wiring board based on the board image (step S12). The inspection unit 22 performs various processing on the board image depending on the type of defect to be detected. In a typical example, the inspection device 11 selects an inspection to be performed from multiple types of inspection, and identifies defects in the board image (more precisely, areas indicating defects) based on whether each area serving as an inspection unit of the board image satisfies the inspection criteria for the selected inspection.

[0046] Fig. 5 is a diagram showing an example of a defect image, which is an image showing a defect detected by the inspection unit 22. In the defect image 811 in Fig. 5, the area marked with reference numeral 812 is a solder resist area, the area marked with reference numeral 813 is a copper area indicating a pad, and the area marked with reference numeral 814 is a through-hole.

[0047] The operation of the inspection unit 22 when the defect image 811 in Fig. 5 is detected is as follows. First, an image of the area to be inspected in the board image (hereinafter referred to as the "target image") is prepared in advance, and a copper area 813 and a through-hole area 814 are detected from the image. The area to be inspected is larger than the defect image 811 that will ultimately be obtained. Meanwhile, the inspection unit 22 also holds CAD data for the printed wiring board, and acquires the copper area and the through-hole area in the CAD data. Then, if the through-hole area 814 relative to the copper area 813 in the target image differs from the information in the CAD data by an amount exceeding an allowable range, it is detected as a defect.

[0048] In practice, many types of inspections are performed on each region of the substrate image, and when one or more defects are detected, as shown in Fig. 1, defect image data 81 and inspection information 82, which is information related to the inspection performed by inspection device 11 when the defect image was detected, are stored in storage unit 221 in association with each other. In the example of Fig. 5, the inspection information is information indicating an inspection type called "hole inspection" (or may be information indicating a defect type called "hole defect"). Data of defect image 811 and inspection information 82 indicating "hole inspection" are stored in storage unit 221 in association with each other.

[0049] Then, when inspection of one substrate image is completed, a combination of the defect image data 81 and the inspection information 82 is output to the additional inspection device 12 (step S13).

[0050] 6A is a diagram showing another example of a defect image showing a defect detected by inspection unit 22. In defect image 821, the area indicated by symbol 823 within circular bright area 822 is slightly dark, and area 823 has been detected as a defect. Such defects are detected due to changes in color across the entire printed wiring board or partial color differences.

[0051] The operation of the inspection unit 22 when a defect image 821 is detected is as follows. First, a target image, which is an image of the area to be inspected in the substrate image, and an image of an area corresponding to the target image in a master image without defects (hereinafter referred to as a "reference image") are prepared in advance. The area to be inspected is larger than the defect image 821 that will ultimately be obtained. Then, a reference binary image is obtained by binarizing the reference image using a predetermined threshold value. FIG. 6B is a diagram showing only the area of ​​the reference binary image that corresponds to FIG. 6A, and the fact that it is actually a larger image is indicated by the outline shown with a dashed line (the same applies hereinafter).

[0052] Next, the target image is also binarized using the same threshold value to obtain a target binary image. Inspection unit 22 obtains a difference image between the target binary image and the reference binary image, and detects the difference region as a defect region 824 as shown in FIG. 6C. Furthermore, inspection unit 22 cuts out defect region 824 and its surrounding region from the target image to obtain defect image 821 as shown in FIG. 6A. As described above, in the example shown in FIGS. 6A to 6C, defects are detected by the above-mentioned "comparative inspection" (algorithm). In the example of FIGS. 6A to 6C, the inspection information is information indicating an inspection type called "comparative inspection." Data of defect image 821 and the inspection information indicating "comparative inspection" are stored in memory unit 221 in association with each other.

[0053] Fig. 7A is a diagram showing another example of a defect image. Defect image 831 in Fig. 7A has a solder resist area 832, a substantially circular plating area 833, and an external area 834 outside the printed wiring board. In defect image 831, there is a slightly bright area 835 in solder resist area 832, and this area 835 has been detected as a defect by inspection device 11.

[0054] The operation of the inspection unit 22 when the defect image 831 is acquired is as follows. First, a target image indicating the area to be inspected in the substrate image and a reference image indicating the area in the master image corresponding to the target image are prepared in advance. As described above, the area to be inspected is larger than the defect image 831 that will ultimately be obtained. FIG. 7B is a diagram showing only the area of ​​the reference image that corresponds to FIG. 7A.

[0055] Next, in the target image and the reference image, areas other than the solder resist area are masked with reference to the design information of the printed wiring board, and a target processed image, which is a processed image of the target image, and a reference processed image, which is a processed image of the reference image, are obtained. Then, by binarizing the difference image between the target processed image and the reference processed image, an image showing the defect area 836 shown in FIG. 7C is obtained. Furthermore, the inspection unit 22 cuts out the defect area 836 and its surrounding area from the target image, and obtains the defect image 821 shown in FIG. 7A. In the example shown in FIGS. 7A to 7C, defects are detected by the above-mentioned "unevenness inspection" (algorithm). The data of the defect image 831 and inspection information indicating the inspection type called "unevenness inspection" (or inspection information indicating the defect type called "unevenness defect") are stored in the memory unit 221 in association with each other.

[0056] Fig. 8 is a diagram showing another example of a defect image, which is an image showing a defect detected by the inspection unit 22. In the defect image 841 in Fig. 8, the area marked with reference numeral 842 is a solder resist area, the area marked with reference numeral 843 is a plating area indicating a pad, and the area marked with reference numeral 844 is a foreign matter area to which the solder resist is attached.

[0057] The operation of the inspection unit 22 when the defect image 841 in FIG. 8 is detected is as follows. First, a target image, which is an image of the area to be inspected in the board image, is prepared in advance, and a plating area 843 is detected from the image based on CAD data. Then, the color distribution of pixels in the plating area 843 in the target image is obtained, and a foreign matter area 844, where foreign matter of solder resist (hereinafter may be referred to as "SR") has adhered to the plating area 843, is detected as a defect. In the example of FIG. 8, the inspection information is information indicating an inspection type called "pad area SR foreign matter inspection" (or may be information indicating a defect type called "pad area SR foreign matter"). The data of the defect image 841 and the inspection information indicating "pad area SR foreign matter inspection" are stored in the storage unit 221 in association with each other.

[0058] Fig. 9 is a diagram showing another example of a defect image, which is an image showing a defect detected by the inspection unit 22. In the defect image 851 in Fig. 9, the area marked with reference numeral 852 is a solder resist area, the area marked with reference numeral 853 is a plating area indicating a pad, and the area marked with reference numeral 854 is a foreign matter area to which foreign matter other than solder resist has adhered.

[0059] The operation of the inspection unit 22 when a defect image 851 in FIG. 9 is detected is almost the same as in the case of FIG. 8, except that after acquiring the color distribution of pixels in a plating region 853 in the target image, a foreign substance region 854 determined to contain a foreign substance other than solder resist from the color distribution is detected as a defect. In the example of FIG. 9, the inspection information is information indicating an inspection type called "pad region foreign substance inspection" (or information indicating a defect type called "pad region foreign substance"). The data of the defect image 851 and the inspection information indicating "pad region foreign substance inspection" are stored in association with each other in the storage unit 221. The "pad region SR foreign substance inspection" in FIG. 8 is a special example of "pad region foreign substance inspection."

[0060] Fig. 10 is a diagram showing another example of a defect image, which is an image showing a defect detected by the inspection unit 22. In the defect image 861 in Fig. 10, the area marked with reference numeral 862 is a solder resist area, the area marked with reference numeral 863 is a plating area indicating a pad, and the area marked with reference numeral 864 is an area where the solder resist has peeled off and the underlying copper is exposed.

[0061] The operation of the inspection unit 22 when the defect image 861 in FIG. 10 is detected is as follows. First, a target image, which is an image of the area to be inspected in the board image, is prepared in advance, and a solder resist area 862 is detected from the image based on CAD data. Then, the color distribution of pixels in the solder resist area 862 in the target image is obtained, and a copper exposure area 864 where copper is exposed in the solder resist area 862 is detected as a defect. In the example of FIG. 10, the inspection information is information indicating the inspection type, "SR peeling copper exposure inspection" (or may be information indicating the defect type, "SR peeling copper exposure"). The data of the defect image 861 and the inspection information indicating "SR peeling copper exposure inspection" are stored in the storage unit 221 in association with each other.

[0062] The combination of the defect image data 81 and the inspection information 82 stored in the inspection device 11 as described above is sent to the additional inspection device 12 (step S13).

[0063] 11 is a diagram showing the flow of operations of the additional inspection device 12. A combination of defect image data 81 and inspection information 82 is received by the receiving unit 31 of the additional inspection device 12 and stored in the storage unit 32 (step S21).

[0064] Next, the learning model selection unit 331 of the reinspection unit 33 receives the inspection information 82 from the storage unit 221 and determines whether a reinspection is necessary (step S22). For example, if the inspection information is "pad area SR foreign matter inspection" as in the defect image 841 of FIG. 8, the pad area and the solder resist each have distinctive colors and are significantly different from each other, so the possibility that the defect image 841 is a false alarm is extremely low. Therefore, in order to efficiently perform a reinspection of the defect image, a reinspection of the defect image 841 is not performed. Similarly, if the inspection information is "SR peeling copper exposure inspection" as in the defect image 861 of FIG. 10, the possibility that the defect image 861 is a false alarm is extremely low, so a reinspection is not performed. If a reinspection is not performed, the defect image data 81 and the inspection information 82 (or only the defect image data 81) are sent to the defect confirmation device 13 (step S26).

[0065] On the other hand, if reinspection is necessary, the learning model selection unit 331 selects a machine learning model 7 to be used (step S23). For example, the learning model selection unit 331 selects a different machine learning model 7 for each of the defect images in FIGS. 5, 6A, 7A, and 9, and the reinspection unit 33 reinspects the defect image using the selected machine learning model 7 (step S24). That is, the reinspection unit 33 inputs defect image data 81 corresponding to the inspection information 82 to the selected machine learning model 7 and obtains, as output, information indicating whether or not a false alarm is detected. In other words, the reinspection unit 33 selects a machine learning model 7 corresponding to the inspection information from the multiple machine learning models 7 and reinspects the defect image using the selected machine learning model 7 to determine whether the defect image indicates a defect.

[0066] If the operation of the re-inspection unit 33 is expressed by focusing only on the first inspection information and second inspection information that are different from each other and are included in many types of inspection information, when the inspection information, which is information related to the inspection selected in the inspection device 11, is the first inspection information, the re-inspection unit 33 re-inspects whether the defect image indicates a defect using a first machine learning model, and when the inspection information is the second inspection information, the re-inspection unit 33 re-inspects whether the defect image indicates a defect using a second machine learning model different from the first machine learning model.

[0067] Note that if retesting is not required in step S22, this may be considered to mean that none of the machine learning models 7 are selected, and in this case, steps S22 and S23 are not distinguishable processes. If the test information when retesting is not required is expressed as "third test information," the operation of the retesting unit 33 is to not perform retesting when the test information is the third test information. By omitting unnecessary retesting, retesting can be performed efficiently.

[0068] The number of machine learning models corresponding to test information is not limited to one or zero, but may be two or more. In this case, depending on the type of test information, a false alarm may be determined when all outputs from two or more machine learning models indicate a false alarm, or when an output from any machine learning model indicates a false alarm. Furthermore, multiple pieces of test information may correspond to one machine learning model. In other words, the same machine learning model may be selected for multiple pieces of test information.

[0069] If the re-inspection unit 33 determines that the defect image is a false alarm, the defect image data 81 is not sent to the defect confirmation device 13 (see FIG. 1) (step S25). If the re-inspection unit 33 determines that the defect image is not a false alarm, the defect image data 81 and inspection information 82 (or only the defect image data 81) are sent from the additional inspection device 12 to the defect confirmation device 13 (step S26). The defect confirmation device 13 displays the defect image and a master image without any defects side by side, and the worker visually checks these images to determine whether the defect image indicates a true defect. If the defect image indicates a true defect, the printed wiring board is discarded or repaired.

[0070] By providing the additional inspection device 12 in the inspection system 1, the number of defect images that the operator must check using the defect confirmation device 13 is reduced. As a result, the probability of human error occurring due to operator fatigue can be reduced, and printed wiring boards with real defects can be appropriately prevented from being used as products.

[0071] The defect images illustrated in Figures 5, 6A, 7A, 8, 9, and 10 are images obtained by cutting out one defect detected from a board image showing a printed wiring board and its surrounding area. This makes it easy to limit the scope of processing during reinspection to the scope of the defect image. Note that "one defect" means that there is one defect of interest, and includes cases where multiple defect areas are perceived as one defect. The defect image may take any other form as long as it is an image showing a defect detected by inspection device 11. For example, the defect image may be a combination of a board image and coordinates indicating the position of the defect. Alternatively, the target image described above, i.e., an image of the area of ​​the board image used for inspection by inspection device 11, may be used as the defect image.

[0072] Furthermore, although the defect image is an image showing one defect detected by the inspection device 11, the defect image does not have to be an image showing only one defect, and one defect image may include multiple defects, i.e., the defect of interest and other defects, because multiple defects are close to each other.

[0073] The inspection device 11 performs so-called "rule-based" inspection. That is, the inspection device 11 detects defects from a target image based on whether the target image satisfies an inspection criterion for an inspection selected from multiple types of inspection. "Based on whether the target image satisfies an inspection criterion" typically means that the presence or absence of a defect is determined by comparing a value derived from the target image (such as the area of ​​a difference region or the number of pixels of a specific color) with a predetermined value. Then, at least a portion of the target image determined to contain a defect is used as a defect image. The above processing enables the inspection device 11 to quickly perform a large amount of processing.

[0074] The above-described examples of inspection by the inspection device 11 and the re-inspection by the additional inspection device 12 are only a few examples, and in reality, many types of inspections are performed on various parts of the board image. For example, various inspections are performed, such as for pattern bending, pattern shorts, pattern opens, through-hole abnormalities, silk printing abnormalities, peeling of solder resist on copper, peeling of solder resist on the base material, foreign matter on copper, foreign matter on solder resist, foreign matter on the pattern, unevenness in the solder resist, copper pattern abnormalities, and abnormalities in the plated area. Furthermore, the inspection by the inspection device 11 based on whether the inspection criteria are met includes a DRC (Design Rule Check) that compares design information with the target image.

[0075] In the above description, the receiving unit 31 of the additional inspection device 12 receives inspection information together with the defect image from the inspection device 11 (step S21), and the reinspection unit 33 performs reinspection using the machine learning model 7 selected according to the inspection information (step S23). By having the additional inspection device 12 receive the inspection information from the inspection device 11, appropriate reinspection can be performed promptly. However, the inspection information may also be derived from the defect image.

[0076] For example, the type of defect, which is the inspection information, may be identified from the defect image, and the machine learning model 7 may be selected based on the identified type of defect. Alternatively, the machine learning model 7 may be selected from the defect image based on the functional components of the printed wiring board (e.g., lands, wiring, through holes, etc.) indicated by the defect image, or the material of the printed wiring board (e.g., solder resist, silk printing, copper, plating, base material, solder resist on copper (pattern), solder resist on base material, etc.). In this way, the inspection information does not need to be sent from the inspection device 11 to the additional inspection device 12.

[0077] Next, a machine learning model generation device that generates a machine learning model will be described. Fig. 12 is a diagram showing the functional configuration of the machine learning model generation device 15. The machine learning model generation device 15 has a reception unit 41, a storage unit 42, an information attachment unit 43, a learning model generation unit 44, an output unit 45, an operation input unit 46, and a display unit 47. The machine learning model generation device 15 is realized by a computer and has, for example, the same configuration as that shown in Fig. 2. The CPU and GPU execute arithmetic processing using RAM and fixed disks in accordance with a program, causing the computer to function as the machine learning model generation device 15.

[0078] That is, the CPU, GPU, ROM, RAM, fixed disk, communication unit, input unit, display, and their peripheral components realize a reception unit 41, a storage unit 42, an information providing unit 43, a learning model generation unit 44, an output unit 45, an operation input unit 46, and a display unit 47 in Fig. 12. All or part of these functions may be realized by dedicated electric circuits, or these functions may be realized by individual programs. Furthermore, these functions may be realized by multiple computers.

[0079] The machine learning model generation device 15 is usually connected to the communication network 14 in FIG. 1, but may also be provided as a function of the additional inspection device 12 or as a function of the inspection device 11.

[0080] The receiving unit 41 receives defect images (more precisely, defect image data 85) used to generate training data from the inspection device 11 and stores them in the storage unit 42. For example, the receiving unit 41 is a communication unit such as an interface that communicates with the communication network 14 and a functional configuration that controls the communication unit, and the defect images are stored on a fixed disk or the like that functions as the storage unit 42. The defect image data 85 may be received from the inspection device 11 or from a storage device in which the defect images are saved. The receiving unit 41 receives inspection information 86, which is information related to the inspection when the defect was detected, along with the defect image data 85, and stores the inspection information 86 in the storage unit 42 as well. The inspection information 86 is the same information as described above for the inspection information 82.

[0081] The information providing unit 43 has an image selecting unit 431. The image selecting unit 431 selects (data of) a target defect image. The operation input unit 46 is realized by a computer keyboard and mouse and a program that controls them. The display unit 47 is realized by a computer display and a program that controls it. The operator selects inspection information via the operation input unit 46. The image selecting unit 431 selects, as a defect image for training data, defect image data 85 that is associated with the same inspection information selected by the operator from the inspection information 86 stored in the memory unit 42.

[0082] The selected defect image is displayed on the display unit 47, and the operator inputs via the operation input unit 46 whether the defect image is a false alarm or an image showing a real defect. That is, the operator inputs the authenticity of the defect image. The input information is stored in the memory unit 42 as authenticity information 87 in association with the corresponding defect image data 85. Note that "storing in the memory unit" is synonymous with "saving in the memory unit." When input for one defect image is complete, the image selection unit 431 selects the next defect image of the selected inspection information and displays it on the display unit 47. The operator then looks at the displayed defect image and inputs the authenticity.

[0083] The learning model generation unit 44 has an image selection unit 441. The image selection unit 441 selects a combination of defect image data 85 and authenticity information 87 associated with the same inspection information 86 as the inspection information selected by the operator. The learning model generation unit 44 performs learning using many combinations of the selected defect image data 85 and authenticity information 87 as training data, generates a machine learning model 7, and stores it in the storage unit 42. In this way, the learning model generation unit 44 generates a machine learning model 7 for each piece of inspection information. The generated multiple machine learning models 7 are sent to the additional inspection device 12 via the output unit 45 and stored in the re-inspection unit 33 of the additional inspection device 12. The output unit 45 is a communication unit such as an interface that communicates with the communication network 14, and a functional configuration that controls this.

[0084] Next, the flow of operations performed by the machine learning model generating device 15 to generate the machine learning model 7 will be further described with reference to FIG. 13. When a large number of combinations of defect image data 85 and inspection information 86 output from the inspection device 11 are prepared in the storage unit 42, the operator selects one piece of inspection information from a list of inspection information displayed on the display unit 47 (step S31). Note that the selection of inspection information may be performed automatically. The image selecting unit 431 of the information assigning unit 43 reads the first piece of defect image data 85 associated with the same inspection information 86 as the selected inspection information (hereinafter referred to as "selected inspection information") and displays the defect image on the display unit 47 (step S32). While visually checking the defect image, the operator inputs via the operation input unit 46 whether the defect image represents a false alarm or a real defect, and the information assigning unit 43 accepts the input of true / false as true / false information 87 (step S33). The information assigning unit 43 associates the true / false information 87 with the defect image data 85 and stores them in the storage unit 42 (step S34).

[0085] If there is next defect image data 85 associated with the same inspection information 86 as the selected inspection information, the information assigning unit 43 reads the defect image data 85 and repeats steps S32 to S34 to associate authenticity information 87 with the defect image data 85 and store it in the storage unit 42. Steps S32 to S35 are repeated until there is no defect image data 85 associated with the same inspection information 86 as the selected inspection information that is not associated with authenticity information 87, i.e., no defect image data to which authenticity information 87 has not been assigned. This generates teaching data related to the selected inspection information.

[0086] Upon receiving a signal indicating the completion of the generation of the training data from the information assignment unit 43 or upon receiving an input from the operator to start the generation of the machine learning model, the learning model generation unit 44 reads out the training data from the storage unit 42, i.e., multiple combinations of defect image data 85 associated with the same inspection information 86 as the selected inspection information and the associated true / false information 87. Then, a machine learning model 7 corresponding to the selected inspection information is generated by machine learning using the training data (step S36). The machine learning model 7 includes (or is associated with) information on the selected inspection information.

[0087] When generation of the machine learning model 7 for one piece of test information is completed, the selected test information is updated automatically or by the operator selecting the next piece of test information (steps S37 and S31). Steps S32 to S35 are then performed according to the updated selected test information to generate training data, and a machine learning model 7 is generated in step S36 (step S36). Steps S31 to S36 are performed for each piece of test information, or for each piece of test information desired by the operator (step S37), to generate a machine learning model 7 for each piece of test information. The multiple machine learning models 7 are different from each other. The operational flow of FIG. 13 may be modified as appropriate, or step S36 may be skipped and training data may be generated for each piece of test information, and then a machine learning model 7 may be generated for each piece of test information. The multiple machine learning models 7 are sent to the additional test device 12 and stored in the re-testing unit 33.

[0088] Conventional additional inspection devices use a single machine learning model to determine various types of defects across the entire surface of a printed wiring board. Therefore, generating a machine learning model requires an operator to input classifications (i.e., assign true / false information) for a huge number of defect images (e.g., tens of thousands). As a result, even experienced operators make misclassifications. In contrast, the additional inspection device 12 stores a machine learning model 7 for each piece of inspection information. This allows an operator to input true / false information for each piece of inspection information when creating training data, allowing for continuous judgments on similar defect images with limited characteristics (e.g., images of hole defects). As a result, the operator can easily assign true / false information, improving the efficiency of generating training data for creating a machine learning model. This also reduces operator misclassifications (i.e., incorrect input of true / false information).

[0089] Furthermore, since the number of defect images required to generate one machine learning model 7 is limited to the number of defect images associated with one piece of inspection information, the number of defect images for which true / false information is input can be reduced compared to when generating a conventional machine learning model.

[0090] In addition, in the past, whenever the product number of a printed wiring board changed and the design of the printed wiring board changed, it was necessary to input the true / false information for tens of thousands of defect images and regenerate the machine learning model.In contrast, the additional inspection device 12 in Figure 3 stores a machine learning model 7 for each piece of inspection information, so even if the product number of the printed wiring board changes, some or all of the machine learning models 7 can be used as is, and the machine learning models 7 can be used efficiently.

[0091] For example, machine learning model 7a may be generated corresponding to the inspection information for "hole inspection" of a printed wiring board with part number A, and machine learning model 7a may be used when re-inspecting a defect image for a printed wiring board with part number B, where the inspection information for "hole inspection" is "hole inspection." Alternatively, machine learning model 7c may be generated corresponding to the inspection information for "hole inspection" of a printed wiring board with part number C, and machine learning model 7a may be switched to and used when re-inspecting a defect image for the printed wiring board with part number C, where the inspection information for "hole inspection" is "hole inspection." In other words, one or more machine learning models 7 corresponding to one piece of inspection information may be generated for printed wiring boards with multiple part numbers, and the learning model selection unit 331 in FIG. 3 may select one of the one or more machine learning models 7 according to the part number.

[0092] Furthermore, the learning model selection unit 331 may select at least one arbitrary machine learning model 7 from a plurality of machine learning models 7 corresponding to a plurality of pieces of inspection information according to the product number of the printed wiring board. That is, the learning model selection unit 331 selects whether to use or not use each of the plurality of machine learning models 7 according to the product number of the printed wiring board.

[0093] In the above description, the defect image is associated with the inspection type, which is one type of inspection information. However, as previously mentioned, the defect image may be associated with inspection information other than the inspection type. In a preferred example, a plurality of machine learning models 7 are provided for each functional component of the wiring of the printed wiring board. Examples of the functional component of the wiring include pads, wiring, and through-holes. In another preferred example, a plurality of machine learning models 7 are provided for each material constituting the printed wiring board. Examples of materials constituting the printed wiring board include solder resist, copper, silk printing, plating, a substrate, solder resist on copper (pattern), and solder resist on a substrate. In yet another preferred example, a plurality of machine learning models 7 are provided for each type of defect. Of course, the machine learning models 7 may be provided corresponding to at least one element selected from the inspection type, the functional component of the wiring, the material constituting the printed wiring board, and the type of defect.

[0094] Even when the inspection information is as in the above example, transmission of the inspection information from inspection device 11 to additional inspection device 12 may be omitted and the inspection information may be obtained from the defect image in additional inspection device 12. In this case, additional inspection device 12 determines information related to the inspection selected and performed in inspection device 11 from the defect image.

[0095] The configurations of the above-described embodiment and each modification may be combined as appropriate as long as they are not mutually contradictory. [Explanation of symbols]

[0096] 1. Inspection system 7 Machine Learning Models 11 Inspection equipment 12 Additional testing equipment 13 Defect checking device 31 Reception 33 Re-examination Department 81 Defect image data 82 Test Information 91 Programs 811,821,831,841,851,861 Defective images 843 Plating Area 844 Foreign matter area 862 solder resist area 864 Copper exposed area Steps S21~S26 Steps S31~S37

Claims

1. An additional inspection device to be added to an inspection device that inspects the appearance of a printed wiring board, a receiving unit that receives a defect image showing a defect in the printed wiring board detected by the inspection device; a re-inspection unit that re-inspects the defect image to determine whether it shows a defect; Equipped with the defect image is at least a portion of an image determined to indicate a defect based on whether or not the image satisfies an inspection criterion in an inspection selected from a plurality of types of inspection in the inspection device; An additional inspection device in which the re-inspection unit re-inspects whether the defect image indicates a defect using a first machine learning model when the inspection information, which is information related to the inspection selected in the inspection device, is first inspection information, and re-inspects whether the defect image indicates a defect using a second machine learning model different from the first machine learning model when the inspection information is second inspection information.

2. The additional inspection device according to claim 1, The receiving unit receives, from the inspection device, the defect image and also the inspection information, which is information related to the inspection performed by the inspection device when the defect image was detected.

3. The additional inspection device according to claim 1, An additional inspection device further comprising an output unit that outputs the defect image to a defect confirmation device for an operator to confirm the defect image when the re-inspection unit determines that the defect image indicates a defect.

4. The additional inspection device according to claim 1, an additional inspection device in which at least some of a plurality of pieces of inspection information corresponding to a plurality of types of inspections performed by the inspection device are associated with functional components of wiring on the printed wiring board;

5. The additional inspection device according to claim 1, An additional inspection device in which at least some of a plurality of pieces of inspection information relating to a plurality of types of inspections performed by the inspection device are associated with a material that constitutes the printed wiring board.

6. The additional inspection device according to claim 1, An additional inspection device in which the reinspection unit does not perform a reinspection when the inspection information is third inspection information.

7. 7. The additional inspection device according to claim 6, An additional inspection device, wherein the third inspection information is information indicating an inspection for defects in which copper is exposed in the solder resist region.

8. 7. The additional inspection device according to claim 6, An additional inspection device, wherein the third inspection information is information indicating an inspection for defects where solder resist has adhered to a plating region.

9. An inspection system for inspecting the appearance of a printed wiring board, comprising: An additional inspection device according to any one of claims 1 to 8; the inspection device that outputs a defect image showing a defect in the printed wiring board to the additional inspection device; An inspection system comprising:

10. A program that causes a computer to function as an additional inspection device that is added to an inspection device that inspects the appearance of a printed wiring board, wherein the execution of the program by the computer includes the steps of: a) receiving a defect image showing a defect of the printed wiring board detected by the inspection device; b) re-examining the defect image to determine whether it shows any defects; Execute the defect image is an image that is determined to indicate a defect based on whether or not it satisfies an inspection standard in an inspection selected from a plurality of types of inspection in the inspection device, The step b) a step of re-inspecting whether the defect image indicates a defect using a first machine learning model when inspection information, which is information related to the type of inspection selected in the inspection device, is first inspection information; If the inspection information is second inspection information, re-inspecting whether the defect image indicates a defect using a second machine learning model different from the first machine learning model; Programs including.

11. 1. A machine learning model generation method for generating a machine learning model to be used for reinspection in an additional inspection device that is added to an inspection device that inspects the appearance of a printed wiring board and reinspects defect images output from the inspection device, comprising: a) for each of a plurality of pieces of inspection information that are information related to a plurality of types of inspections performed by the inspection device, a1) receiving from an operator authenticity information that is an input of authenticity of a defect image output from the inspection device, a2) associating the defect image with the authenticity information, and a3) repeating the a1) and a2) steps; b) generating a machine learning model by learning using a plurality of combinations of defect images and associated authenticity information for each of the plurality of pieces of inspection information; A machine learning model generation method including:

Images