Inspection support device, inspection support method and computer-readable program

Abstract

To improve visual determination accuracy of an operator to a defect candidate image.SOLUTION: An inspection support device supports inspection on a substrate having a pattern on its surface. A storage part 301 stores a plurality of defect candidate images detected through primary inspection on a substrate image. An image classification part 302 performs rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based upon defect kinds or fatal degrees of defect candidates included in the respective defect candidate images. The image classification 302 also generates classification display data enabling a display part to display the plurality of defect candidate images by the classification image groups. Consequently, visual determination accuracy of an operator to defect candidate images can be improved.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to techniques for assisting in the inspection of substrates having patterns on their surfaces. [Background technology]

[0002] Conventionally, in the inspection of a printed wiring board, an inspection device inspects images of each area of ​​the printed wiring board and extracts defect candidate images that are determined to have defects. Then, a defect confirmation operation (so-called verify operation) is performed in which an operator visually checks the area on the printed wiring board that corresponds to the defect candidate image. The defect candidates extracted by the inspection device include true defects that are likely to cause quality problems and false reports that are substantially unlikely to cause quality problems, and in the above-mentioned defect confirmation operation, an operator visually extracts true defects from the defect candidates extracted by the inspection device.

[0003] Also, Patent Document 1 proposes an appearance inspection system that includes a primary inspection unit that performs defect judgment based on an image of an object without using machine learning, and a secondary inspection unit that uses a machine learning model to distinguish between true defective products (true defects) and over-judged products (false reports). This improves productivity by detecting only true defective products in the appearance inspection and suppressing over-judged products from being detected as defective products.

[0004] In order to perform defect detection using the above-mentioned machine learning model, it is necessary to create learning data for the machine learning model in advance. When creating the learning data, a large number of images (e.g., 10,000 or more) including defect candidates are displayed on a display in predetermined numbers, and an operator visually checks each displayed image to determine whether the defect candidates in each image are real defects or false alarms. If the defect candidates are real defects, the operator also determines the type of the real defect. The operator inputs these determination results using a mouse or keyboard, and performs a teaching operation (i.e., annotation) to associate the results with each image. In this way, the above-mentioned learning data is created (see Patent Document 2). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2021-177154 A [Patent Document 2] Patent Publication No. 2021-139769 Summary of the Invention [Problem to be solved by the invention]

[0006] Incidentally, in order to create a machine learning model with high accuracy in defect detection, it is necessary to suppress deviations in teaching in the creation of the above-mentioned learning data. Specifically, in the above-mentioned teaching work, it is necessary to teach the same judgment result to similar defect candidate images. However, in the creation of learning data, as described above, teaching work is required for a large number of defect candidate images, so there is a risk that the operator's judgment criteria will change over time, etc., causing deviations in judgment. As a result, in the machine learning model created using the learning data, there is a risk that the accuracy of defect detection will be low, and re-teaching work for a large number of defect candidate images will be required.

[0007] The present invention has been made in consideration of the above-mentioned problems, and has an object to improve the accuracy of visual judgment by an operator on defect candidate images. [Means for solving the problem]

[0008] A first aspect of the present invention is an inspection support device that supports the inspection of a substrate having a pattern on its surface, and includes a memory unit that stores a plurality of defect candidate images detected by a primary inspection of a substrate image, and an image classification unit that performs rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate contained in each defect candidate image, and generates classified display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit.

[0009] A second aspect of the present invention is the inspection support device of the first aspect, further comprising: a display unit that displays the plurality of defect candidate images for each of the classified image groups; an input receiving unit that receives an input of an evaluation of a defect or false alarm by an operator who visually inspects each defect candidate image displayed on the display unit as a true / false evaluation of each of the defect candidate images; a learning dataset generation unit that generates a learning dataset by associating the true / false evaluation with each of the defect candidate images in the classified display data; and a learning unit that creates a trained model to be used for AI inspection of substrate images by performing machine learning on an initial model using the learning dataset.

[0010] A third aspect of the present invention relates to the inspection support device of the second aspect, further comprising a provisional setting unit that sets a provisional authenticity evaluation for each defect candidate image included in each classified image group based on a defect type or criticality corresponding to each classified image group. When the input receiving unit receives an input different from the provisional authenticity evaluation, the learning dataset generation unit associates the authenticity evaluation received by the input receiving unit with the defect candidate image, and otherwise associates the provisional authenticity evaluation with the defect candidate image.

[0011] A fourth aspect of the present invention is the inspection support device according to any one of the first to third aspects, wherein the image classification unit classifies each of the defect candidate images based on the criticality of the defect candidate included in each of the defect candidate images. The criticality of the defect candidate is determined based on the size of the defect candidate included in each of the defect candidate images and the width of a pattern element or a background region at the position of the defect candidate.

[0012] A fifth aspect of the present invention is the inspection support device according to any one of the first to third aspects (or any one of the first to fourth aspects), in which the image classification unit classifies each of the defect candidate images based on the criticality of the defect candidate included in each of the defect candidate images. The criticality of the defect candidate is determined based on the concentration of the defect candidate included in each of the defect candidate images.

[0013] A sixth aspect of the present invention is the inspection support device according to any one of the first to third aspects (or any one of the first to fifth aspects), wherein the image classification unit classifies each of the defect candidate images based on the defect types of the defect candidates included in each of the defect candidate images. The image classification unit converts each of the defect candidate images into a first image expressed in a first color that is any one of RGB. The image classification unit generates a second image in which a reference image that does not have a defect candidate included in each of the defect candidate images is expressed in a second color of RGB that is different from the first color. The image classification unit classifies a third image obtained by superimposing and synthesizing the first image and the second image, by classifying defect types in an area displayed in the first color as bright defect candidates and defect types in an area displayed in the second color as dark defect candidates.

[0014] A seventh aspect of the present invention is the inspection support device according to any one of the first to third aspects (or any one of the first to sixth aspects), in which the image classification unit classifies each of the defect candidate images based on both the defect type and the criticality of the defect candidate included in each of the defect candidate images. In the classification display data, two or more defect candidate images corresponding to one defect type can be displayed based on the magnitude relationship of the criticality.

[0015] An eighth aspect of the present invention is an inspection support method for supporting the inspection of a substrate having a pattern on its surface, comprising the steps of: a) storing a plurality of defect candidate images detected by a primary inspection of a substrate image; and b) performing rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate contained in each defect candidate image, and generating classified display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit.

[0016] A ninth aspect of the present invention is a computer-readable program that causes a computer to execute assistance in inspection of a substrate having a pattern on its surface, the program causing the computer to execute the steps of: a) storing a plurality of defect candidate images detected by a primary inspection of a substrate image; and b) performing rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate contained in each defect candidate image, and generating classified display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit. Effect of the Invention

[0017] According to the present invention, it is possible to improve the accuracy of visual judgment by an operator on a defect candidate image. [Brief description of the drawings]

[0018] [Figure 1] 1 is a diagram showing an inspection support device according to an embodiment together with a primary inspection device; [Diagram 2] FIG. 1 is a diagram illustrating a configuration of an inspection support device. [Diagram 3] FIG. 2 is a diagram illustrating a functional configuration of the inspection support device. [Figure 4] FIG. 13 is a diagram showing a flow of creating a trained model in the inspection support device. [Diagram 5] FIG. 1 is a diagram showing defect candidates of a plurality of defect types; [Figure 6] FIG. 13 is a diagram showing a display on which a defect candidate image is displayed. [Figure 7] FIG. 13 is a diagram showing a display on which a defect candidate image is displayed. [Figure 8] FIG. 13 is a diagram showing a display on which a defect candidate image is displayed. [Figure 9] FIG. 13 is a diagram showing a defect candidate image. [Figure 10] FIG. 13 is a diagram showing a defect candidate image. [Figure 11A] FIG. 13 is a diagram showing a defect candidate image. [Figure 11B]FIG. 13 is a diagram showing a defect candidate image. [Figure 11C] FIG. 13 is a diagram showing a defect candidate image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] FIG. 1 is a diagram showing an inspection support device 1 according to one embodiment of the present invention together with a primary inspection device 7. The inspection support device 1 is a device that supports visual inspection of, for example, a printed wiring board 9 (hereinafter also simply referred to as a "board 9") before electronic components are mounted thereon. A pattern (for example, a copper wiring pattern or an electrode pattern) is formed on the surface of the board 9, which is the inspection target of the visual inspection. In the inspection support device 1, for example, a trained model is created that is used for detecting defects in the board 9 by AI inspection.

[0020] The primary inspection device 7 is a device that performs a primary inspection on the substrate 9 to detect defect candidates and sends a defect candidate image including the defect candidates to the inspection support device 1. The primary inspection device 7 includes an image acquisition device 72 that images the substrate 9, and a computer 73. The computer 73 is a processing device that includes a calculation unit. The computer 73 also controls the overall operation of the primary inspection device 7. The image acquisition device 72 includes an imaging unit 74, a stage 75 that holds the substrate 9, and a stage movement mechanism 76.

[0021] The imaging unit 74 emits illumination light toward the substrate 9 and receives reflected light from the substrate 9 with an imaging sensor or the like, thereby capturing an image of the substrate 9. The image is, for example, a grayscale image of the entire region to be inspected on the substrate 9, and is also referred to as a "substrate image" in the following description. The substrate image captured by the imaging unit 74 may be, for example, a multi-tone color image. The stage 75 is a holding unit that holds the substrate 9. The stage 75 includes, for example, a vacuum chuck that adsorbs and holds the lower surface of the substrate 9. The stage movement mechanism 76 is composed of a ball screw, a guide rail, a motor, etc., and moves the stage 75 relative to the imaging unit 74.

[0022] The computer 73 has the configuration of a typical computer system including a CPU, GPU, ROM, RAM, fixed disk, display, keyboard, mouse, reading device, communication unit, etc. In the primary inspection device 7, the computer 73 controls the stage moving mechanism 76 and the imaging unit 74, so that the stage 75 moves in the horizontal direction and an image of a desired area on the substrate 9 is captured. The substrate image captured by the imaging unit 74 is sent to the computer 73.

[0023] In the computer 73, an inspection is performed on the board image, and defect candidates are detected. In the following description, the inspection by the primary inspection device 7 is also referred to as a "primary inspection." In the primary inspection, for example, an image showing a portion of the board 9 (i.e., a portion of the captured image) is subject to a single inspection, and the inspection is repeated over the entire area of ​​the board 9 to be inspected, thereby performing a primary inspection of the entire captured image. In the following description, the image that is subject to a single inspection in the primary inspection (i.e., an image showing a portion of the board 9) is also referred to as an "inspected image."

[0024] In the primary inspection, for example, a rule-based inspection is performed. The rule-based inspection is an inspection method in which a certain rule is set for the shape of the pattern on the substrate 9, and a portion of the pattern on the substrate 9 that does not comply with the rule is judged to be defective, and various methods are known. In the primary inspection, for example, a predetermined feature amount is calculated from each portion of the pattern included in the inspection image by a judgment algorithm created in advance based on a design rule, and the feature amount is compared with a threshold value for distinguishing between a non-defective product and a defect candidate. Then, based on a comparison result between the feature amount and the threshold value, it is judged whether the inspection image is a defect candidate image including a defect candidate (i.e., the presence or absence of a defect candidate in the inspection image). The feature amount is calculated based on, for example, the line width of a linear pattern element included in the pattern, the end shape of the pattern element, and the like. Note that, in the primary inspection, a predetermined image processing may be performed on the inspection image before extracting the feature amount.

[0025] An inspection target image determined to include a defect candidate in the primary inspection is sent from the primary inspection device 7 to the inspection support device 1 and stored in the inspection support device 1 as a defect candidate image. In the primary inspection device 7, for example, each time a defect candidate image is detected, the defect candidate image is transmitted to the inspection support device 1. In the inspection support device 1, a plurality of defect candidate images are stored as data (hereinafter also referred to as "defect candidate group data") arranged in the order of detection in the primary inspection device 7. Alternatively, in the primary inspection device 7, a plurality of defect candidate images detected in the primary inspection may be stored in the computer 73 as defect candidate group data arranged in the order of detection. In this case, the defect candidate group data is transmitted from the primary inspection device 7 to the inspection support device 1 after the primary inspection of all the inspection target images is completed. The number of defect candidate images included in the defect candidate group data is, for example, 10,000 or more. In addition, an inspection target image determined not to include a defect candidate in the primary inspection is stored in the computer 73 as a non-defective image, for example.

[0026] 2 is a diagram showing the configuration of the test support device 1. The test support device 1 has a configuration of a general computer system including a CPU 31, a ROM 32, a RAM 33, a fixed disk 34, a display 35, an input unit 36, a reading device 37, a communication unit 38, a GPU 39, and a bus 30. The CPU 31 performs various arithmetic processing. The GPU 39 performs various arithmetic processing related to image processing. The ROM 32 stores a basic program. The RAM 33 stores various information. The fixed disk 34 stores information. The display 35 is a display unit that displays various information such as images. The input unit 36 ​​includes a keyboard 36a and a mouse 36b that receive input from an operator.

[0027] The reading device 37 reads information from a computer-readable recording medium 81 such as an optical disk, a magnetic disk, a magneto-optical disk, or a memory card. The display 35, the keyboard 36a, the mouse 36b, and the reading device 37 are connected to the bus 30 via an interface I / F. The communication unit 38 transmits and receives signals to and from other components of the examination support device 1 and external devices. The bus 30 is a signal circuit that connects the CPU 31, the GPU 39, the ROM 32, the RAM 33, the fixed disk 34, the display 35, the input unit 36, the reading device 37, and the communication unit 38.

[0028] In the inspection support device 1, the program 811 is read in advance from the recording medium 81 via the reading device 37 and stored in the fixed disk 34. The program 811 may be stored in the fixed disk 34 via a network. The CPU 31 and the GPU 39 execute arithmetic processing while using the RAM 33 and the fixed disk 34 in accordance with the program 811. The CPU 31 and the GPU 39 function as a calculation unit. Other configurations functioning as a calculation unit other than the CPU 31 and the GPU 39 may be employed.

[0029] FIG. 3 is a diagram showing a functional configuration realized by the inspection support device 1 executing arithmetic processing and the like according to the program 811. These functional configurations include a storage unit 301, an image classification unit 302, a provisional setting unit 303, a display control unit 304, an input reception unit 305, a learning data set generation unit 306, and a learning unit 307. All or a part of these functions may be realized by a dedicated electric circuit. Also, these functions may be realized by a plurality of computers. Among the functional configurations shown in FIG. 3, the image classification unit 302, the provisional setting unit 303, the display control unit 304, the input reception unit 305, the learning data set generation unit 306, and the learning unit 307 are realized by the CPU 31, the GPU 39, the ROM 32, the RAM 33, the fixed disk 34, and their peripheral configurations. Also, the storage unit 301 is mainly realized by the RAM 33 and the fixed disk 34.

[0030] Next, a flow of creating the above-mentioned trained model in the inspection support device 1 will be described with reference to FIG. 4 etc. First, the above-mentioned defect candidate group data (i.e., data of a plurality of defect candidate images) sent from the primary inspection device 7 (see FIG. 1) to the inspection support device 1 is stored by the storage unit 301 shown in FIG. 3 (step S11). In addition to the defect candidate group data, various information related to the inspection of the substrate 9 is also stored in advance in the storage unit 301. In the defect candidate group data, as described above, the plurality of defect candidate images are arranged in the order in which they were detected by the primary inspection device 7. Therefore, in the defect candidate group data, the plurality of defect candidate images are arranged randomly (i.e., in a disorderly manner) with respect to the defect type indicating the type of the defect candidate and the criticality of the defect candidate.

[0031] Next, the image classification unit 302 reads out the defect candidate group data stored in the storage unit 301, and classifies each of the plurality of defect candidate images included in the defect candidate group data. The image classification unit 302 classifies each defect candidate image based on the defect type and / or the criticality of the defect candidate included in each defect candidate image.

[0032] Fig. 5 is a diagram showing examples of defect candidates of a plurality of defect types included in the defect candidate image 61. In Fig. 5, a plurality of pattern elements 611-613 included in the defect candidate image 61 are marked with parallel diagonal lines. Note that in the actual defect candidate image 61, the pattern elements 611-613 are bright regions that are brighter than the surrounding background region 614, and the background region 614 is a dark region that is brighter than the pattern elements 611-613. Also, the modes of the defect types of the defect candidates and the names of the defect types shown below may be changed in various ways.

[0033] Defect candidate D11 is an "open defect (i.e., a break)" in which the linear pattern element 611 is completely interrupted in the longitudinal direction. Defect candidates D12 to D14 are "chips" in which a portion of the linear pattern element 612 is missing in the width direction. Defect candidate D15 is a "void" in which a dot-like portion of the solid pattern element 613 is missing. Defect candidates D11 to D15 are defect candidates in which a pattern element is not present in a position where a pattern element should originally be present. Defect candidates D11 to D15 are defect candidates with lower brightness than pattern elements 611 to 613, and are hereinafter also referred to as "dark defect candidates".

[0034] The defect candidate D21 is a "short defect (i.e., a short circuit)" in which the pattern element 611 and the pattern element 612 that should not be connected are connected. The defect candidates D22 to D24 are "protrusions" in which an unnecessary convex portion of the pattern element 611 protrudes in the width direction. The defect candidate D25 is a "foreign object" in which an isolated point-like unnecessary pattern element or the like exists in the background region 614. The defect candidates D21 to D25 are defect candidates in a mode in which some pattern element exists in a position where no pattern element should exist. The defect candidates D21 to D25 are defect candidates that are brighter than the background region 614, and are hereinafter also referred to as "bright defect candidates".

[0035] In Figure 5, multiple defect candidates D11 to D15, D21 to D25 are depicted in one defect candidate image 61 to make it easier to understand the shape and characteristics of each defect candidate, but the number of defect candidates contained in the actual defect candidate image 61 is, for example, one.

[0036] The criticality of a defect candidate is an index showing the degree of adverse effect that the defect candidate has on the performance of the substrate 9. For example, the criticality of the defect candidate D11 (defect type: open defect) and the defect candidate D21 (defect type: short defect) is high, and the criticality of the defect candidate D15 (defect type: missing) and the defect candidate D25 (defect type: foreign matter) is low. The criticality of the defect candidates D12 to D14 (defect type: chipping) is medium, and among the defect candidates D12 to D14, for example, the criticality increases as the missing portion of the pattern element 612 becomes larger (i.e., in the order of the defect candidates D14, D13, and D12). The criticality of the defect candidates D22 to D24 (defect type: protrusion) is medium, and among the defect candidates D22 to D24, for example, the criticality increases as the convex portion protruding from the pattern element becomes larger (i.e., in the order of the defect candidates D24, D23, and D22).

[0037] The classification of the criticality of the defect candidates described above may be changed in various ways. For example, for the defect candidate D15 (missing part), two or more classifications may be set based on the magnitude relationship of the criticality so that the criticality increases as the missing part becomes larger. Similarly, for the defect candidate D25 (foreign matter), two or more classifications may be set based on the magnitude relationship of the criticality so that the criticality increases as the foreign matter becomes larger.

[0038] The image classification unit 302 performs rule-based classification on the multiple defect candidate images included in the defect candidate group data, thereby implementing the above classification of the defect candidate images. In rule-based classification, certain rules are set for the shapes, etc. of pattern elements in the defect candidate images, and the defect type and criticality of the defect candidate are determined based on the conformity, etc., to the rules. Various specific methods of rule-based classification are known.

[0039] In the rule-based classification, for example, a predetermined feature amount is calculated from each portion of a pattern element included in a defect candidate image by a judgment algorithm created in advance based on a design rule, and the feature amount is compared with a threshold value for discriminating the predetermined defect candidate. Then, based on a comparison result between the feature amount and the threshold value, the defect type and the criticality of the defect candidate included in the inspection image are judged. The feature amount is calculated based on, for example, the size of the defect candidate, the distance between the defect candidate and a pattern element adjacent to the defect candidate, etc. In the rule-based classification, a predetermined image processing may be performed on the defect candidate image before the feature amount is extracted.

[0040] In this embodiment, the image classification unit 302 performs the rule-based classification on the defect candidate images included in the defect candidate group data, thereby classifying the defect candidate images into a plurality of classified image groups set based on both the defect type and the criticality of the defect candidate included in each defect candidate image (step S12). Specifically, ten classified image groups G11-G15 and G21-G25 corresponding to the above-mentioned defect candidates D11-D15 and defect candidates D21-D25, respectively, are set, and each defect candidate image is classified into one of the classified image groups G11-G15 and G21-G25.

[0041] In the image classification unit 302, one defect candidate image is read out in sequence in the order in which the defect candidate images are arranged in the defect candidate group data (i.e., in the order in which they are detected by the primary inspection device 7), and the defect candidate image is classified into a plurality of classified image groups G11-G15, G21-G25 based on both the defect type and the criticality of the defect candidate included in the one defect candidate image. Then, display data (hereinafter also referred to as "classified display data") that enables the defect candidate images included in the defect candidate group data to be displayed for each of the classified image groups G11-G15, G21-G25 on a display unit such as the display 35 is generated (step S13). Note that in the image classification unit 302, the classification of the defect candidate images into the classified image groups G11-G15, G21-G25 may be performed based on either the defect type or the criticality of the defect candidate.

[0042] Next, the provisional setting unit 303 sets provisional authenticity evaluations for the defect candidate images included in each of the classified image groups G11-G15, G21-G25 based on the defect types and criticalities corresponding to the classified image groups G11-G15, G21-G25, respectively (step S14). The provisional authenticity evaluations are the results of provisional determination as to whether the defect candidates included in the defect candidate images are defects (hereinafter also referred to as "true defects") that are likely to cause quality problems for the substrate 9, or false reports that are unlikely to cause such problems. The provisional setting unit 303 collectively sets the same provisional authenticity evaluations for all the defect candidate images included in one classified image group.

[0043] For example, the provisional authenticity evaluation of each defect candidate image included in the classified image groups G11 and G21 is set to "true defect." That is, if the defect type of the defect candidate is "open defect (D11)" or "short defect (D21)," which are highly fatal, the provisional authenticity evaluation of the defect candidate image including the defect candidate is set to "true defect." Also, the provisional authenticity evaluation of each defect candidate image included in the classified image groups G15 and G25 is set to "false report." That is, if the defect type of the defect candidate is "void (D15)" or "foreign matter (D25)," which are less fatal, the provisional authenticity evaluation of the defect candidate image including the defect candidate is set to "true defect."

[0044] The provisional authenticity evaluation of each defect candidate image included in the classified image groups G12 to G14, G22 to G24 is set to "true defect" or "false report" based on the criticality of the defect candidate of the defect candidate image included in each classified image group. That is, when the defect type of the defect candidate is "chipped (D12 to D14)" or "protrusion (D22 to D24)" with a medium criticality, the provisional authenticity evaluation of the defect candidate image included in each classified image group is set to "true defect" or "false report" based on the criticality corresponding to each classified image group. For example, the provisional authenticity evaluation of each defect candidate image included in the classified image groups G12 to G13, G22 to G23 with a relatively high criticality among the classified image groups G12 to G14, G22 to G24 is set to "true defect", and the provisional authenticity evaluation of each defect candidate image included in the classified image groups G14, G24 with a relatively low criticality is set to "false report".

[0045] The provisional authenticity evaluation of each defect candidate image set in the provisional setting unit 303 is associated with each defect candidate image in the classification display data generated by the image classification unit 302. Note that the provisional setting unit 303 may set the provisional authenticity evaluation for the defect candidate image based on either the defect type or the fatality corresponding to the classification image groups G11 to G15, G21 to G25.

[0046] Next, the display control unit 304 controls the display 35 etc. based on the classification display data generated by the image classification unit 302, and the defect candidate images are displayed on the display 35 for each of the classification image groups G11 to G15, G21 to G25 (step S15). Specifically, a plurality of defect candidate images included in one classification image group out of the classification image groups G11 to G15, G21 to G25 are sequentially displayed on the display 35. Switching from one defect candidate image to the next defect candidate image on the display 35 is performed, for example, by an operator operating the inspection support device 1.

[0047] Then, when the display of all the defect candidate images included in the one classified image group has been completed, the defect candidate images included in the next classified image group are displayed in sequence on the display 35. In the inspection support device 1, the processing is performed on all of the classified image groups G11 to G15, G21 to G25, thereby displaying all the defect candidate images included in the defect candidate group data in sequence. The number of defect candidate images displayed on the display 35 at one time may be one, or two or more. In this embodiment, the description will be given assuming that the number of defect candidate images 61 displayed on the display 35 at one time is one.

[0048] As shown in FIG. 6, when a defect candidate image 61 is displayed on the display 35, the defect type of the defect candidate included in the defect candidate image 61 (i.e., the defect type assigned to the defect candidate image 61 by the image classification unit 302, which is any one of “open defect”, “chipped”, “missing”, “short defect”, “protrusion”, and “foreign substance”) is displayed in a cell 62 adjacent to the defect candidate image 61. Also, a provisional true / false evaluation (i.e., true defect or false report) associated with the defect candidate image 61 by the provisional setting unit 303 is displayed in a cell 63 adjacent to the defect candidate image 61. In the example shown in FIG. 6, the cell 62 displays “open defect”, which is the defect type of the defect candidate D11, and the cell 63 displays “true defect”, which is the true / false evaluation provisionally set for the classification image group G11 including the defect candidate D11.

[0049] When the defect candidate image 61 is displayed on the display 35, the operator visually examines the defect candidate image 61 on the display 35 and considers whether the defect type in the cell 62 displayed together with the defect candidate image 61 and the provisional true / false evaluation in the cell 63 are correct.

[0050] For example, if the operator determines that the defect type displayed in cell 62 is incorrect, he or she changes the display content in cell 62. Specifically, as illustrated in FIG. 7, the operator moves the mouse cursor on display 35 to position it on cell 62 and clicks the mouse button to display pull-down menu 64. The operator changes the defect type assigned to defect candidate image 61 by image classification unit 302 by selecting one defect type from pull-down menu 64. On the other hand, if the operator determines that the defect type in cell 62 is correct, no particular operation is performed on cell 62. Note that the change of defect type by the operator is not limited to the above example and may be performed by various methods.

[0051] Also, for example, when the operator judges that the provisional authenticity evaluation displayed in the cell 63 is incorrect, the operator changes the display content in the cell 63. Specifically, as illustrated in FIG. 8, the operator moves the mouse cursor on the display 35 to place it on the cell 63, and clicks the mouse button to display the pull-down menu 65. The operator changes the provisional authenticity evaluation assigned to the defect candidate image 61 in the provisional setting unit 303 by selecting either a true defect or a false report from the pull-down menu 65. On the other hand, when the operator judges that the provisional authenticity evaluation in the cell 63 is correct, no particular operation is performed on the cell 63. Note that the change in authenticity evaluation by the operator is not limited to the above example, and may be performed by various methods.

[0052] When the operator has finished examining the defect type and the true / false evaluation of the defect candidate image 61 displayed on the display 35 and changing the defect type and the true / false evaluation as necessary, he / she places the mouse cursor on the decision button 66 displayed below the cells 62 and 63 in FIG. 6 and clicks the mouse button. As a result, the defect type displayed in the cell 62 and the true / false evaluation displayed in the cell 63 are input and are accepted by the input acceptance unit 305 (step S16). That is, the operation of clicking the decision button 66 by the operator is an input operation for inputting the defect type of the defect candidate image 61 and the evaluation of the defect candidate image 61 as a true defect or a false report via the input unit 36. Note that the manner of the input operation may be changed in various ways, such as input via the keyboard 36a. Furthermore, when the operator clicks the decision button 66, the defect candidate image 61 displayed on the display 35 is switched and the next defect candidate image in the classification display data is displayed.

[0053] In the inspection support device 1, the operator sequentially examines the above-mentioned defect type and authenticity evaluation for each of the multiple defect candidate images included in the classification display data (i.e., each defect candidate image classified into the classified image groups G11 to G15, G21 to G25), and changes the defect type and authenticity evaluation as necessary, and the results are accepted by the input accepting unit 305.

[0054] As described above, in the inspection support device 1, a plurality of defect candidate images are displayed on the display 35 for each of the classified image groups G11 to G15, G21 to G25. Specifically, for example, a plurality of defect candidate images included in the classified image group G11 are displayed in sequence, followed by a plurality of defect candidate images included in the classified image group G12. Next, a plurality of defect candidate images included in the classified image group G13 are displayed in sequence, a plurality of defect candidate images included in the classified image group G14 are displayed in sequence, and a plurality of defect candidate images included in the classified image group G15 are displayed in sequence. After that, a plurality of defect candidate images included in the classified image group G21 are displayed in sequence, followed by a plurality of defect candidate images included in the classified image group G22 are displayed in sequence. Next, a plurality of defect candidate images included in the classified image group G23 are displayed in sequence, a plurality of defect candidate images included in the classified image group G24 are displayed in sequence, and a plurality of defect candidate images included in the classified image group G25 are displayed in sequence.

[0055] That is, in the inspection support device 1, the multiple defect candidate images are displayed on the display 35 in a state where they are classified based on both the defect type and the criticality. Therefore, compared to a case where the multiple defect candidate images are randomly displayed with respect to the defect type and the criticality (for example, a case where the multiple defect candidate images are displayed in the order in which they were detected by the primary inspection device 7), the operator can continuously observe multiple defect candidate images having the same defect type and criticality together and determine their defect type and true / false evaluation. Therefore, it is possible to suppress changes in the operator's judgment criteria over time, etc., and to suppress inconsistencies in the judgment of the defect type and true / false evaluation of the defect candidate images.

[0056] When step S16 for all defect candidate images included in the classification display data is completed, the learning data set generation unit 306 associates the above-mentioned defect type and authenticity evaluation with each defect candidate image of the classification display data based on the information accepted by the input acceptance unit 305. That is, when the input acceptance unit 305 accepts an input of a defect type different from the defect type determined by the image classification unit 302 for each defect candidate image, the learning data set generation unit 306 associates the defect type accepted by the input acceptance unit 305 with the defect candidate image. On the other hand, in other cases (that is, when the defect type determined by the image classification unit 302 has not been changed), the defect type determined by the image classification unit 302 is directly associated with the defect candidate image.

[0057] Furthermore, when an input of an authenticity evaluation different from the provisional authenticity evaluation set in the provisional setting unit 303 is received by the input receiving unit 305 for each defect candidate image, the learning dataset generating unit 306 associates the authenticity evaluation received by the input receiving unit 305 with the defect candidate image. On the other hand, in other cases (i.e., when the provisional authenticity evaluation set in the provisional setting unit 303 has not been changed), the provisional authenticity evaluation set in the provisional setting unit 303 is directly associated with the defect candidate image. As a result, a learning dataset for creating a trained model used for defect detection of the substrate 9 by the AI ​​inspection is generated (step S17). Note that the learning dataset may include non-defective product images stored in the computer 73 in the above-mentioned primary inspection device 7.

[0058] When step S17 is completed, the learning unit 307 performs machine learning on the initial model using the above-mentioned learning dataset, and creates the above-mentioned trained model (step S18). The machine learning is performed, for example, by deep learning using a neural network. The learning by deep learning is performed, for example, by using ResNet. Note that the machine learning may be performed by a method other than deep learning.

[0059] The trained model created by the training unit 307 is sent to a secondary inspection device (not shown) that performs AI inspection on the defect candidate image to detect the true defect, and is used for the AI ​​inspection in the secondary inspection device. Note that the function of the secondary inspection device may be realized by a computer (see FIG. 2) that constitutes the inspection support device 1.

[0060] Next, a specific example of a classification method based on the fatality of the defect candidates in the above-mentioned step S12 will be described. For example, when the defect candidates D12 to D14, whose defect type is "chipped" shown in Fig. 5, are classified based on the fatality, the image classification unit 302 acquires the feature amounts of the defect candidates D12 to D14, and determines the fatality of the defect candidates D12 to D14 based on the feature amounts. Then, the fatality is compared with a predetermined threshold value to perform the rule-based classification of the defect candidate images each including the defect candidates D12 to D14.

[0061] 9 includes the size of the defect candidate D12 (i.e., the area of ​​the portion missing as the defect candidate D12 in the pattern element 612). The feature amount also includes the width B11 of the pattern element 612 at the position of the defect candidate D12 (i.e., the total width in the width direction of the pattern element 612 at the position where the defect candidate D12 exists in the pattern element 612, assuming that the defect candidate D12 does not exist).

[0062] The criticality determined based on the feature amount increases as the size of the defect candidate D12 increases. The criticality determined based on the feature amount increases as the ratio of the width B12 (i.e., the maximum width in the width direction) of the defect candidate D12 to the width B11 of the pattern element 612 at the position of the defect candidate D12 increases. Note that the widths B11 and B12 are not necessarily limited to the width in the width direction (i.e., the direction perpendicular to the longitudinal direction) of the pattern element 612, and may be the width in any direction intersecting the longitudinal direction.

[0063] Furthermore, the criticality of the defect candidate D12 may be determined based on the density of the defect candidate D12 in addition to or instead of the size of the defect candidate D12 and the width B11 of the pattern element 612 at the position of the defect candidate D12. In this case, the criticality of the defect candidate D12 increases as the density of the defect candidate D12, which is a dark defect candidate, becomes closer to the density of the background region 614 than to the density of the pattern element 612. On the other hand, the criticality of the defect candidate D12 decreases as the density of the defect candidate D12 becomes closer to the density of the pattern element 612 than to the density of the background region 614.

[0064] The feature amounts of the defect candidates D13 and D14 and the criticality determined based on the feature amounts are similar to those of the defect candidate D12. This makes it possible to determine the criticality of the defect candidates D12 to D14, whose defect type is "chip," by accurately considering the effect of the defect candidates D12 to D14 on the performance of the substrate 9.

[0065] Furthermore, when the defect candidates D22 to D24 (see FIG. 5) whose defect type is "protrusion" are classified based on the fatality, the image classification unit 302 acquires the feature amounts of the defect candidates D22 to D24, and determines the fatality of the defect candidates D22 to D24 based on the feature amounts. Then, the fatality is compared with a predetermined threshold value to perform the rule-based classification of the defect candidate images each including the defect candidates D22 to D24.

[0066] 10 includes the size of the defect candidate D22 (i.e., the area of ​​the defect candidate D22 protruding from the pattern element 611). The feature amount also includes a width B21 of the background region 614 at the position of the defect candidate D22 (i.e., the width in the width direction of the background region 614 adjacent to the pattern element 611 on the side where the defect candidate D22 protrudes at the position where the defect candidate D22 exists on the pattern element 611, and the total width in the width direction when it is assumed that the defect candidate D22 does not exist).

[0067] The criticality determined based on the feature amount increases as the size of the defect candidate D22 increases. The criticality determined based on the feature amount increases as the ratio of the width B22 (i.e., the maximum width in the width direction) of the defect candidate D22 to the width B21 (the interval between the pattern element 611 and the pattern element 612 in the example shown in FIG. 9) of the background region 614 at the position of the defect candidate D22 increases. Note that the widths B21 and B22 are not necessarily limited to the width in the width direction (i.e., the direction perpendicular to the longitudinal direction) of the pattern element 611, and may be the width in any direction intersecting the longitudinal direction.

[0068] Furthermore, the criticality of the defect candidate D22 may be determined based on the density of the defect candidate D22 in addition to or instead of the size of the defect candidate D22 and the width B21 of the background region 614 at the position of the defect candidate D22. In this case, the criticality of the defect candidate D22 increases as the density of the defect candidate D22, which is a bright defect candidate, becomes closer to the density of the pattern element 611 than to the density of the background region 614. On the other hand, the criticality of the defect candidate D22 decreases as the density of the defect candidate D22 becomes closer to the density of the background region 614 than to the density of the pattern element 611.

[0069] The feature amounts of the defect candidates D23 and D24 and the criticality determined based on the feature amounts are similar to those of the defect candidate D22. This makes it possible to determine the criticality of the defect candidates D22 to D24, whose defect type is "protrusion," by accurately considering the effect of the defect candidates D22 to D24 on the performance of the substrate 9.

[0070] It should be noted that the criticality of the defect candidate described above does not necessarily have to be determined based on the size, density, etc., of the defect candidate exemplified above, but may be determined based on other characteristic quantities of the defect candidate.

[0071] In the above example, the image classification unit 302 classifies the defect candidates into six types of defect types, namely, "open defect," "chipping," "void," "short defect," "protrusion," and "foreign matter," but the present invention is not limited to this. For example, the defect candidates may be classified into two types, the above-mentioned "bright defect candidate" and "dark defect candidate." Alternatively, after the defect candidates are classified into either "bright defect candidate" or "dark defect candidate," the defect candidates classified into "bright defect candidate" may be classified into either "open defect," "chipping," or "void," and the defect candidates classified into "dark defect candidate" may be classified into either "short defect," "protrusion," or "foreign matter."

[0072] When a defect candidate is classified as a "bright defect candidate" or a "dark defect candidate," for example, the defect candidate image 61 shown in FIG. 5 is converted into a first image 67 expressed in a first color (in this embodiment, "green (G)") that is any one of RGB, as shown in FIG. 11A. In FIG. 11A, the areas colored green in the first image 67 (i.e., the pattern elements 611-613 and the defect candidates D21-D25) are marked with parallel diagonal lines. In FIG. 11A, the background region 614 and the defect candidates D11-D15 are not colored.

[0073] 11B, the reference image corresponding to the defect candidate image 61 shown in FIG. 5 is converted into a second color (in this embodiment, "red (R)") different from the first color of RGB, and a second image 68 expressed in the second color is generated. The reference image is an image that does not have the defect candidates included in the defect candidate image 61. In other words, the reference image is an image in a state where all the defect candidates D11 to D15 and D21 to D25 included in the defect candidate image 61 have been removed (i.e., an inspection image that does not have any defect candidates). The reference image is stored in advance in, for example, the storage unit 301. In FIG. 11B, the areas colored red in the second image 68 (i.e., the pattern elements 611 to 613) are marked with parallel diagonal lines. In FIG. 11B, the background area 614 is not colored.

[0074] When the first image 67 and the second image 68 are prepared, the first image 67 and the second image 68 are superimposed and synthesized to generate a third image 69 shown in FIG. 11C. In the third image 69, the areas that are colored in both the first image 67 and the second image 68 (i.e., the pattern elements 611-613) are colored yellow. On the other hand, the areas that are colored in the first image 67 and not colored in the second image 68 (i.e., the defect candidates D21-D25) are displayed in the third image 69 in a first color, green. Moreover, the areas that are not colored in the first image 67 and are colored in the second image 68 (i.e., the defect candidates D11-D15) are displayed in the third image 69 in a second color, red.

[0075] In the image classification unit 302, the defect type of the area displayed in the third image 69 in the first color (i.e., defect candidates D21 to D25) is determined to be a "bright defect candidate," and the defect type of the area displayed in the third image 69 in the second color (i.e., defect candidates D11 to D15) is determined to be a "dark defect candidate." This method allows the defect candidates in the defect candidate image 61 to be easily classified as "bright defect candidates" or "dark defect candidates." Note that the defect candidates may be classified as "bright defect candidates" or "dark defect candidates" by methods other than those described above.

[0076] In the above example, in step S15, the multiple defect candidate images are displayed on the display 35 for each of the classified image groups G11 to G15, G21 to G25 (i.e., classified based on both the defect type and the criticality), but this is not limited to the above. For example, in step S15, the multiple defect candidate images may be displayed on the display 35 in a state where they are classified based on only either the defect type or the criticality.

[0077] Specifically, when a plurality of defect candidate images are classified based only on the defect type and displayed on display 35 based on the classification results, for example, defect candidate images each including defect candidates D12 to D14 whose defect type is "chip" are displayed in random order on display 35 as defect candidate images classified into the same defect type. Similarly, defect candidate images each including defect candidates D22 to D24 whose defect type is "protrusion" are displayed in random order on display 35 as defect candidate images classified into the same defect type.

[0078] Alternatively, when a plurality of defect candidate images are classified based only on the criticality and displayed on the display 35 based on the classification result, for example, defect candidate images each including defect candidates D11 and D21 with high criticality are displayed in random order on the display 35 as defect candidate images classified into the same grade of criticality. Similarly, defect candidate images each including defect candidates D15 and D25 with low criticality are displayed in random order on the display 35 as defect candidate images classified into the same grade of criticality.

[0079] As described above, the inspection support device 1 is a device that supports the inspection of the substrate 9 having a pattern on its surface. The inspection support device 1 includes a storage unit 301 and an image classification unit 302. The storage unit 301 stores a plurality of defect candidate images detected by a primary inspection of a substrate image. The image classification unit 302 performs rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups (classified image groups G11-G15, G21-G25 in the above example) based on the defect type or criticality of the defect candidate (defect candidates D11-D15, D21-D25 in the above example) included in each defect candidate image. The image classification unit 302 generates classification display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit (display 35 in the above example).

[0080] This allows the operator to continuously observe multiple defect candidate images having the same defect type or criticality collectively on the display unit. Therefore, when the operator makes a predetermined judgment on the multiple defect candidate images (in the above example, judgment on the defect type of the defect candidate and judgment on whether the defect is a real defect or a false alarm), it is possible to suppress changes in the operator's judgment criteria over time, etc., and to suppress inconsistencies in the judgment. In other words, the inspection support device 1 can improve the accuracy of the operator's visual judgment on the defect candidate images.

[0081] As described above, the inspection support device 1 preferably further includes a display unit (i.e., display 35), an input receiving unit 305, a learning data set generating unit 306, and a learning unit 307. The display 35 displays a plurality of defect candidate images for each of the classified image groups. The input receiving unit 305 receives an input of an evaluation of a defect (i.e., a true defect) or false information by an operator who visually inspects each defect candidate image displayed on the display 35, as a true / false evaluation of the previous defect candidate image. The learning data set generating unit 306 generates a learning data set by associating the true / false evaluation with each defect candidate image in the classified display data. The learning unit 307 creates a trained model used for AI inspection of a board image by performing machine learning on an initial model using the learning data set.

[0082] This makes it possible to generate a learning data set with less deviation in the operator's instruction for authenticity evaluation (i.e., high accuracy of authenticity judgment), and to create a trained model that can perform highly accurate authenticity judgment. As a result, it is possible to improve the inspection accuracy in AI inspection of circuit board images.

[0083] As described above, it is preferable that the inspection support device 1 further includes the provisional setting unit 303. The provisional setting unit 303 sets a provisional authenticity evaluation for each defect candidate image included in each classified image group, based on the defect type or criticality corresponding to each classified image group. When the input receiving unit 305 receives an input different from the provisional authenticity evaluation, the above-mentioned learning dataset generation unit 306 associates the authenticity evaluation received by the input receiving unit 305 with the defect candidate image, and otherwise associates the provisional authenticity evaluation with the defect candidate image.

[0084] In this way, by provisionally setting the authenticity evaluation by the provisional setting unit 303 before the authenticity evaluation is input by the operator who visually inspects each defect candidate image, it becomes more likely that the authenticity evaluation is input by the operator only for some of the defect candidates, or that it is not necessary to input the authenticity evaluation at all. As a result, the time required for the operator to input the authenticity evaluation can be shortened.

[0085] As described above, in the image classification unit 302, each defect candidate image is classified based on the criticality of the defect candidate included in each defect candidate image, and the criticality of the defect candidate is preferably determined based on the size of the defect candidate included in each defect candidate image and the width of the pattern element or background region at the position of the defect candidate. This makes it possible to accurately determine the criticality by appropriately considering the effect of the defect candidate on the performance of the substrate 9.

[0086] As described above, it is also preferable that the image classification unit 302 classifies each defect candidate image based on the criticality of the defect candidate contained in the defect candidate image, and the criticality of the defect candidate is determined based on the concentration of the defect candidate contained in the defect candidate image. This makes it possible to accurately determine the criticality by appropriately considering the effect of the defect candidate on the performance of the substrate 9.

[0087] As described above, it is also preferable that the image classification unit 302 classifies each defect candidate image based on the defect type of the defect candidate included in each defect candidate image. Preferably, the image classification unit 302 converts each defect candidate image into a first image 67 expressed in a first color (green (G) in the above example) that is one of RGB. The image classification unit 302 also generates a second image 68 expressed in a reference image that does not have a defect candidate included in each defect candidate image in a second color (red (R) in the above example) that is different from the first color of RGB. Then, in a third image 69 obtained by superimposing and synthesizing the first image 67 and the second image 68, the image classification unit 302 classifies the defect type of the area displayed in the first color as a bright defect candidate and the defect type of the area displayed in the second color as a dark defect candidate. This makes it possible to easily classify the defect candidates in the defect candidate images into "bright defect candidates" or "dark defect candidates."

[0088] As described above, in the image classification unit 302, each defect candidate image is classified based on both the defect type and the criticality of the defect candidate included in each defect candidate image, and it is also preferable that the classification display data can display two or more defect candidate images corresponding to one defect type based on the magnitude relationship of the criticality. In this way, by sequentially displaying a plurality of defect candidate images classified into the same defect type by criticality (for example, in descending order of criticality), it is possible to further prevent inconsistencies in the judgment by the operator. In other words, it is possible to further improve the accuracy of the operator's visual judgment of the defect candidate images.

[0089] The above-mentioned inspection support method includes a step (step S11) of storing a plurality of defect candidate images detected by a primary inspection of a substrate image, and a step (steps S12-S13) of classifying the plurality of defect candidate images into a plurality of classified image groups (in the above example, classified image groups G11-G15, G21-G25) based on the defect type or criticality of the defect candidate (in the above example, defect candidates D11-D15, D21-D25) included in each defect candidate image by performing rule-based classification on the plurality of defect candidate images, and generating classified display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit (in the above example, display 35). This makes it possible to improve the accuracy of visual judgment of the defect candidate images by an operator, as described above.

[0090] The above-mentioned computer-readable program 811 causes a computer to execute the steps of: storing a plurality of defect candidate images detected by a primary inspection of a substrate image (step S11); and classifying the plurality of defect candidate images into a plurality of classified image groups (in the above example, classified image groups G11-G15, G21-G25) based on the defect type or criticality of the defect candidate (in the above example, defect candidates D11-D15, D21-D25) contained in each defect candidate image by performing rule-based classification on the plurality of defect candidate images, and generating classified display data (steps S12-S13) that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit (in the above example, display 35). This makes it possible to improve the accuracy of visual judgment of the defect candidate images by an operator, as described above.

[0091] The above-described inspection support device 1, inspection support method, and program 811 can be modified in various ways.

[0092] For example, it is not necessary to set a provisional true / false evaluation for each defect candidate image in step S14. In this case, in step S16, an operation is performed in which an operator performs a true / false evaluation for all defect candidate images and inputs the evaluation. In this case, the provisional setting unit 303 may be omitted.

[0093] In the above-mentioned step S16, it is not necessarily required that the operator is allowed to change the defect type.

[0094] Among the above-mentioned multiple classification image groups G11 to G15, G21 to G25, the defect candidate images classified into some classification image groups may not be included in the classification display data. For example, each defect candidate image included in the classification image groups G11, G21 (i.e., a defect candidate image including an open defect or a short defect) may not be included in the classification display data. In this case, each defect candidate image included in the classification image groups G11, G21 is not set as a provisional authenticity evaluation target by the provisional setting unit 303, and is not displayed on the display 35 and is not evaluated for authenticity by the operator. Each defect candidate image included in the classification image groups G11, G21 is directly sent from the image classification unit 302 to the learning data set generation unit 306, and is included in the learning data set as a defect candidate image including a true defect with a high criticality.

[0095] The above-mentioned trained model does not necessarily need to be created in the inspection support device 1. For example, the inspection support device 1 may transmit the classification display data generated by the image classification unit 302 to a verifying device that is a device different from the inspection support device 1. In the verifying device, a plurality of defect candidate images included in the classification display data are sequentially displayed on a display unit such as a display, and an operator visually checks the defect candidate images on the display unit to evaluate whether they are true defects or false alarms.

[0096] In this case, the operator can continuously observe a plurality of defect candidate images having the same defect type or criticality together on the display unit of the verifying device. As a result, the accuracy of the operator's visual judgment of the defect candidate images can be improved in a manner similar to that described above. When the classification display data is transmitted to the verifying device, the inspection support device 1 is required to include at least the above-mentioned storage unit 301 and image classification unit 302, and other components (e.g., the display unit, the input receiving unit 305, the learning data set generating unit 306, the learning unit 307, etc.) may be omitted.

[0097] In the primary inspection by the primary inspection device 7, in addition to or instead of the above-mentioned rule-based inspection, inspection by another method (for example, AI inspection) may be performed.

[0098] The above-mentioned substrate 9 is not necessarily limited to a printed wiring board, and may be, for example, a semiconductor substrate, a substrate for a semiconductor package, a glass substrate for a flat panel display device such as a liquid crystal display device or a plasma display device, a glass substrate for a photomask, a substrate for a solar cell panel, or the like.

[0099] The configurations in the above-described embodiment and each of the modified examples may be combined as appropriate as long as they are not mutually inconsistent. [Explanation of symbols]

[0100] 1. Inspection support equipment 9 Substrate 35 Display 61 Defect candidate images 67 1st image 68 2nd image 69 3rd image 301 Storage section 302 Image Classification Unit 303 Provisional Settings 305 Input Reception Section 306 Learning Dataset Generation Unit 307 Learning Department 611~613 Pattern elements 614 Background area 811 Program D11~D15, D21~D25 Defect candidates G11~G15,G21~G25 Classified image group S11~S18 steps

Claims

1. An inspection support apparatus for supporting inspection of a substrate having a pattern on a surface thereof, comprising: a storage unit that stores a plurality of defect candidate images detected by a primary inspection of a substrate image; an image sorting unit that performs rule-based classification on the plurality of defect candidate images to sort the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate included in each defect candidate image, and generates classified display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit; An inspection support device comprising:

2. The inspection support device according to claim 1 , a display unit that displays the plurality of defect candidate images for each of the classified image groups; an input receiving unit that receives an input of an evaluation of a defect or false information by an operator who visually checks each of the defect candidate images displayed on the display unit as a true / false evaluation of each of the defect candidate images; a learning dataset generation unit that generates a learning dataset by associating the authenticity evaluation with each of the defect candidate images in the classification display data; A learning unit that performs machine learning on an initial model using the learning dataset to create a trained model used for AI inspection of substrate images; The inspection support device further comprises:

3. The inspection support device according to claim 2, a provisional setting unit that sets a provisional authenticity evaluation for each defect candidate image included in each of the classified image groups based on a defect type or a criticality corresponding to each of the classified image groups; The learning dataset generation unit is an inspection support device that, when an input different from the provisional authenticity evaluation is received by the input receiving unit, matches the authenticity evaluation received by the input receiving unit to a defect candidate image, and in other cases matches the provisional authenticity evaluation to a defect candidate image.

4. 4. The inspection support device according to claim 1, the image classification unit classifies each of the defect candidate images based on a criticality of the defect candidate included in each of the defect candidate images; An inspection support device in which the criticality of the defect candidate is determined based on the size of the defect candidate included in each of the defect candidate images and the width of a pattern element or a background area at the position of the defect candidate.

5. 4. The inspection support device according to claim 1, the image classification unit classifies each of the defect candidate images based on a criticality of the defect candidate included in each of the defect candidate images; An inspection support device in which the criticality of the defect candidate is determined based on a density of the defect candidate included in each of the defect candidate images.

6. 4. The inspection support device according to claim 1, the image classification unit classifies each of the defect candidate images based on a defect type of the defect candidate included in each of the defect candidate images; The image classification unit converts each of the defect candidate images into a first image expressed in a first color which is one of RGB, generates a second image in which a reference image that does not have a defect candidate included in each of the defect candidate images is expressed in a second color which is different from the first color among RGB, and in a third image obtained by overlaying and synthesizing the first image and the second image, defect types in the area displayed in the first color are designated as bright defect candidates and defect types in the area displayed in the second color are designated as dark defect candidates.

7. 4. The inspection support device according to claim 1, the image classification unit classifies each of the defect candidate images based on both a defect type and a criticality of the defect candidate included in each of the defect candidate images; In the classification display data, two or more defect candidate images corresponding to one defect type can be displayed based on the magnitude relationship of their fatalities.

8. 1. A method for assisting inspection of a substrate having a pattern on a surface thereof, comprising: a) storing a plurality of defect candidate images detected by a first inspection of a substrate image; b) performing rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate included in each defect candidate image, and generating classification display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit; The inspection support method includes:

9. 1. A computer readable program for causing a computer to execute a program to assist in the inspection of a substrate having a pattern on a surface thereof, the program comprising: a) storing a plurality of defect candidate images detected by a first inspection of a substrate image; b) performing rule-based classification on the plurality of defect candidate images to classify the plurality of defect candidate images into a plurality of classified image groups based on the defect type or criticality of the defect candidate included in each defect candidate image, and generating classification display data that enables the plurality of defect candidate images to be displayed for each classified image group on a display unit; A computer-readable program that causes a computer to execute the above.

Images