Image determination device and image determination method

Abstract

The present invention discloses an image judgment device and an image judgment method. The image judgment device stores a judgment model in a memory, the judgment model being configured to learn each of the plurality of judgment items based on first data and second data, the first data corresponding to an image as a reference for good judgment, and the second data corresponding to an image as a reference for poor judgment; set a plurality of magnifications for each of the plurality of judgment items; segment a portion of an image, and generate image data from the segmented image at the plurality of magnifications; input the generated image data corresponding to each of the plurality of magnifications into the judgment model, and judge whether the quality of the image is good or poor relative to each of the plurality of judgment items based on an output result from the judgment model.

Description

Technical Field

[0001] The present invention relates to an image determination device and an image determination method, and particularly to an image determination device and an image determination method for inspecting the manufacturing quality of, for example, a printed substrate. Background Art

[0002] Conventionally, it is known to use a computer incorporating an AI function to determine whether an image has a defect. For example, Japanese Patent Laid-Open No. 2016-040650 discloses a device that acquires image data, analyzes the image data, and determines whether there are defects in the surface of the semiconductor substrate by using an image capture device to capture an image of the surface of a semiconductor substrate. When performing image analysis, the device acquires an image indicating a defective portion and automatically classifies the image. In this automatic classification, the device uses knowledge engineering techniques such as learning-based classification, rule-based classification, or ensemble learning. More specifically, according to Japanese Patent Laid-Open No. 2016-040650, a plurality of classifiers incorporated into a host computer analyze a feature vector obtained from an image and classify the image based on the number of weighted votes.

[0003] The above-mentioned related art has a problem of insufficient judgment accuracy. For example, when a skilled expert in substrate inspection verifies the judgment result obtained by the device, a substrate that is still in good condition is judged to be defective. In particular, when there are many judgment target items, the number of false judgments increases.

[0004] First, semiconductor substrate inspection costs a lot of human resources and time. Inspection accuracy depends largely on the skills of experts. Therefore, this inspection costs a lot of cost (time and expenses), and there has long been a demand for reducing costs through automation. Recently, attempts have been made to automate substrate inspection by using AI technology, which has been significantly developed. However, as mentioned above, sufficient judgment accuracy has not been obtained so far. Summary of the invention

[0005] Therefore, the present invention is conceived as a response to the above-mentioned disadvantages of the conventional technology.

[0006] For example, the image determination device and the image determination method according to the present invention can accurately perform inspection determination on, for example, a printed substrate and a semiconductor substrate.

[0007] According to one aspect of the present invention, there is provided an image judgment device for judging the quality of an image by using a plurality of judgment items, the device comprising: a judgment model, the judgment model being configured to learn each of the plurality of judgment items based on first data and second data, the first data corresponding to an image as a basis for judging good, and the second data corresponding to an image as a basis for judging poor; a setting unit, the setting unit being configured to set a plurality of magnifications for each of the plurality of judgment items; a generation unit, the generation unit being configured to segment a portion of an image and generate image data from the segmented image at the plurality of magnifications set by the setting unit; and a judgment unit, the judgment unit being configured to input the image data corresponding to each of the plurality of magnifications generated by the generation unit into the judgment model, and judge whether the quality of the image is good or poor relative to each of the plurality of judgment items based on an output result from the judgment model.

[0008] According to another aspect of the present invention, there is provided an image judgment method in an image judgment device for judging the quality of an image by using multiple judgment items, the method comprising: storing a judgment model in a memory, the judgment model being configured to learn each of the multiple judgment items based on first data and second data, the first data corresponding to an image as a basis for judging good, and the second data corresponding to an image as a basis for judging poor; setting multiple magnifications for each of the multiple judgment items; segmenting a portion of an image and generating image data from the segmented image at the multiple magnifications; and inputting the generated image data corresponding to each of the multiple magnifications into the judgment model, and judging whether the quality of the image is good or poor relative to each of the multiple judgment items based on an output result from the judgment model.

[0009] The present invention is particularly advantageous because the presence / absence of a defect in a surface of, for example, a printed substrate or a semiconductor substrate can be judged more accurately based on image data acquired by capturing an image of the surface.

[0010] Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram showing an outline of the arrangement of an image determination system according to an exemplary embodiment of the present invention;

[0012] Figure 2A and 2B is a block diagram showing the functional arrangement of an image acquisition device and an image determination device;

[0013] Figure 3A , 3B and 3C is a diagram for explaining segmentation from image data;

[0014] Figure 4 is a diagram showing a relationship between a segmented image corresponding to a type of defect found in a surface of a printed substrate and a partial image segmented from the image;

[0015] Figure 5 is a flowchart showing a process of judging the presence / absence of a defect in the surface of a printed substrate performed by an image judging device;

[0016] Figure 6 is a diagram showing an example of the relationship between a judgment item (defect type) and two types of enlarged images used for judgment;

[0017] Figure 7 is a diagram for explaining a combination of segmentation sizes regarding a defective item;

[0018] Figure 8 is shown for verification Figure 1 A block diagram of the arrangement of the system for determining the accuracy of the system shown; and

[0019] Fig. 9 The flowchart shows the inspection process and the processing of the inspection result in the inspection step of learning the printed circuit board. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the present invention will now be described in detail according to the accompanying drawings. It should be noted that the following embodiments are not intended to limit the scope of the appended claims. Multiple features are described in the embodiments. Not all multiple features are necessarily essential to the present invention, and multiple features can be combined arbitrarily. In addition, throughout the accompanying drawings, the same reference symbols represent the same or similar parts, and repeated descriptions will be omitted.

[0021] Note that in this specification, "substrate" includes a semiconductor substrate and a printed substrate manufactured by a semiconductor manufacturing process, and a head substrate provided with printing elements, wirings, etc. manufactured by using a manufacturing technology and used as a head chip of an inkjet print head.

[0022] In addition, the term "inspection" mentioned in this specification means to inspect the surface state of the transfer member used in the transfer type inkjet printing device in addition to the semiconductor substrate, the printed substrate, and the head substrate as described above. This inspection inspects the presence / absence of scratches, the presence / absence of adhesion of foreign matter, the presence / absence of soldering unevenness, the presence / absence of soldering stripping, the presence / absence of plating stripping, etc. on the surface of the "substrate" or the transfer member. In addition, the above term "inspection" means to inspect the quality of the inkjet print head by inspecting the image itself printed on the print medium by the inkjet print head.

[0023] Described here is a specific example of verifying image data obtained by capturing an image of the surface of a substrate (such as a printed substrate or a glass substrate) using an image capture device, and detecting defects (such as the presence / absence of scratches caused on the substrate or adhesion of foreign matter to the substrate) based on the image data.

[0024] Figure 1 : is a block diagram showing the arrangement of an image determination system (hereinafter, will be referred to as a system) as an exemplary embodiment of the present invention.

[0025] like Figure 1 As shown, the system 1 includes an image capturing device 30 that generates image data by capturing an image of the surface of a printed substrate as an inspection target, an image acquiring device 10 that acquires the image data, and an image judging device 20 that acquires the image data from the image acquiring device 10 and judges the image. The image acquiring device 10 and the image judging device 20 are computers, each including a CPU and a storage device (memory) that stores image data and judgment results. The image judging device 20 can, for example, issue instructions to execute processing described later, and can use a keyboard, a mouse, and a computer to perform the processing described later. Wait for settings.

[0026] Figure 2A and 2B is a block diagram showing the functional arrangement of the image acquisition device and the image determination device. Figure 2A is a block diagram showing the functional arrangement of the image acquisition device 10 . Figure 2B : is a block diagram showing the functional arrangement of the image determination device 20 .

[0027] like Figure 2A As shown, the image acquisition device 10 stores the image data received from the image capture device 30 in the image holding unit 11 constituted by an SSD memory or the like.

[0028] The defect detection unit 12 detects the presence / absence of defects in the image data stored in the image holding unit 11. More specifically, the defect detection unit 12 holds image data acquired by capturing an image of a printed substrate having an excellent surface state as reference image data (IREF), and compares the reference image data with image data received from the image capture device 30. This reference image data is data of a defect-free image or data of an image having few defects. The defect detection unit 12 extracts corresponding pixels of two images having significant differences in brightness values ​​or density values, and estimates what type of defect the received image has based on the distribution of the extracted pixels and the degree of difference in brightness values ​​or density values. These processes can be implemented by using known pattern recognition techniques.

[0029] Assume that an image is acquired by capturing an image of a pattern wiring portion of a printed substrate. In this case, when the surface state of the portion is good, the portion is identified as a region having uniform density (brightness) and having no variation in density (brightness). In contrast, if the pattern wiring has plating peeling, the region having uniform density (brightness) has a place where the density (brightness) is discontinuously different. Since such a place is locally unique, the place where the defect occurs and the type of defect can be estimated by pattern recognition.

[0030] It is assumed that the above defect detection processing has estimated the presence of defects. In this case, the defect detection unit 12 generates a defect code (FCODE) indicating the type of defect and defect position information (FPOS). Images estimated to have defects are classified into the following three types. The first type is an NG image in terms of quality standards. This image is classified as an NG image (true_NG image) through AI learning. The second type is an image that is classified as an OK image in terms of quality standards although defects and dust are recognized by human vision, but is classified as an NG image (NG image) through AI learning. The third type is an image that is classified as an OK image in terms of quality standards, does not have any defects and dust recognized by human vision, and is also classified as an OK image (OK image) through AI learning.

[0031] The segmentation unit 13 performs segmentation processing on the image data estimated to have defects. At this time, the image acquisition device 10 transmits the image data (IMG) segmented by the segmentation unit 13 to the image determination device 20 together with the defect code (FCODE) and the defect position information (FPOS).

[0032] like Figure 2BAs shown, the image judgment device 20 receives the image data (IMG) segmented by the segmentation unit 13. The image storage unit 210 includes a first storage unit 211 storing reference image data for judging that the inspection result of the printed substrate is good (OK), a second storage unit 212 storing reference image data for judging as bad (NG), and a third storage unit 213 storing image data of the judgment target. The image storage unit 210 is composed of SRAM, DRAM, SSD, etc. The image processing unit 220 enlarges the image of the judgment target based on the input defect code (FCODE). The judgment unit 230 includes an OK / NG judgment model (neural network) 231 (hereinafter referred to as a model) that outputs information indicating that the inspection result of the segmented image data is OK or NG. The model 231 has a plurality of weighting coefficients.

[0033] Note that the first storage unit 211 stores a plurality of OK judgment reference data for each type of defect, and the second storage unit 212 stores a plurality of NG judgment reference data for each type of defect. These data are used for judgment performed by the judgment unit 230. The types of defects in the printed substrate will be described later.

[0034] The image judgment device 20 includes a learning unit 240 having an AI function that learns which part of the image it focuses on at the time of segmentation and how to perform image analysis each time the presence / absence of a defect in the printed substrate is judged by using image data input from the image acquisition device 10. The system 1 sets the printed substrate as an inspection target and sets various events occurring in the surface of the printed substrate as judgment items of image judgment, and performs image judgment according to the judgment items.

[0035] Note that the learning unit 240 inputs various reference images for each judgment item when learning the model and changes the weighting coefficient based on the judgment result. Repeating this learning process will optimize the weighting coefficient of the model 231. The judgment unit 230 inputs the image data of the judgment target to the learned model 231 when judging the presence / absence of a defect in the printed substrate, and the model 231 outputs a result indicating the presence / absence of a defect in the image.

[0036] Reference Figures 3A to 3C Describes segmentation from image data.

[0037] Figure 3A FIG. 2 shows the relationship between the image of the printed substrate acquired by the image acquisition device 10 from the image capture device 30 and the image segmented by the segmentation unit 13. Figure 3AAs shown, the defect detection unit 12 generates defect position information (FPOS) by specifying a defect position 302 from an original image 301 acquired from the image capture device 30. Then, the segmentation unit 13 segments an image 303 of 256×256 pixels centered at the defect position 302.

[0038] Figure 3B It is shown that by Figure 3A The segmented image 303 shown is a diagram of an image 304 obtained by segmenting the image 303 into 1 / 16 {(1 / 4) × (1 / 4)} regions. Figure 3A and 3B As is clear from the comparison between the two, the image 304 is acquired by dividing the 64 (vertical) × 64 (horizontal) pixel image from the 256 (vertical) × 256 (horizontal) pixel image 303 in the vertical and horizontal directions with the defect position 302 as the center. The image acquisition device 10 transmits the image data (IMG) representing the divided image 303 to the image determination device 20. The image processing unit 220 generates image data representing the image 304.

[0039] The image processing unit 220 generates image data representing the image 304 based on the value of the defect code (FCODE) and the value of the defect position information (FPOS). For example, if the defect code (FCODE) indicates another type of defect, the image processing unit 220 generates 32 pixels×32 pixels image data representing a portion of the image 303.

[0040] Figure 3C 2 shows an image (reference image) used as a reference for image determination stored in the image storage unit 210 of the image determination device 20. Figure 3C As shown, the reference image 305 has the same size (256 pixels × 256 pixels) as the size of the image as the judgment target (comparison target). This reference image includes an image for normal (OK) determination and an image for NG determination (i.e., for determining the presence of a defect) for each defect type. As described above, these images are stored in the first storage unit 211 and the second storage unit 212, respectively. These images are used when the above-mentioned learning unit 240 causes the model 231 to learn.

[0041] Figure 4 The relationship between the segmented images corresponding to the types of defects found in the surface of the printed substrate and the partial images segmented from the images is shown. Figure 4 The types of defects are shown in the left vertical direction, and the image sizes of the respective images are shown in the upper horizontal direction.

[0042] Note that images with the same size are displayed in Figure 4. Taking the image of 256 pixels × 256 pixels arranged in the leftmost column as a reference, a 128 × 128 image, a 64 × 64 image, a 32 × 32 image, and an 8 × 8 image are displayed side by side in the right direction. The image displayed on the right side of the reference image is enlarged to have the same size as the reference image when displayed on the display or printed. Therefore, when an image having a size of 128 pixels × 128 pixels is generated from an image having a size of 256 pixels × 256 pixels, this operation will be referred to as 2× magnification image generation. Similarly, the generation of a 64 × 64 image, the generation of a 32 × 32 image, and the generation of an 8 × 8 image will be referred to as 4× magnification image generation, 8× magnification image generation, and 32× magnification image generation, respectively.

[0043] Figure 4 A reference image and four enlarged images segmented from the reference image are shown with respect to each of five defect types including foreign matter in plating, SR (solder resist) pattern shape, SR peeling, SR (solder resist) unevenness, and SR unevenness in a fine line pattern.

[0044] Note that the defect types are not limited to the above examples. For example, in addition to them, the defect types include copper plating color, wire shape, pad shape, and the position of a hole such as a through hole.

[0045] Figure 5 : is a flowchart showing a process of judging the presence / absence of a defect in the surface of a printed substrate, which is performed by the image judging device.

[0046] First, in step S401, the image judgment device selects an image (image 303) of a judgment target including a judgment item specified by a defect code (FCODE). In step S402, the image judgment device divides an image magnified at a predetermined magnification from the image of the judgment target. In this embodiment, the image judgment device divides the image at two types of magnifications specified by the judgment item. For example, when judging the presence / absence of "SR unevenness" as a judgment item (defect type), the image judgment device divides a 64×64 pixel image (image 404 in FIG. 3) from a 256×256 pixel image (image 303 in FIG. 3) and enlarges the divided image to a 256×256 pixel image. Similarly, the image judgment device divides a 128×128 pixel image and enlarges it to a 256×256 pixel image. The image judgment device generates two types of enlarged images in this way.

[0047] This judgment item corresponds to the above defect type. In this embodiment, two types of magnifications are defined according to the judgment item (defect type).

[0048] Figure 6An example of the relationship between the judgment item (defect type) and two types of enlarged images used for judgment is shown.

[0049] refer to Figure 6 If the judgment item (defect type) is foreign matter in the plating layer, the image judgment device generates images of two sizes (i.e., 64×64 pixels and 8×8 pixels) from the 256×256 pixel image. If the judgment item (defect type) is an SR pattern, the image judgment device generates images of two sizes (i.e., 128×128 pixels and 32×32 pixels) from the 256×256 pixel image.

[0050] Assume that the judgment item is a foreign body in the plating layer. Such a defect often occurs in a part of the wiring pattern on the surface of the printed substrate. It is known that this foreign body is identified because there is a concentration (brightness) that is different from the concentration (brightness) around the foreign body locally. In contrast, the wiring pattern is detected as an area with a uniform concentration (brightness) on the image. Therefore, when a relatively wide area with a uniform concentration (brightness) on the image includes a small area with a concentration (brightness) that is discontinuously different from the concentration (brightness) of the wide area, the small area may be a foreign body in the plating layer.

[0051] On the other hand, a through hole in a printed substrate is identified as a small area on the image having a density (brightness) that is discontinuously different from that of the surrounding area. However, a through hole is generally circular, so it can be identified as a through hole by specifying the shape of a small area having a density (brightness) that is discontinuously different from that of the surrounding area.

[0052] In contrast, since foreign matter in the plating layer has an indefinite shape, the probability of being able to specify the presence of foreign matter in the plating layer is increased by specifying the shape of the foreign matter and specifying a small area having a concentration (brightness) discontinuously different from the surrounding area.

[0053] Combining a plurality of factors as described above can accurately identify the presence of a target as a judgment item.

[0054] It is not always easy to specify the existence of each of multiple factors by using a 256×256 pixel image that is first segmented. This is because this image includes various features of the printed substrate, and multiple image processing algorithms are required to find the existence of a factor from these features, while the processing time becomes long. In contrast, using an image obtained by segmenting and enlarging a small area including a judgment item will simplify changes in density (brightness) or shape and facilitate specifying the judgment item. Considering the design of the printed substrate itself, each judgment item and its surrounding area have certain tendencies in their characteristics and where the factors occur. For this reason, depending on the judgment item, the size (i.e., magnification) of the image area that can most easily capture the characteristics of the item varies.

[0055] Therefore, the present embodiment is configured to select images obtained at two different magnifications that enable the image judgment device to most easily capture features for each judgment item, and use the images for image judgment.

[0056] Here, a method of selecting the two best images will be described.

[0057] This embodiment is configured to perform image judgment using six combinations of images for each defect type. Figure 4 As shown, images with sizes of 128×128 pixels, 64×64 pixels, 32×32 pixels, and 8×8 pixels are generated from an image with a size of 256 pixels×256 pixels. The correct answer rates associated with the six combinations of images are evaluated to ultimately select the two best images. The six combinations of images are as follows:

[0058] (1) 32×32 pixel images and 8×8 pixel images;

[0059] (2) 64×64 pixel images and 8×8 pixel images;

[0060] (3) 128×128 pixel images and 8×8 pixel images;

[0061] (4) 64×64 pixel images and 32×32 pixel images;

[0062] (5) 128×128 pixel images and 32×32 pixel images; and

[0063] (6) 128×128 pixel images and 64×64 pixel images.

[0064] Then, the image judgment device 20 performs image judgment using the above six combinations of images with respect to each of a plurality of (predetermined number) of test images. The person (expert) visually confirms the image of the judgment result. The person (expert) verifies the correct answer rate of the image judgment by the judgment confirmation using this visual confirmation.

[0065] Figure 7 It is a diagram for explaining the combination of the partition sizes regarding one defect item.

[0066] Figure 7 The judgment results of, for example, two images (i.e., a 64×64 pixel image and a 128×128 pixel image) and the judgment results obtained when the two images are combined (overall results) are shown. The judgment results include "NG correct answer rate", "OK correct answer rate" and "true_NG correct answer rate". Each correct answer rate is expressed as a percentage (%).

[0067] In this case, the "NG correct answer rate" indicates the ratio of the number of NG judgments made by humans (experts) that are also confirmed by the image judgment device 20 to the number of samples of test images verified with respect to the corresponding defect items, which is expressed as a percentage (%). The "OK correct answer rate" indicates the ratio of the number of OK judgments made by humans (experts) that are also confirmed by the image judgment device 20 to the number of samples of test images verified with respect to the corresponding defect items, which is expressed as a percentage (%). The "True_NG correct answer rate" indicates the ratio of the number of NG judgments made by both the image judgment device 20 and humans (experts) to the number of samples of test images verified with respect to the corresponding defect items, which is expressed as a percentage (%).

[0068] As from Figure 7 It is clearly seen that the correct answer rate obtained by making judgment using a combination of two images with different magnifications is higher than the correct answer rate obtained by making judgment using one image with a certain magnification.

[0069] This embodiment is configured to perform the following with respect to all six combinations described above: Figure 7 The judgment evaluation shown. First, a combination of segmentation sizes that presents a "true_NG correct answer rate" of 100% (or the highest value) in the overall judgment is selected from the six combinations. A combination of segmentation sizes that presents a high "NG correct answer rate" in the overall judgment is selected from the selected combinations. A combination of segmentation sizes that presents a high "true_NG correct answer rate" for each segmentation size is selected from the selected combinations. Performing the selection in three steps in this way will select the two best images.

[0070] As a result of the above processing, images with the following size are used Figure 4 Judgment of the five defective items shown:

[0071] Foreign matter in the coating → 32×32 pixels and 128×128 pixels;

[0072] SR pattern shape → 32×32 pixels and 128×128 pixels;

[0073] SR peeling → 64×64 pixels and 128×128 pixels;

[0074] SR non-uniform → 32×32 pixels and 64×64 pixels; and

[0075] SR unevenness in fine line pattern → 32×32 pixels and 128×128 pixels.

[0076] Note that the above processing involves verification by a person (expert) through visual confirmation and learning by the learning unit 240 of the image determination device 20, so the learning will be described below.

[0077] Figure 8 Is to show verification Figure 1 A block diagram of the arrangement of a system for determining accuracy of the system shown. Figure 8 The same reference symbols in the Figure 1 The same constituent elements described above are the same as those described above, and their description will be omitted.

[0078] refer to Figure 8 , a judgment confirmation device 40 is provided at a later stage of the image judgment device 20 to receive the judgment result from the image judgment device 20. Then, a person (expert) visually confirms the image of the judgment result obtained by the judgment confirmation device 40. Using this visual confirmation, each target image is classified. The above selection and evaluation are reflected in the learning unit 240 so that it learns them. The learning unit 240 defines the magnification according to which combination of magnifications presents the highest correct answer rate. Therefore, when the judgment item (defect type) is specified, the learning unit 240 automatically outputs two magnifications as the learning result.

[0079] As described above, a combination of changing the division size for each judgment item can improve the correct answer rate.

[0080] Return to reference Figure 5 4. In the flowchart of FIG. 4, in step S403, the two types of enlarged images are input to the learned model.

[0081] In step S404, if it is judged that both images of the two sizes are not defective images (OK) with respect to the judgment item with respect to the output result from the learning model, the process proceeds to step S405 to judge that "the image is not a defective image" with respect to the judgment item. In contrast, if it is judged that at least one of the images of the two sizes has a defect (NG) with respect to the judgment item, the process proceeds to step S406 to judge that "the image is a defective image" with respect to the judgment item. In step S407, the judgment target image is assigned a label indicating an NG image and is stored in the third storage unit 213. Such a judgment will be referred to as a so-called overall judgment.

[0082] The above processing is performed for each of the plurality of judgment items to obtain any one of the judgment results, that is, for each judgment item, "the image is not a defective image (OK)" or "the image is a defective image (NG)". In this embodiment, the printed substrate is inspected, and thus the inspection result (good or bad) of the printed substrate is judged based on the judgment result of each of the plurality of judgment items. For example, if at least one of the plurality of judgment items is judged to be bad, the printed substrate is judged to be bad.

[0083] Fig. 9 The flowchart shows the inspection process and the processing of the inspection result in the inspection step of learning the printed circuit board.

[0084] In step S501, the printed substrate is inspected to check the presence / absence of an image for judging a defect. More specifically, as described above, it is judged whether an image obtained by capturing an image of the surface of the printed substrate using the defect detection unit 12 is an image for which a defect should be judged. If it is judged that the obtained image is an image for which a defect should be judged, then the judgment item (defect type) is judged. Such inspection is performed on a predetermined number of printed substrates.

[0085] Subsequently, in step S502, the image judgment device 20 performs judgment processing on the image obtained by capturing the image of the printed substrate judged to have a defect. The judgment confirmation device 40 verifies the judgment result to confirm whether the judgment is OK (correct answer) or NG (wrong answer) when viewed by a person (expert). This verifies what judgment process has led the image judgment to OK (correct answer). Therefore, the learning unit 240 is made to learn this judgment process.

[0086] This learning result revealed that the combination of judgment segmentation sizes for each judgment item improved the correct answer rate. Therefore, after learning, images with the best magnification reflecting the learning result according to the judgment item (defect type) were used for comparison.

[0087] Therefore, according to the above embodiment, the judgment accuracy can be improved, and the number of times that an image judged by the human eye as not having a defect is judged by the image judgment device as having a defect (error information) can be reduced. This can reduce the number of images to be visually recognized and greatly reduce the recognition work.

[0088] According to the above-described embodiment, the presence / absence of a defect is determined by comparing two images having different magnifications for each defect item (defect type) with an image based on OK judgment reference data and an image based on NG judgment reference data. However, the present invention is not limited thereto. For example, three or more images having different magnifications may be compared with a reference image.

[0089] In this case, when an odd number (three or more) of magnified images are used, the majority rule can be used to finally judge whether a defect exists (NG) or does not exist (OK) for each judgment item. Assume that an even number (four or more) of magnified images are used. In this case, when the majority rule is used, the number of times that a defect exists (NG) is judged to be equal to the number of times that a defect does not exist (OK) is judged to be equal, resulting in failure to make a final judgment. Therefore, if the above numbers are equal to each other, it should be finally judged that a defect exists (NG).

[0090] It is also possible to obtain a judgment result by weighting the judgment results obtained from each of two or more magnified images having different magnifications. Assume that two images are used, and the results obtained from one image and the other image are weighted by 0.7 and 0.3, respectively. In this case, when the judgment result obtained from the image with a weight of 0.7 is OK, even if the judgment result obtained from the image with a weight of 0.3 is NG, the final judgment is OK.

[0091] although Figure 1 The system shown includes the image acquisition device 10 and the image determination device 20 as separate arrangements, but the system may use a device having a single-body arrangement obtained by integrating the two devices.

[0092] The size of the above-mentioned reference image 305, that is, the size of the image as the determination target (comparison target) is not limited to 256 pixels × 256 pixels, and may be another size. For example, the size may be 224 pixels × 224 pixels.

[0093] refer to Figure 2AThe described defect detection unit 12 may hold CAD data as reference image data, and compare the CAD data with the image data received from the image capture device 30. Although the embodiment configured to judge an image estimated to have defects with respect to a plurality of judgment items has been described by exemplifying printing inspection, the image may be judged with respect to one judgment item depending on the content of the inspection.

[0094] In the implementation of the reference Figure 7 When the image selection is described, this operation is not limited to the above-mentioned 3-step selection. For example, when priority is given to the execution of the selection within a short period of time, 2-step selection or 1-step selection can be performed. Assume that 2-step selection is performed. In this case, after the selection of the combination of segmentation sizes that presents a "true_NG correct answer rate" of 100% (or the highest value) in the overall judgment, the combination of segmentation sizes that presents a high "NG correct answer rate" in the overall judgment is selected from the selected combinations. Assume that 1-step selection is performed. In this case, for example, a combination of segmentation sizes that presents a "true_NG correct answer rate" of 100% (or the highest value) in the overall judgment is selected. Alternatively, a combination of segmentation sizes that presents a high "true_NG correct answer rate" can be selected for each segmentation size. In addition, any one of 3-step selection, 2-step selection and 1-step selection can be selected for each judgment item according to the judgment accuracy, the time required for judgment, etc.

[0095] In addition, the above embodiment has exemplified a semiconductor substrate or a printed substrate as a “substrate.” However, this embodiment can be applied to a flexible printed circuit (FPC) having wiring formed on a single surface or double surfaces (where the surface layer is covered with a protective film).

[0096] <Other embodiments>

[0097] Embodiments of the present invention may also be implemented by reading out and executing computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a 'non-transitory computer-readable storage medium') to perform one or more of the functions of the above-mentioned embodiments and / or a computer of a system or device including one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more of the functions of the above-mentioned embodiments, and by a computer of the system or device, for example, reading out and executing computer executable instructions from a storage medium to perform one or more of the functions of the above-mentioned embodiments and / or controlling one or more circuits to perform one or more of the functions of the above-mentioned embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read out and execute computer executable instructions. Computer executable instructions may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage device of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray disk (BD)) TM ), one or more of flash memory devices, memory cards, etc.

[0098] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0099] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image judging device for judging the quality of an image by using a plurality of judging items, the device comprising: a judgment model configured to learn each of the plurality of judgment items based on first data corresponding to an image as a reference for good judgment and second data corresponding to an image as a reference for bad judgment; a setting unit configured to set a combination of a plurality of magnifications for each of the plurality of judgment items; a generating unit configured to segment a part of an image and generate image data from the segmented image at the plurality of magnifications in the combination corresponding to the determination item set by the setting unit; as well as a judgment unit configured to input the image data corresponding to each of the plurality of magnifications in the combination corresponding to the judgment item, which is generated by the generation unit, into the judgment model, and judge whether the quality of the image is good or bad with respect to each of the plurality of judgment items based on an output result from the judgment model, wherein the plurality of judgment items include a first judgment item and a second judgment item, and A combination of a plurality of magnifications set for the first judgment item is different from a combination of a plurality of magnifications set for the second judgment item. 2 . The apparatus according to claim 1 , wherein the plurality of magnifications set by the setting unit are defined based on a verification result obtained by verifying a determination result obtained by the determination unit with respect to a plurality of images. 3 . The apparatus according to claim 2 , further comprising a learning unit configured to learn the verification result and reflect the verification result in the judgment of the judgment unit.

4. An apparatus according to claim 1, wherein if the multiple magnifications are a first magnification and a second magnification as two different magnifications, and an output result obtained by the judgment model with respect to image data having the first magnification and an output result obtained by the judgment model with respect to image data having the second magnification are both good, then the judgment unit judges the quality of the image as good, and if any one of the output result obtained by the judgment model with respect to image data having the first magnification and the output result obtained by the judgment model with respect to image data having the second magnification is poor, then the judgment unit judges the quality of the image as poor.

5. The apparatus according to claim 1, wherein the image is acquired by causing an image capturing device to capture an image of the inspection target.

6. An apparatus according to claim 5, wherein image data corresponding to an image acquired by the image capture device and image data corresponding to a reference image corresponding to the image are compared in advance to specify, from the comparison, a position of an inspection target in the image as a judgment target and a judgment item at the position. The apparatus according to claim 6 , wherein the segmented image is acquired from the image based on a designated position. 8 . The apparatus according to claim 6 , wherein the setting unit sets the plurality of magnifications according to a designated judgment item.

9. The apparatus according to claim 5, wherein the inspection target includes a printed substrate, a glass substrate, a semiconductor substrate, and a head substrate provided with printing elements and wirings of a head chip serving as an inkjet print head.

10. The apparatus according to claim 9, wherein the plurality of judgment items include the presence / absence of foreign matter in a plating layer on a printed substrate, the shape of an SR pattern, SR peeling, SR unevenness, the position of a through hole, a fine line pattern, plating peeling, the shape of a pad, and a plating color.

11. An image judging method in an image judging device for judging the quality of an image by using a plurality of judging items, the method comprising: storing a judgment model in a memory, the judgment model being configured to learn each of the plurality of judgment items based on first data corresponding to an image as a reference for good judgment and second data corresponding to an image as a reference for bad judgment; Setting a plurality of magnification combinations for each of the plurality of judgment items; dividing a portion of an image and generating image data from the divided image at the plurality of magnifications in the combination corresponding to the determination item; as well as inputting the generated image data corresponding to each of the plurality of magnifications in the combination corresponding to the judgment item into the judgment model, and judging whether the quality of the image is good or bad with respect to each of the plurality of judgment items based on an output result from the judgment model, wherein the plurality of judgment items include a first judgment item and a second judgment item, and A combination of a plurality of magnifications set for the first judgment item is different from a combination of a plurality of magnifications set for the second judgment item. 12 . The method according to claim 11 , wherein the plurality of set magnifications are defined based on a verification result obtained by verifying the judgment results with respect to the plurality of images. 13 . The method according to claim 12 , further comprising learning the verification result and reflecting the verification result in the judgment.

14. A method according to claim 11, wherein in the judgment, if the multiple magnifications are a first magnification and a second magnification as two different magnifications, and an output result obtained by the judgment model with respect to image data having the first magnification and an output result obtained by the judgment model with respect to image data having the second magnification are both good, then the quality of the image is judged to be good, and if any one of the output result obtained by the judgment model with respect to image data having the first magnification and the output result obtained by the judgment model with respect to image data having the second magnification is poor, then the quality of the image is judged to be poor.

15. The method according to claim 11, wherein the image is acquired by causing an image capture device to capture an image of the inspection target.

16. A method according to claim 15, wherein image data corresponding to the image acquired by the image capture device and image data corresponding to a reference image corresponding to the image are compared in advance to specify a position of an inspection target in the image as a judgment target and a judgment item at the position from the comparison. The method of claim 16 , wherein the segmented image is acquired from the image based on a specified location.

18. The method according to claim 16, wherein in the setting, the plurality of magnifications are set according to a designated judgment item.

19. The method according to claim 15, wherein the inspection target includes a printed substrate, a glass substrate, a semiconductor substrate, and a head substrate provided with printing elements and wirings of a head chip used as an inkjet print head.

20. The method of claim 19, wherein the plurality of judgment items include presence / absence of foreign matter in a plating layer on a printed substrate, shape of an SR pattern, SR peeling, SR unevenness, location of a through hole, a fine line pattern, plating peeling, shape of a pad, and plating color.

Images