Computer system, appearance inspection method, and appearance inspection program

The computer system optimizes defect detection parameters using a function that emphasizes low overdetection and high defect detection rates, addressing the suboptimal issues in existing methods by enhancing the accuracy and reliability of visual inspection.

JP2024156354APending Publication Date: 2024-11-06HITACHI LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023070747
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-24
Publication Date
2024-11-06

AI Technical Summary

Technical Problem

Existing automatic visual inspection methods optimize defect detection parameters based on the Area Under the Receiver Operating Characteristic (AUROC) curve, but fail to account for the variability in defect detection rates and overdetection rates across different threshold areas, leading to suboptimal parameters for actual operational conditions.

Method used

A computer system and method that optimizes defect detection parameters using a first function emphasizing low overdetection rates and high defect detection rates, incorporating a GUI for setting the function, and updating parameters to maximize an evaluation value calculated from these rates, with hyperparameters and network coefficients optimized for the defect detection engine.

Benefits of technology

The method enables the optimization of defect detection parameters suitable for actual operational conditions, improving the accuracy and reliability of visual inspection by focusing on areas with low overdetection and high defect detection rates.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156354000001_ABST
    Figure 2024156354000001_ABST
Patent Text Reader

Abstract

To optimize a defect detection parameter based on an over-detection rate and a defect detection rate.SOLUTION: A computer system executes an image acquisition step for acquiring an image of an object, and a parameter optimization step for optimizing a parameter for detecting a defective part in the image. The parameter optimization step includes: a defective part acquisition step for acquiring defective part information indicating the defective part in the image; a first function acquisition step for acquiring a first function for calculating an evaluation value for evaluating the performance of detecting the defective part from an over-detection rate and / or a defect detection rate; a defect detection step for detecting the defective part of the object using the image of the object and the parameter; an evaluation value calculation step for calculating the evaluation value taken by the first function with respect to the over-detection rate and / or the defect detection rate; and a parameter update step for updating the parameter so that the evaluation value is maximized.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a computer system, an appearance inspection method, and an appearance inspection program. [Background technology]

[0002] In industrial products including machines, metals, chemicals, food, and textiles, appearance inspection is widely performed to check whether the product is good or defective based on the image obtained by capturing an image of the product to be inspected. In appearance inspection, for example, shape defects, assembly defects, adhesion of foreign matter, internal defects and criticality, scratches, spots, and dirt on the surface are inspected.

[0003] Many of these appearance inspections are performed by inspectors through visual judgment. However, with the increasing demand for mass production and improved quality, the inspection costs and the burden on inspectors are increasing. In addition, sensory inspections based on human senses require particularly high levels of experience and skill. Issues include the personal nature of the inspection, such as different evaluation values ​​depending on the inspector and different results each time an inspection is performed, and reproducibility. There is a strong demand for the automation of inspections to address issues such as the cost, skill, and personal nature of these inspections.

[0004] Automated visual inspection is a method of automatically determining whether an object is defective (positive) or good (negative) based on the image obtained by capturing the object, and the performance of automated visual inspection is mainly defined by the overdetection rate and the defect detection rate. The overdetection rate is the rate at which good items are mistakenly determined to be defective, and is also called the false positive rate. The defect detection rate is the rate at which defective items are correctly determined to be defective, and is also called the true positive rate.

[0005] The overdetection rate and the defect detection rate are calculated for each pass / fail discrimination threshold for discriminating whether an object to be inspected is a defective product or a non-defective product, and the overdetection rate is plotted on the horizontal axis (x-axis) and the defect detection rate on the vertical axis (y-axis), forming a curve called a ROC (Receiver Operating Characteristic) curve. The area surrounded by this ROC curve and the x-axis and the line at x=1.0 is the AUROC (Area Under ROC). The AUROC is an index indicating that the closer to 1 the AUROC is, the higher the performance of the automatic visual inspection. Therefore, a technology for automatically optimizing the defect detection parameters used in the automatic visual inspection based on this AUROC is being considered. For example, Patent Document 1 discloses a method of calculating the area surrounded by the ROC curve as an ROC evaluation value and storing an algorithm that optimizes this ROC evaluation value. In practical operation, emphasis is placed on a high defect detection rate in an area where the overdetection rate is low (about 0.0 to 0.1) and a low overdetection rate in an area where the defect detection rate is high (about 0.9 to 1.0). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2006-260410 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, even if the AUROC value is the same, the defect detection rate in the region where the overdetection rate is low and the overdetection rate in the region where the defect detection rate is high may differ, and even if the defect detection parameters are optimized based on the AUROC, the defect detection parameters suitable for practical use may not be obtained. This will be described with reference to Figures 12A and 12B.

[0008] Fig. 12A is an example of an ROC curve according to a conventional example obtained using the first defect detection parameter, and Fig. 12B is an example of an ROC curve according to a conventional example obtained using the second defect detection parameter.

[0009] As shown in Fig. 12A, for an ROC curve 1201 obtained using the first defect detection parameter, the area under the shaded region 1202 (AUROC) is 0.8. Also, as shown in Fig. 12B, for an ROC curve 1203 obtained using the second defect detection parameter, the AUROC of a shaded region 1204 is also 0.8.

[0010] However, in the region where the overdetection rate is low, especially in the region where the overdetection rate is 0.0 to 0.1, the ROC curve with the first defect detection parameter has a higher defect detection rate, and in the region where the defect detection rate is even higher, especially in the region where the defect detection rate is 0.9 to 1.0, the ROC curve with the first defect detection parameter has a lower overdetection rate. Therefore, although the first defect detection parameter is considered to be more suitable for practical use, optimization based on AUROC cannot determine which of the first and second defect detection parameters is optimal, and therefore defect detection parameters suitable for practical use may not be obtained.

[0011] As described above, in conventional methods, the evaluation values ​​used when optimizing defect detection parameters do not take into account the high defect detection rate in areas with a low overdetection rate, or the low overdetection rate in areas with a high defect detection rate, and therefore it is not possible to optimize defect detection parameters to suit practical use.

[0012] The present invention has been made in view of the above circumstances, and has an object to optimize defect detection parameters based on the overdetection rate and the defect detection rate. [Means for solving the problem]

[0013] In order to solve the above problem, a computer system according to one embodiment of the present invention is a computer system having one or more processors and one or more memory resources, wherein the one or more processors execute an image acquisition step of acquiring an image of an object, and a parameter optimization step of optimizing parameters for detecting defective portions in the image of the object, the parameter optimization step including: a defective portion acquisition step of acquiring defective portion information indicating the defective portions in the image of the object; a first function acquisition step of acquiring a first function that calculates an evaluation value evaluating performance for detecting the defective portions from an overdetection rate and / or a defective detection rate; a defect detection step of detecting the defective portions of the object using the image of the object and the parameters; an evaluation value calculation step of calculating the evaluation value taken by the first function for at least one of the overdetection rates and / or the defective detection rates obtained based on the detection result in the defect detection step and the defective portion information; and a parameter update step of updating the parameters so that the evaluation value is maximized. Effect of the Invention

[0014] According to the present invention, the defect detection parameters can be optimized based on the overdetection rate and the defect detection rate.

[0015] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an example of an entire processing sequence of the appearance inspection method according to the first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of a table showing the relationship between whether an object is actually defective (positive) or non-defective (negative) and whether it was determined to be defective or non-defective in the defect detection step of the first embodiment. [Diagram 3] FIG. 3 is a schematic diagram showing an example of the first function in the first embodiment. [Figure 4] FIG. 4 is a schematic diagram of an example of a GUI (Graphical User Interface) for acquiring the first function in the first function acquisition step of the first embodiment. [Diagram 5] FIG. 5 is a schematic diagram showing an example of a neural network used in the first embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a processing sequence of the parameter optimization phase in the second embodiment. [Figure 7] FIG. 7 is a diagram showing an example of a processing sequence of the image comparison step in the second embodiment. [Figure 8] FIG. 8 is a diagram showing an example of a processing sequence of the appearance inspection method according to the third embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of a GUI for acquiring the second function in the second function acquiring step of the third embodiment. [Figure 10] FIG. 10 is a schematic diagram showing an example of a GUI for classifying the images of objects according to their level of fatality in the classification step S803 in the third embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a hardware configuration of a computer system that executes the appearance inspection methods according to the first to third embodiments. [Figure 12A] FIG. 12A is an example of an ROC curve according to a conventional example obtained using the first defect detection parameter. [Figure 12B] FIG. 12B is an example of an ROC curve according to the conventional example obtained using the second defect detection parameter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings for describing the embodiment, the same members are generally given the same reference numerals, and the repeated description thereof will be omitted as appropriate. In the following embodiment, the components (including element steps, etc.) are not necessarily essential, unless otherwise specified or considered to be obviously essential in principle. In addition, when "consisting of A", "consisting of A", "having A", or "including A" is used, it is needless to say that other elements are not excluded, except when it is specifically specified that only the element is included. Similarly, in the following embodiment, when referring to the shape, positional relationship, etc. of the components, etc., it includes those that are substantially similar or similar to the shape, etc., except when it is specifically specified or considered to be obviously not essential in principle. Furthermore, the embodiment does not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiment are not necessarily essential to the solution of the invention.

[0018] First Embodiment [Overall process sequence of visual inspection method] The appearance inspection method according to the first embodiment will be described with reference to Fig. 1 to Fig. 4. Fig. 1 is a diagram showing an example of an entire processing sequence of the appearance inspection method according to the first embodiment.

[0019] As shown in FIG. 1, the appearance inspection method according to this embodiment includes an image acquisition step S102 for acquiring an image 103 of an object 101, and a parameter optimization step S104 for optimizing parameters (defect detection parameters 105) for detecting defective areas in the image 103 of the object. The parameter optimization step S104 includes a defect portion acquisition step S106 for acquiring defect portion information 107 indicating a defect portion in an image 103 of the object, a first function acquisition step S108 for acquiring a first function w (109) for calculating an evaluation value e (113) for evaluating the performance of detecting a defect portion from the overdetection rate x and / or the defect detection rate y, a defect detection step S110 for detecting a defect portion of the object using the image 103 of the object and defect detection parameters 105, an evaluation value calculation step S112 for calculating an evaluation value e (113) that the first function w (109) takes for at least one or more overdetection rates x and / or defect detection rates y obtained from the detection result 111 in the defect detection step S110 and the defect portion information 107, and a parameter updating step S114 for updating the defect detection parameters 105 so as to maximize the evaluation value e (113).

[0020] In conventional methods, the evaluation values ​​used when optimizing defect detection parameters do not take into account the high defect detection rate in areas with a low overdetection rate, or the low overdetection rate in areas with a high defect detection rate, and therefore it is not possible to optimize defect detection parameters to suit practical use.

[0021] On the other hand, if the evaluation value used to optimize the fault detection parameters can be calculated with emphasis on the areas with low overdetection rates and areas with high fault detection rates, it is expected that the fault detection parameters suitable for practical use can be obtained by optimizing the evaluation value to maximize it. Therefore, in this embodiment, the degree of emphasis is changed according to the values ​​of the overdetection rate and the fault detection rate using a first function w(x, y) for the overdetection rate x and the fault detection rate y, so that the fault detection parameters suitable for practical use can be obtained.

[0022] As shown in FIG. 1, the processing sequence in this embodiment is divided into a parameter optimization phase 121 and an inspection phase 122.

[0023] In the parameter optimization phase 121, first, an image of the object 101 is captured to obtain an optimization image 103 (step S102). The optimization image 103 can be obtained by capturing an image of the surface or inside of the learning object as a digital video using an imaging device such as a CCD (Charge Coupled Device) camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, or an X-ray inspection device.

[0024] Next, defect detection parameters 105 for detecting defective portions in the image of the object are optimized using optimization image 103 (step S104). Step S104 will be described later in detail.

[0025] In the inspection phase 122, similarly to the parameter optimization phase 121, an object 115 such as an actual product is imaged to obtain an inspection image 117 (step S116). Next, defective parts are detected in the inspection image 117 using the defect detection parameters 105 optimized in step S104, and a detection result 119 is obtained (step S118). Finally, an inspector checks the inspection image 117 in which the defective parts are detected (step S120), and if there are any defects, countermeasures are fed back to the manufacturing process.

[0026] Details of the defect detection parameter optimization step S104 will be described. In this step, defective portion information 107 indicating defective portions in the optimization-use image 103 is acquired (step S106) for each of the multiple optimization-use images 103 in advance. A first function 109 is also acquired (step S108). Details of step S108 will be described later.

[0027] The defective portion information 107 is information indicating defective portions in the optimization image 103. For example, an inspector may generate the defective portion information 107 by visually checking the defective portions in the optimization image 103. Alternatively, the optimization image 103 may be used as an input, and defective portions may be automatically detected based on a previously set rule, and the detection result may be the defective portion information 107.

[0028] Thereafter, defect detection parameters 105 are used to detect defect areas in each of the optimization images 103, and detection results 111 are obtained for each of the optimization images 103 (step S110).

[0029] Furthermore, an evaluation value 113 is calculated based on the detection result 111, the defective portion information 107, and the first function 109 (step S112), and the defect detection parameters 105 are updated (step S114) so ​​as to maximize the evaluation value 113, and this process is repeated to optimize the defect detection parameters 105. Details of step S112 will be described later.

[0030] If a defective portion is detected in the defect detection step S110, the object is judged to be a defective product, and otherwise (if no defective portion is detected), the object is judged to be a non-defective product, and the judgment result is set as the detection result 111.

[0031] In evaluation value calculation step S112, first, an overdetection rate x and a defect detection rate y are calculated based on the detection result 111 and the defective portion information 107. A method for calculating the overdetection rate x and the defect detection rate y will be described with reference to FIG.

[0032] 2 is a schematic diagram showing an example of a table 201 showing the relationship between whether an object is actually a defective product (positive) or a non-defective product (negative) and whether it has been determined to be a defective product or a non-defective product in the defect detection step S110 of this embodiment. Whether an object is actually a defective product (positive) or a non-defective product (negative) can be specified from the defect portion information 107.

[0033] Correctly identifying a defective product as defective is called a true positive, incorrectly identifying a good product as defective is called a false positive, incorrectly identifying a defective product as good is called a false negative, and correctly identifying a good product as good is called a true negative.In Table 201 in Figure 2, the number of true positives is represented by TP, the number of false positives by FP, the number of false negatives by FN, and the number of true negatives by TN.

[0034] The number of true positives TP is the number of optimization images 103 for which a defective portion is indicated in the defective portion information 107 and which are indicated to be defective in the detection result 111 .

[0035] The number of false positives FP is the number of optimization images 103 for which no defective portion is indicated in the defective portion information 107 and which are indicated to be defective in the detection result 111 .

[0036] The number of false negatives FN is the number of optimization images 103 for which a defective portion is indicated in the defective portion information 107 and which are not indicated as defective in the detection result 111 .

[0037] The number of true negatives TN is the number of optimization images 103 for which no defective portion is indicated in the defective portion information 107 and which are not indicated as defective in the detection results 111.

[0038] The overdetection rate x calculated in evaluation value calculation step S112 is the rate at which good products are erroneously determined to be defective, and is expressed as FP / (TN+FP). The defect detection rate y is the rate at which defective products are correctly determined to be defective, and is expressed as TP / (TP+FN).

[0039] The value calculated in evaluation value calculation step S112 may be either the overdetection rate x or the defect detection rate y. When only the overdetection rate x is calculated, the first function is w(x), and when only the defect detection rate y is calculated, the first function is w(y). When both the overdetection rate x and the defect detection rate y are calculated in evaluation value calculation step S112, the first function is w(x, y).

[0040] Also, index values ​​other than the overdetection rate x and the defect detection rate y may be calculated. For example, instead of the overdetection rate x, a conformance rate (TP / (TP+FP)) indicating the rate at which objects determined to be defective are actually defective may be calculated. Also, instead of the defect detection rate y, a false negative rate (FN / (TP+FN)) indicating the rate at which defective items are erroneously determined to be non-defective (defects are overlooked) may be calculated. In the following explanation, a method for calculating both the overdetection rate x and the defect detection rate y will be explained.

[0041] Returning to the explanation of evaluation value calculation step S112, after calculating the overdetection rate x and the defect detection rate y, an evaluation value e is calculated using the overdetection rate x, the defect detection rate y, and a first function w(x, y). The evaluation value e is a value that evaluates the performance of detecting defective parts from the overdetection rate x and / or the defect detection rate y, and is calculated, for example, by the formula (1).

[0042]

number

[0043] In addition, in the defect detection step S110, a different detection result may be output for each pass / fail judgment threshold {T_i (i is a positive integer from 1 to N; N is the number of thresholds)} that judges the object to be inspected as defective or pass. Here, {T_i} is a set of pass / fail judgment thresholds. In the following, a set of elements will be expressed in parentheses like this: "{}".

[0044] In this case, in the evaluation value calculation step S112, the overdetection rate {x_i} and the defect detection rate {y_i} may be calculated for each threshold value from the detection result for each threshold value, and the evaluation value e may be calculated using the overdetection rate {x_i}, the defect detection rate {y_i}, and the first function w(x, y). In this case, the evaluation value e is calculated, for example, by the formula (2).

[0045]

number

[0046] Alternatively, an ROC curve may be drawn using the overdetection rate {x_i} and the defective detection rate {y_i} calculated for each pass / fail threshold, and an evaluation value e may be calculated by weighting the ROC curve with a first function w(x, y). In this case, the evaluation value e is calculated, for example, by the formula (3).

[0047]

number

[0048] Here, ROC(x) in formula (3) is an ROC curve (a function that represents the defect detection rate relative to the overdetection rate x).

[0049] [First function w] The first function w will now be described in detail. The first function w in this embodiment has a larger value in an area where the overdetection rate x is lower and / or the defect detection rate y is higher.

[0050] As described above, the first function w is a function for calculating the evaluation value e that places emphasis on the region with a low overdetection rate x and the region with a high defect detection rate y. Although the first function w is an arbitrary function, by placing emphasis on the region with a low overdetection rate x and the region with a high defect detection rate y and by increasing the value in the region with a low overdetection rate x and the region with a high defect detection rate y, it is possible to obtain defect detection parameters suitable for practical use.

[0051] 3 is a schematic diagram showing an example of the first function w(x, y) (301) in this embodiment. In FIG. 3, a white area 302 (an area with a low overdetection rate x and a high defect detection rate y) is an area where w(x, y) = 1.0, and a shaded area 303 is an area where w(x, y) = 0.0.

[0052] When the first function w, which has a large value in the region where the overdetection rate x is low and the defect detection rate y is high as in the example of Figure 3, is used to calculate the evaluation value e using the formula (3) for the ROC curves of the first defect detection parameter and the second defect detection parameter shown in Figures 12A and 12B, the evaluation value for the first defect detection parameter is 0.58, and the evaluation value for the second defect detection parameter is 0.45. This shows that the first defect detection parameter, which shows a large evaluation value, is suitable for practical use.

[0053] [Settings screen for the first function w] A method for acquiring the first function w in the first function acquisition step S108 will be described below. In this embodiment, in the first function acquisition step S108, a GUI (Graphical User Interface) for setting the first function w is displayed on a GUI device.

[0054] Basically, the first function w can be any function, but it is desirable for it to have a large value in the region with a low overdetection rate and a region with a high defect detection rate.

[0055] However, the range where the overdetection rate is low and the defect detection rate is high differ depending on the purpose of the inspection. For example, when the object is close to completion, it is not desirable to ship defective products, so the defect detection rate range of about 0.98 to 1.0 is likely to be emphasized. Also, when the impact is relatively small even if defects are overlooked in the early stages of manufacturing, the overdetection rate range of about 0.0 to 0.03 is likely to be emphasized in order to reduce the frequency of inspections by inspectors.

[0056] Therefore, in this embodiment, a GUI for setting the first function w is displayed. This allows the first function w to be changed according to the purpose of the inspection, and the evaluation value e can be calculated based on the set first function w, thereby optimizing the defect detection parameters.

[0057] 4 is a schematic diagram of an example of a GUI for acquiring the first function w in the first function acquisition step S108 of this embodiment. Examples of GUI devices that display this GUI include a display and a projector. Examples of input devices that allow a user to perform input operations such as pressing a button while viewing the GUI include a keyboard, a mouse, and a touch panel.

[0058] As shown in FIG. 4, the GUI has buttons 401 and 402 for selecting whether to set the first function w by selecting it from predefined presets or by inputting a mathematical expression, an area 403 for selecting a preset, an area 404 for inputting a mathematical expression, an area 405 showing the value of the first function w, a button 406 for completing the setting of the first function w, and a button 407 for canceling the setting of the first function w.

[0059] When a user clicks a button 401 for selecting from presets and selects a preset in area 403, the value of the function displayed in area 403 is displayed in area 405. In area 405, areas with larger function values ​​are displayed in a color closer to white, and areas with smaller function values ​​are displayed in a color closer to black.

[0060] When button 402 for inputting a mathematical expression is clicked and a mathematical expression is input in area 404, the value of the function displayed in area 404 is displayed in area 405. The user can set a first function w according to the purpose of the inspection while checking the value of the first function w displayed in area 405. Setting of the first function w can be ended by pressing button 406, and setting of the first function w can be canceled by pressing button 407.

[0061] [Fault detection parameters] The defect detection parameters 105 will be described in detail. In this embodiment, the defect detection parameters 105 are hyperparameters related to a network structure of a defect portion identification engine that identifies a defect portion from an input image, and / or weighting coefficients of the network of the defect portion identification engine. In this embodiment, the defect detection parameters 105 are optimized by using the image 103 of the object 101 and defect portion information 107 in a parameter optimization step S104.

[0062] In recent years, the performance of machine learning has improved dramatically with the proposal of deep network models. In defect detection based on machine learning, an inspector visually instructs the defective parts in advance on an image of the object, and the system learns the correspondence between the image of the object and the teaching results. However, even if the inspector accurately instructs the system, if the structure of the network model or the optimization of the network weighting coefficients is insufficient, the defective parts cannot be identified with high accuracy in the image of the object.

[0063] Therefore, in this embodiment, hyperparameters related to the network structure in a defective portion identification engine that identifies a defective portion from an input image and network weighting coefficients are optimized as defect detection parameters 105. This makes it possible to obtain a defective portion identification engine that is suitable for practical use.

[0064] Fig. 5 is a schematic diagram showing an example of a neural network used in this embodiment. As the defective portion identification engine in this embodiment, a neural network having a three-layer structure as shown in Fig. 5 may be used. Here, Y indicates the input image, F1(Y) and F2(Y) indicate intermediate data, and F(Y) is the identification result of the defective portion. The intermediate data and the estimation result are calculated by the following formulas (4) to (6).

[0065]

number

[0066]

number

[0067]

number

[0068] Here, "*" represents a convolution operation. Here, U1 is A1 filters of size A0×f1×f1, A0 is the number of channels of the input image, and f1 is the size of the spatial filter. A1-dimensional feature map is obtained by convolving the input image with filters of size A0×f1×f1 A1 times. B1 is an A1-dimensional vector and a bias component corresponding to the A1 filters. Similarly, U2 is A2 filters of size A1×f2×f2, B2 is an A2-dimensional vector, U3 is A0 filters of size A2×f3×f3, and B3 is an A0-dimensional vector. The above-mentioned A0 is a value determined by the number of channels of the input image. Also, f1, f2, A1, and A2 are hyperparameters related to the network structure of the defective part identification engine, and U1, U2, U3, B1, B2, and B3 are weight coefficients of the network of the defective part identification engine. These are targets to be optimized as defect detection parameters 105 in the parameter optimization step S104 in this embodiment.

[0069] As the network structure of the defect portion identification engine, other than the structure shown in FIG. 5, for example, the network structure described in the following paper (Ronneberger et al.) may be used.

[0070] Olaf Ronneberger, Philipp Fischer, Thomas Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, arXiv preprint arXiv:1505.04597 (2015)

[0071] The number of layers in this network model, the number and size of convolution filters, and the type of activation function are hyperparameters related to the network structure of the defect portion identification engine, and the weighting coefficients between nodes are weighting coefficients of the network of the defect portion identification engine, and these are targets to be optimized as defect detection parameters 105 in parameter optimization step S104 in this embodiment.

[0072] In the present embodiment described above, as shown in FIG. 1, a parameter optimization phase 121 and an inspection phase 122 are included. The same business entity may perform the parameter optimization phase 121 and the inspection phase 122. Alternatively, different business entities may perform the parameter optimization phase 121 and the inspection phase 122. In this case, for example, a certain business entity performs the parameter optimization phase 121 to update defect detection parameters so that the evaluation value e is maximized. Then, a business entity other than the one that performed the parameter optimization phase 121 may execute the inspection phase using the updated parameters. This allows inspection of an object 115 such as an actual product to be performed using defect detection parameters optimized to be suitable for practical operation.

[0073] <Second embodiment> An appearance inspection method in the second embodiment will be described with reference to Fig. 6 to Fig. 7. In the first embodiment, a method for obtaining defect detection parameters suitable for practical use by optimizing the defect detection parameters using a first function for the overdetection rate and / or defect detection rate has been described. In particular, a method for optimizing hyperparameters and network weighting coefficients related to a network structure in a defect portion identification engine that identifies a defect portion from an input image has been described as the defect detection parameters.

[0074] In the second embodiment, a method is described in which a pseudo-good image is estimated from an image of an object using a pseudo-good image estimation engine that estimates a pseudo-good image corresponding to an input image, and the parameters used when comparing the image of the object and the pseudo-good image are optimized as defect detection parameters.

[0075] In the processing sequence of the appearance inspection method in the second embodiment, the image acquisition step S102, the defective part acquisition step S106, and the first function acquisition step (S108) in the parameter optimization phase 121, and the inspection phase 122 are similar to those in FIG. 1 of the first embodiment, and therefore will not be described.

[0076] The appearance inspection method in the second embodiment further includes a pseudo-good image estimation engine learning step S602 in which a pseudo-good image estimation engine 603, which estimates a pseudo-good image from an input image, is trained using an optimization-use good image 601 of the object. The defect detection step S608 includes a pseudo-good image estimation step S604 in which a pseudo-good image 605 is estimated from the image 103 of the object using the pseudo-good image estimation engine 603, and an image comparison step S606 in which a defective portion of the object is detected by comparing the image 103 of the object with the pseudo-good image 605. The parameters optimized in the parameter optimization step S104 are the parameters used in the image comparison step S606.

[0077] In defect detection based on machine learning, it is desirable to prepare a large number of images of good products and images of defective products during learning, but in industrial product manufacturing lines, while it is relatively easy to collect a small number of images of defective products, it is often difficult or time-consuming to collect a large number of images of defective products. Therefore, in this embodiment, learning is performed using only images of good products, and when optimizing defect detection parameters, evaluation values ​​are calculated using images of good products and images of defective products. It is possible to optimize defect detection parameters using only images of good products without collecting a large number of images of defective products.

[0078] 6 is a diagram showing an example of a processing sequence of the parameter optimization phase in this embodiment. Note that in FIG. 6, the target object 101 and steps S102, S106, and S108 common to FIG.

[0079] First, a plurality of optimization-use good product images 601, which are images of good products among the optimization-use images 103, are used to train the pseudo-good product image estimation engine 603 (step S702). The pseudo-good product image estimation engine 603 will be described in detail later, but when an image of an object is input to the pseudo-good product image estimation engine 603, the pseudo-good product image estimation engine 603 estimates and outputs a pseudo-good product image corresponding to the input image. Images of defective products are not trained on the pseudo-good product image estimation engine 603. Therefore, when an image of a defective product is input to the pseudo-good product image estimation engine 603, an image different from the input image of the defective product is output. Therefore, by comparing the input image with the output image, it is possible to detect defective parts.

[0080] Next, a plurality of optimization-use images 103 are input to a pseudo-good image estimation engine 603, and a pseudo-good image 605 corresponding to each optimization-use image 103 is estimated for each optimization-use image 103 (step S604).

[0081] Thereafter, the optimization image 103 is compared with the corresponding pseudo-good image 605 using the defect detection parameters 607 (step S606). By performing such a comparison for all optimization images 103, defective areas in each optimization image 103 are detected, and detection results 111 indicating the defective areas are obtained for each optimization image 103 (step S608).

[0082] Next, as in the first embodiment, an overdetection rate x and a defect detection rate y are obtained from the defective portion information 107, the detection result 111, and the first function 109, and an evaluation value 113 is calculated based on the overdetection rate x and the defect detection rate y (step S112).

[0083] Then, the defect detection parameters 607 are repeatedly updated (step S114) so ​​as to maximize the evaluation value 113, thereby optimizing the defect detection parameters.

[0084] Details of the image comparison step S606 will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of a processing sequence of the image comparison step S606 in this embodiment.

[0085] In the image comparison step S606, for example, a difference image 702 having, as pixel values, difference values ​​for each pixel between the optimization image 103 and a pseudo-good image 605 estimated therefrom is calculated (step S701).

[0086] Next, a portion (candidate defect portion) 704 where the pixel value of the difference image 702 is greater than a difference threshold value 708 is detected (step S703), and an abnormality score 706 is calculated for the candidate defect portion 704 (step S705).

[0087] Next, if the abnormality score 706 is greater than the pass / fail determination threshold 709, the object is determined to be a defective product, and a detection result 111 indicating whether the product is pass or fail is obtained (step S707).

[0088] The difference threshold value 708 and the pass / fail determination threshold value 709 described above are targets to be optimized as the defect detection parameters 607 in the parameter optimization step S104 in this embodiment. As the abnormality score 706, for example, the sum of the pixel values ​​of the difference image in the defect candidate portion 704 may be used.

[0089] The pseudo-good image estimation engine 603 will now be described in detail. As the pseudo-good image estimation engine 603, for example, a neural network having a three-layer structure as shown in Fig. 5 or a network structure described in the aforementioned paper (Ronneberger et al.) may be used, like the defective portion identification engine.

[0090] In the learning step S602 of the pseudo-good image estimation engine, the weight coefficients between the nodes in the pseudo-good image estimation engine are optimized so that the estimated image is the same as the input image, using the optimization good image 601 as input. The network structure described in the paper (Ronneberger et al.) is called an encoder-decoder model, which compresses (encodes) the high-dimensional features of an image into low-dimensional features and then restores (decodes) them to high-dimensional features again. By learning so that the estimated image matches the input good image, the original good image is restored when a good image is input, but the defective parts cannot be restored when a defective image is input, so it is expected that the difference in the defective parts will be large when the input image and the estimated image are compared.

[0091] <Third embodiment> The appearance inspection method according to the third embodiment will be described with reference to Fig. 8 to Fig. 10. Fig. 8 is a diagram showing an example of a processing sequence of the appearance inspection method according to the third embodiment. Note that in Fig. 8, an object 101 and steps S102 and S106 common to Fig. 1 are omitted.

[0092] In the first and second embodiments, a method for obtaining a defect detection parameter suitable for practical use by optimizing the defect detection parameter using a first function for the overdetection rate and / or the defect detection rate is described. In the third embodiment, a method for obtaining a defect detection parameter suitable for practical use by optimizing the defect detection parameter using a second function for the overdetection rate and / or the defect detection rate for each defect type (category) is described.

[0093] In the processing sequence of the appearance inspection method in the third embodiment, the image acquisition step S102 and the defective portion acquisition step S106 in the parameter optimization phase 121, and the inspection phase 122 are similar to those in FIG. 1 of the first embodiment, and therefore the description thereof will be omitted.

[0094] In the appearance inspection method of the third embodiment, the parameter optimization step S104 includes a classification step S803 for classifying the image 103 of the object into each defect category c (c is a positive integer from 1 to M; M is the number of categories), and a second function acquisition step S801 for acquiring a second function v_c (802) which is an element for each defect category c of the first function and calculates an element evaluation value d_c which is an element of the evaluation value e (213) from an overdetection rate x and / or a defect detection rate y. In an evaluation value calculation step S805, the evaluation value e (213) is calculated based on the element evaluation value d_c which the second function v_c (802) takes for at least one or more overdetection rates x and / or defect detection rates y obtained for each defect category c from the detection result 111 in the defect detection step S110.

[0095] Defects that occur on industrial product manufacturing lines are diverse, and the impact they have on the target varies depending on the type (category) of the defect, so in general, the overdetection rate and defect detection rate required for each defect category also differ. For example, defects such as processing errors where a hole is not drilled where a hole should be, have a large negative impact on the performance of the target, so a very high defect detection rate (around 0.98 to 1.0) is required. On the other hand, fine scratches such as marks from cutting processing often have a small impact on the performance of the target, so a low overdetection rate is required.

[0096] However, in the past, when the items to be emphasized differ depending on the defect category, it was not possible to obtain an appropriate defect detection parameter. Therefore, in this embodiment, a second function for each defect category is used as an element constituting the first function in the first and second embodiments. This makes it possible to obtain an appropriate defect detection parameter even when the items to be emphasized differ depending on the defect category.

[0097] In this embodiment, first, a second function v_c (802) for each defect category c (c is a positive integer from 1 to M; M is the number of categories) is obtained (step S801).

[0098] Next, the multiple optimization images 103 are classified into defect categories (S803), and a classification result 804 indicating the classification result is obtained.

[0099] Then, defect detection parameters 105 are used to detect defect areas in each of the optimization images 103, and detection results 111 are obtained for each optimization image 103 (step S110).

[0100] Next, an element evaluation value d_c (807) is calculated for each defect category c based on the detection result 111, the defective portion information 107, the classification result 804, and the second function v_c (802) (step S806). Details of step S806 will be described later.

[0101] Next, the evaluation value e(113) is calculated using the element evaluation value {d_c} (step S808), and the defect detection parameters 105 are updated so that the evaluation value e(113) is maximized (step S114). Then, steps S806, S808, and S114 are repeated to optimize the defect detection parameters.

[0102] In element evaluation value calculation step S806, for each defect category c (c is a positive integer from 1 to M; M is the number of categories), an overdetection rate {x_(i, c) (i is a positive integer from 1 to N; N is the number of thresholds)} and a defect detection rate {y_(i, c)} are calculated based on the detection results 111 and the defective part information 107, and an element evaluation value d_c is calculated using equations (1) to (3) of the first embodiment by replacing e with d_c.

[0103] When calculating the element evaluation value d_c using formula (2), x_i and y_i are replaced with x_(i, c) and y_(i, c), respectively. When calculating the element evaluation value d_c using formula (3), the overdetection rate {x_(i, c)} and the defect detection rate {y_(i, c)} calculated for each defect category c and pass / fail judgment threshold T_i are used to draw an ROC curve for each defect category c, and the element evaluation value d_c is calculated by weighting the ROC curve with the second function v_c(x, y).

[0104] [Severity p_c for each defect category c] A method of calculating an evaluation value using the element evaluation values ​​in the evaluation value calculation step S805 will be described in detail. The evaluation value calculation step S805 in this embodiment calculates an evaluation value e (113) based on the importance p_c (810) for each defect category c and the element evaluation value d_c.

[0105] As mentioned above, there are many different types of defects that occur on industrial production lines. In actual operation, the importance of each defect category is different, and it is important to detect and address defect categories with higher importance before those with lower importance. For example, by prioritizing measures against defect categories that occur frequently over those that occur infrequently, the defect rate can be reduced early on.

[0106] However, in the past, it was difficult to give priority to detection of a defect category with a high importance. Therefore, in this embodiment, the evaluation value is calculated by weighting the element evaluation value d_c using the importance p_c of each defect category c. This makes it possible to increase the contribution rate of a defect category with a high importance in the calculation of the evaluation value, and as a result, it becomes possible to detect a defect category with a high importance with priority.

[0107] In the parameter optimization phase 121 in this embodiment, the importance p_c (810) for each defect category c is obtained (step S809), and an evaluation value e is calculated using the importance {p_c} and the element evaluation value {d_c} (step S808). The evaluation value e is calculated, for example, by the formula (7).

[0108]

number

[0109] The importance p_c may be a value determined in advance for each defect category, or may be determined by a user's input using a GUI or the like.

[0110] [Setting screen for second function v_c for each defect category c] 9 is a schematic diagram showing an example of a GUI for acquiring the second function in the second function acquisition step S801 of this embodiment. Examples of GUI devices that display this GUI include a display and a projector. Examples of input devices that allow a user to perform operations such as pressing a button while viewing the GUI include a keyboard, a mouse, and a touch panel.

[0111] The GUI has an area 901 for selecting a defect category, buttons 902 and 903 for selecting whether to set the second function for the defect category selected in area 901 by selecting it from pre-defined presets or by entering a formula, an area 904 for selecting a preset, an area 905 for entering a formula, an area 906 showing the value of the second function, a button 907 for completing the setting of the second function, and a button 908 for canceling the setting of the second function.

[0112] When a defect category is selected in area 901, it becomes possible to set a second function for the selected defect category. When a select from presets button 902 is clicked and a preset is selected in area 904, the value of the function displayed in area 904 is displayed in area 906. In area 906, areas with larger function values ​​are displayed in a color closer to white, and areas with smaller function values ​​are displayed in a color closer to black.

[0113] When a formula input button 903 is clicked and a formula is input in area 905, the value of the function displayed in area 905 is displayed in area 906. The user can set the second function according to the defect category while checking the value of the second function displayed in area 906. The user can finish setting the second function by pressing button 907, and cancel setting the second function by pressing button 908.

[0114] In the example of FIG. 9, the second function is displayed with the label "v", but the second function may be represented with a different label "v_c" for each category c as described above.

[0115] [Defect category] The defect categories classified in the classification step will now be described. In the classification step S803 in this embodiment, the image 103 of the object is classified into different defect categories based on the degree of fatality of the defect.

[0116] As mentioned above, there are many different types of defects that occur on industrial production lines. In actual operation, when a defect occurs, it is treated as a highly fatal defect if it has a large adverse effect on the performance of the production line or the target product, and a high defect detection rate (approximately 0.98 to 1.0) is required for highly fatal defects. On the other hand, a low overdetection rate is required for non-fatal defects.

[0117] However, in the conventional appearance inspection method, when there is a mixture of highly fatal and non-fatal defects, it is not possible to obtain defect detection parameters suitable for practical use. Therefore, in this embodiment, the image of the object is classified into different defect categories according to the degree of fatality, and an evaluation value is calculated using a different second function for each defect category, thereby optimizing the defect detection parameters. As a result, it is possible to obtain defect detection parameters that emphasize a high defect detection rate for highly fatal defects and a low overdetection rate for non-fatal defects, for example.

[0118] FIG. 10 is a schematic diagram showing an example of a GUI for classifying the image of an object according to the degree of fatality in the classification step S803 of this embodiment.

[0119] The GUI has, for each of the object images 1001-1004, areas 1005-1008 for classifying the defect category, a button 1012 for completing the classification, and a button 1013 for canceling the classification.

[0120] 10, image 1001 contains a chip defect 1009, image 1002 contains a scratch defect 1010, and image 1014 contains a foreign substance defect 1011. For each image of the object, classification is performed by selecting a defect category in areas 1005 to 1008, and the classification of the defect category can be ended by pressing button 1012, and the classification can be canceled by pressing button 1013.

[0121] 10, image 1001 is classified as a fatal defect, which indicates a defect with a high degree of fatality, image 1002 is classified as a non-fatal defect, which indicates a defect with a low degree of fatality, image 1003 is classified as a non-defective product, which indicates that the object is not a defective product, and image 1004 is unclassified, which indicates that the object is not classified into any defect category or a non-defective product.

[0122] In FIG. 10, an example of classifying defects into two categories, fatal defects and non-fatal defects, has been described, but defects may be classified into categories other than these, for example, into different defect categories depending on the size of the defect. Also, it may be possible to classify defects into three or more categories. An example of classifying an image of an object using a GUI has been described, but a classification engine that automatically classifies an input image using a convolutional neural network or the like may also be used. Also, an image of an object may be input and automatically classified based on previously set rules.

[0123] [Computer System] FIG. 11 is a diagram showing an example of a hardware configuration of a computer system that executes the appearance inspection methods according to the first to third embodiments.

[0124] The computer system includes an imaging device 1101 and a computer 1102. As described in the first embodiment, examples of the imaging device 1101 include a CCD camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, and an X-ray inspection device.

[0125] The computer 1102 is a component that processes the appearance inspection method described in this embodiment, and includes the following:

[0126] Processor 1103: Examples of the processor 1103 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array), but other components may be used as long as they can process the outer pipe inspection method. One or more processors 1103 may be included in the computer 1102.

[0127] Memory resource 1104: Examples of the memory resource 1104 include a RAM (Random Access Memory), a ROM (Read Only Memory), a HDD (Hard Disk Drive), and a non-volatile memory (such as a flash memory). The memory resource may store a program (called an appearance inspection program) that causes a processor to execute the appearance inspection method described in the first to third embodiments. One or more memory resources 1104 may be included in the computer 1102.

[0128] GUI device 1105: As mentioned above, examples of the GUI device 1105 include a display and a projector, but other devices may be used as long as they are capable of displaying a GUI.

[0129] Input device 1106: As mentioned above, examples of the input device 1106 include a keyboard, a mouse, and a touch panel, but other devices may be used as long as they are components that can accept operations from a user. Also, the input device 1106 and the GUI device 1105 may be integrated into one device.

[0130] Communication interface 1107: Examples of the communication interface 1107 include USB (Universal Serial Bus), ETHERNET (registered trademark), and Wi-Fi, but other interface devices may be used as long as they are interfaces that can directly receive images from the imaging device 1101 or that allow a user to transmit the images to the computer 1102. In addition, a portable non-volatile storage medium (such as a flash memory, a DVD (Digital Versatile Disc), a CD (Compact Disk)-ROM, or a Blu-ray disc) on which the images are stored may be connected to the communication interface 1107, and the images may be stored in the computer 1102.

[0131] The above is the hardware configuration of the computer. Note that the computer system may include a plurality of computers 1102 and a plurality of image capturing devices 1101.

[0132] The above-mentioned appearance inspection program may be stored in the computer 1102 via the following route.

[0133] The appearance inspection program is stored in a non-volatile portable non-volatile storage medium, and the medium is connected to the communication interface 1107 to distribute the program to the computer 1102 .

[0134] The appearance inspection program is distributed to the computer 1102 by a program distribution server. The program distribution server has a memory resource storing the appearance inspection program, a processor performing a distribution process to distribute the appearance inspection program, and a communication interface capable of communicating with the communication interface of the computer.

[0135] This concludes the description of the embodiments. As mentioned above, the embodiments described thus far do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0136] For example, the configurations, functions, processing means, and parts of the computer system described above are realized in part or in whole by executing an appearance inspection program on the processor 1103. However, some or all of these may be realized in hardware by designing an integrated circuit, etc. Also, the computer system may be realized by a user (operator) executing some or all of the functions and processes realized by each program.

[0137] There are cases where the computer system does not have the GUI device 1105, and instead leaves some of the output processing to the user and input processing from the user to a processor system (called an external processor system) such as a smartphone or tablet external to the system. In such cases, the computer system (or the processor 1103, the appearance inspection program) may do the following to execute the processing as described above and other parts of the program.

[0138] Instead of outputting to the user using the GUI device 1105 described above, data required for outputting to the user is transmitted to an external processor system via a communication interface 1107. Examples of the data include the data to be output itself and data for generating output data in another processor system, but it may also be a program describing the process of performing user output in the external processor system or Web data.

[0139] As an alternative to receiving input or actions from a user using the input device 1106 as described above, data indicative of user input or actions is received from an external processor system via the communications interface 1107 .

[0140] From another point of view, the meaning of outputting data to a user may include not only the computer system itself outputting the data, but also having another entity other than the computer system output the data (using the computer system to do so). Also, the meaning of receiving input or operation from a user may include the computer system indirectly receiving input, in addition to the user directly inputting via the input device 1106 of the computer system. [Explanation of symbols]

[0141] 101, 115... object, 103... image for optimization, 105... defect detection parameters, 107... defective part information, 109... first function, 111, 119... detection result, 113... evaluation value, 117... inspection image, 121... parameter optimization phase, 122... inspection phase, 302, 303, 403 to 405, 901, 904 to 906, 1005 to 1008... area, 401, 402, 406, 407, 902, 903, 907, 908, 1012, 1013... button, 601... good product image, 601... good product image for optimization, 603... pseudo-good product image estimation engine, 605 ...pseudo-good product image, 607...defect detection parameters, 702...difference image, 704...defect candidate area, 706...anomaly score, 708...difference threshold, 709...good / bad discrimination threshold, 804...classification result, 1001-1004, 1014...image, 1009...chipped defect, 1010...scratch defect, 1011...foreign object defect, 1101...imaging device, 1102...computer, 1103...processor, 1104...memory resources, 1105...GUI device, 1106...input device, 1107...communication interface, 1201, 1203...ROC curve, 1202, 1204...area.

Claims

1. 1. A computer system having one or more processors and one or more memory resources, The one or more processors: An image acquisition step of acquiring an image of the object; a parameter optimization step of optimizing parameters for detecting defects in the image of the object; The parameter optimization step includes: a defect portion acquiring step of acquiring defect portion information indicating the defect portion in the image of the object; a first function acquisition step of acquiring a first function for calculating an evaluation value for evaluating the performance of detecting the defective portion from an overdetection rate and / or a defect detection rate; a defect detection step of detecting the defect portion of the object using the image and the parameters of the object; an evaluation value calculation step of calculating the evaluation value that the first function takes for at least one of the overdetection rate and / or the defect detection rate obtained based on the detection result in the defect detection step and the defective portion information; A parameter updating step of updating the parameters so as to maximize the evaluation value. Computer system.

2. 2. The computer system of claim 1, the evaluation value obtained from the first function is larger in an area where the overdetection rate is low and / or the defect detection rate is high; Computer system.

3. 2. The computer system of claim 1, The one or more processors display a GUI (Graphical User Interface) for setting the first function on a GUI device in the first function acquisition step. Computer system.

4. 2. The computer system of claim 1, the parameter is a hyperparameter related to a network structure of a defect portion identification engine that identifies a defect portion from an input image and / or a weighting coefficient of the network of the defect portion identification engine, the one or more processors optimize the parameters by using the image of the object and the defective portion information in the parameter optimization step; Computer system.

5. 2. The computer system of claim 1, The one or more processors further execute a pseudo-good image estimation engine learning step of learning a pseudo-good image estimation engine that estimates a pseudo-good image from an input image by using the good image of the object; The defect detection step includes: a pseudo-quality image estimation step of estimating the pseudo-quality image from the image of the object using the pseudo-quality image estimation engine; and an image comparison step of detecting the defective portion of the object by comparing the image of the object with the pseudo-good image, The parameters optimized in the parameter optimization step are parameters used in the image comparison step. Computer system.

6. 2. The computer system of claim 1, The parameter optimization step includes: classifying the images of the objects into defect categories; a second function acquisition step of acquiring a second function which is an element of the first function for each of the defect categories, the second function being used to calculate an element evaluation value which is an element of the evaluation value from the overdetection rate and / or the defect detection rate; the one or more processors, in the evaluation value calculation step, calculate the evaluation value based on the element evaluation value that the second function takes for at least one of the overdetection rate and / or the defect detection rate obtained for each of the defect categories from the detection result in the defect detection step. Computer system.

7. 7. The computer system of claim 6, the one or more processors calculate the evaluation value based on the importance of each of the defect categories and the element evaluation value in the evaluation value calculation step; Computer system.

8. 7. The computer system of claim 6, and wherein the one or more processors classify the images of the object into different defect categories based on a degree of criticality of the defect in the classifying step. Computer system.

9. 2. The computer system of claim 1, The one or more processors further perform the step of detecting defects in an image of the product using the updated parameters. Computer system.

10. A visual inspection method executed by a computer system having one or more processors and one or more memory resources, comprising: An image acquisition step of acquiring an image of the object; and optimizing parameters for detecting defects in the image of the object, The parameter optimization step includes: a defect portion acquiring step of acquiring defect portion information indicating the defect portion in the image of the object; a first function acquisition step of acquiring a first function for calculating an evaluation value for evaluating the performance of detecting the defective portion from an overdetection rate and / or a defect detection rate; a defect detection step of detecting the defect portion of the object using the image and the parameters of the object; an evaluation value calculation step of calculating the evaluation value that the first function takes for at least one of the overdetection rate and / or the defect detection rate obtained based on the detection result in the defect detection step and the defective portion information; A parameter updating step of updating the parameters so as to maximize the evaluation value. Visual inspection method.

11. The appearance inspection method according to claim 10, the evaluation value obtained from the first function is larger in an area where the overdetection rate is low and / or the defect detection rate is high; Visual inspection method.

12. The appearance inspection method according to claim 10, In the first function acquisition step, a GUI for setting the first function is displayed on a GUI device. Visual inspection method.

13. The appearance inspection method according to claim 10, the parameter is a hyperparameter related to a network structure of a defect portion identification engine that identifies a defect portion from an input image and / or a weighting coefficient of the network of the defect portion identification engine, In the parameter optimization step, the parameters are optimized using the image of the object and the defective portion information. Visual inspection method.

14. The appearance inspection method according to claim 10, The method further includes a pseudo-good image estimation engine learning step of learning a pseudo-good image estimation engine that estimates a pseudo-good image from an input image by using the good image of the object; The defect detection step includes: a pseudo-quality image estimation step of estimating the pseudo-quality image from the image of the object using the pseudo-quality image estimation engine; and an image comparison step of detecting the defective portion of the object by comparing the image of the object with the pseudo-good product image, The parameters are parameters used in the image comparison step. Visual inspection method.

15. A visual inspection method according to any one of claims 10 to 14, wherein the visual inspection method is executed by a computer system. Visual inspection program.

Citation Information

Patent Citations

  • Evaluation apparatus and method for image processing algorithm, producing system and method, program for making computer function as evaluation system, and program for making computer function as producing system

    JP2006260410A