Computer system, appearance inspection method, and appearance inspection program

Abstract

And optimizing disqualification detection parameters according to the over-detection rate and the disqualification detection rate. A computer system executes: an image acquisition step of acquiring an image of an object; and a parameter optimization step for optimizing parameters for detecting the unqualified parts in the image, the parameter optimization step comprising: an unqualified part acquisition step for acquiring unqualified part information representing the unqualified parts in the image; a first function acquisition step for acquiring a first function for calculating an evaluation value for evaluating the performance of detecting the defective part on the basis of an over-detection rate and / or a defective detection rate; a disqualification detection step of detecting the disqualification part of the object by using the image of the object and the parameter; an evaluation value calculation step of calculating the evaluation value obtained by the first function according to the over-detection rate and / or the unqualified detection rate; and a parameter updating step in which the parameter is updated so that the evaluation value is maximized.

Description

Technical Field

[0001] The present invention relates to a computer system, a visual inspection method, and a visual inspection program. This application claims priority to Japanese Patent Application No. 2023-070747, filed on April 24, 2023. The contents of this application are incorporated herein by reference in designated countries where incorporation by reference is permitted. Background Art

[0002] Visual inspections are widely used in industrial products, including machinery, metals, chemicals, foods, and textiles, to check whether a product is acceptable or defective by capturing images of the product. Examples of visual inspections include checking for shape defects, assembly defects, foreign matter adhesion, internal defects or damage, and surface damage, stains, and dirt.

[0003] These appearance inspections are mostly performed visually by human inspectors. However, with the increasing demand for mass production and improved quality, the cost of inspection and the burden on inspectors are increasing. Furthermore, sensory inspections based on human senses require a high level of experience and skill. Evaluations vary between inspectors, and results vary from one inspection to the next, creating challenges related to individuality and reproducibility. To address these challenges, there is a strong demand for automated inspections.

[0004] Automated visual inspection uses images of the object to automatically determine whether it is a defective (positive) or a qualified (negative) product. The performance of automated visual inspection is primarily defined by its over-detection rate and unqualified detection rate. The over-detection rate is the percentage of qualified products mistakenly identified as defective, also known as the false positive rate. The unqualified detection rate is the percentage of unqualified products correctly identified as defective, also known as the true positive rate.

[0005] The over-detection rate and the unqualified detection rate are calculated for each pass / fail discrimination threshold for distinguishing the inspection object as a defective product or a qualified product, and the curve drawn with the over-detection rate as the horizontal axis (x-axis) and the unqualified detection rate as the vertical axis (y-axis) is the ROC (Receiver Operating Characteristic) curve. The area of ​​the portion surrounded by the ROC curve, the x-axis, and the straight line of x=1.0 is the AUROC (Area Under ROC). The AUROC is an indicator indicating that the closer it is to 1, the higher the performance of the automatic appearance inspection. Therefore, a technology for automatically optimizing the unqualified detection parameters used in the automatic appearance inspection based on the AUROC is being studied. For example, in Patent Document 1, a method is disclosed of calculating the area of ​​the area surrounded by the ROC curve as the ROC evaluation value and saving the ROC evaluation value as the optimal algorithm. In actual use, the high unqualified detection rate in areas with low over-detection rates (about 0.0 to 0.1) and the low over-detection rate in areas with high unqualified detection rates (about 0.9 to 1.0) are given importance. Prior art literature Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2006-260410 Summary of the Invention Problems to be solved by the invention

[0007] However, even if the AUROC value is the same, there may be cases where the failure detection rate in the area with low over-detection rate is different from the over-detection rate in the area with high failure detection rate. Even if the failure detection parameters are optimized based on AUROC, there may be cases where failure detection parameters suitable for actual application cannot be obtained. Figure 12A 、 12B Provide explanation.

[0008] Figure 12A This is an example of a conventional ROC curve obtained using the first failure detection parameter. Figure 12B This is an example of a conventional ROC curve obtained using the second failure detection parameter.

[0009] like Figure 12A As shown in FIG. 1 , with respect to the ROC curve 1201 obtained using the first unqualified detection parameter, the area (AUROC) of the region 1202 indicated by the oblique lines is 0.8. Figure 12B As shown, with respect to the ROC curve 1203 obtained using the second failure detection parameter, the AUROC of the region 1204 indicated by the diagonal lines is also 0.8.

[0010] However, in areas with low overdetection rates, particularly those with overdetection rates of 0.0 to 0.1, the ROC curve for the first failure detection parameter shows a high failure detection rate. Furthermore, in areas with high failure detection rates, particularly those with overdetection rates of 0.9 to 1.0, the ROC curve for the first failure detection parameter shows a low overdetection rate. Therefore, the first failure detection parameter is considered more suitable for practical applications. However, in AUROC-based optimization, it is impossible to determine which of the first and second failure detection parameters is most suitable, and therefore it is sometimes impossible to obtain a failure detection parameter suitable for practical applications.

[0011] As mentioned above, in the existing methods, the evaluation values ​​used when optimizing the unqualified detection parameters do not take into account the high unqualified detection rate in areas with low over-detection rates and the low over-detection rate in areas with high unqualified detection rates. Therefore, the unqualified detection parameters cannot be optimized to be suitable for practical applications.

[0012] The present invention has been made in view of such a situation, and an object of the present invention is to optimize the failure detection parameters according to the over-detection rate or the failure detection rate. Technical means to solve the problem

[0013] In order to solve the above-mentioned problem, a computer system of one embodiment of the present invention is a computer system having one or more processors and one or more memory resources, wherein the one or more processors execute: an image acquisition step of acquiring an image of an object; and a parameter optimization step of optimizing parameters for detecting unqualified parts in the image of the object, wherein the parameter optimization step includes: an unqualified part acquisition step of acquiring unqualified part information representing the unqualified part in the image of the object; a first function acquisition step of acquiring a first function, which calculates an evaluation value, and the evaluation value evaluates the performance of detecting the unqualified part based on the over-detection rate and / or the unqualified detection rate; an unqualified detection step of detecting the unqualified part of the object using the image of the object and the parameters; an evaluation value calculation step of calculating the evaluation value taken by the first function for at least one of the over-detection rates and / or the unqualified detection rates obtained based on the detection results in the unqualified detection step and the unqualified part information; and a parameter updating step of updating the parameters in a manner such that the evaluation value becomes maximum. Effects of the Invention

[0014] According to the present invention, the unqualified detection parameters can be optimized according to the over-detection rate and the unqualified detection rate.

[0015] Other problems, structures, and effects than those described above will become clear from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a diagram showing an example of the overall processing procedure of the appearance inspection method according to the first embodiment. Figure 2 This is a diagram showing an example of a table showing the relationship between whether an object is actually a defective product (positive) or a conforming product (negative) and whether it is determined to be a defective product or a conforming product in the defect detection step of the first embodiment. Figure 3 This is a schematic diagram showing an example of the first function in the first embodiment. Figure 4 This is a schematic diagram of an example of a GUI (Graphical User Interface) for acquiring the first function in the first function acquisition step of the first embodiment. Figure 5 This is an illustration of an example of a neural network used in the first embodiment. Figure 6 This is a diagram showing an example of the processing procedure in the parameter optimization phase in the second embodiment. Figure 7 This is a diagram showing an example of the processing procedure of the image comparison step in the second embodiment. Figure 8 This is a diagram showing an example of the processing procedure of the appearance inspection method according to the third embodiment. Figure 9 This is a schematic diagram showing an example of a GUI for acquiring the second function in the second function acquisition step of the third embodiment. Figure 10 This is a schematic diagram showing an example of a GUI for classifying images of objects according to the level of lethality in the classification step S803 of the third embodiment. Figure 11 This is a diagram showing an example of the hardware configuration of a computer system that executes the appearance inspection methods according to the first to third embodiments. Figure 12A This is an example of a conventional ROC curve obtained using the first failure detection parameter. Figure 12B This is an example of a conventional ROC curve obtained using the second failure detection parameter. DETAILED DESCRIPTION

[0017] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In addition, in all the figures used to illustrate the embodiments, the same symbols are given to the same components in principle, and their repeated descriptions are appropriately omitted. In addition, in the following embodiments, except for the cases that are specifically stated and the cases that are clearly considered to be necessary in principle, the constituent elements (including element steps, etc.) are not necessarily necessary. In addition, when it is said that "consisting of A", "composed of A", "having A", "containing A", except for the cases that are specifically stated to be only the elements, etc., elements other than these are certainly not excluded. Similarly, in the following embodiments, when the shapes, positional relationships, etc. of the constituent elements, etc. are mentioned, except for the cases that are specifically stated and the cases that are clearly considered not to be such in principle, the cases that are substantially similar or similar to the shapes, etc. are included. In addition, the embodiments do not limit the present invention related to the claims, and all the elements and their combinations described in the embodiments are not necessary for the solution of the present invention.

[0018] <First embodiment> [Overall process sequence of appearance inspection method] use Figures 1 to 4 The appearance inspection method in the first embodiment will be described. Figure 1 This is a diagram showing an example of the overall processing procedure of the appearance inspection method according to the first embodiment.

[0019] like Figure 1 As shown, the appearance inspection method of this embodiment includes: an image acquisition step S102 for acquiring an image 103 of an object 101, and a parameter optimization step S104 for optimizing parameters (failure detection parameters 105) for detecting defective parts in the image 103 of the object. The parameter optimization step S104 includes: an unqualified part acquisition step S106, which acquires unqualified part information 107 of an unqualified part of an image 103 representing an object; a first function acquisition step S108, which acquires a first function w (109), wherein the first function w (109) calculates an evaluation value e (113), wherein the evaluation value e (113) evaluates the performance of detecting unqualified parts based on the over-detection rate x and / or the unqualified detection rate y; an unqualified detection step S110, which detects unqualified parts of the object using the image 103 of the object and the unqualified detection parameter 105; an evaluation value calculation step S112, which calculates the evaluation value e (113) taken by the first function w (109) for at least one over-detection rate x and / or unqualified detection rate y obtained based on the detection result 111 in the unqualified detection step S110 and the unqualified part information 107; and a parameter updating step S114, which updates the unqualified detection parameter 105 in a manner that maximizes the evaluation value e (113).

[0020] In the existing methods, the evaluation values ​​used when optimizing the unqualified detection parameters do not take into account the high unqualified detection rate in areas with low over-detection rates, or the low over-detection rate in areas with high unqualified detection rates. Therefore, the unqualified detection parameters cannot be optimized to be suitable for practical applications.

[0021] In contrast, if the evaluation values ​​used in optimizing the failure detection parameters can be calculated by prioritizing regions with low overdetection rates and regions with high failure detection rates, optimization can be performed to maximize the evaluation values, thereby potentially yielding failure detection parameters suitable for practical applications. Therefore, in this embodiment, a first function w(x, y) is used for the overdetection rate x and the failure detection rate y. The degree of emphasis is varied depending on the overdetection rate and the failure detection rate, thereby yielding failure detection parameters suitable for practical applications.

[0022] like Figure 1 As shown, the processing sequence in this embodiment is divided into a parameter optimization stage 121 and a checking stage 122.

[0023] In the parameter optimization stage 121, the object 101 is first photographed to obtain an optimization image 103 (step S102). The optimization image 103 can be obtained by capturing the surface or interior of the object to be learned as a digital image using an imaging device such as a CCD (Charge Coupled Device) camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, or an X-ray inspection device.

[0024] Next, the optimization image 103 is used to optimize the failure detection parameters 105 for detecting the failure portion in the image of the object (step S104). The details of step S104 will be described later.

[0025] In the inspection stage 122, similar to the parameter optimization stage 121, an actual object 115, such as a product, is photographed to obtain an inspection image 117 (step S116). Next, the inspection image 117 is inspected for defective areas using the defect detection parameters 105 optimized in step S104, resulting in an inspection result 119 (step S118). Finally, the inspector verifies the inspection image 117 with the defective areas detected (step S120). If any defect is detected, countermeasures are fed back to the manufacturing process.

[0026] The details of the defect detection parameter optimization step S104 will be described. In this step, defective location information 107 indicating defective locations in each of the multiple optimization images 103 is obtained in advance (step S106). Furthermore, the first function 109 is obtained (step S108). The details of step S108 will be described later.

[0027] The defective area information 107 is information indicating defective areas in the optimization image 103. For example, an inspector may visually check defective areas in the optimization image 103 to generate the defective area information 107. Alternatively, the optimization image 103 may be used as input, and defective areas may be automatically detected based on pre-set rules, with the detection results being used as the defective area information 107.

[0028] Then, using the defect detection parameters 105 , defective portions are detected from each of the plurality of optimization images 103 , and a detection result 111 is obtained for each of the plurality of optimization images 103 (step S110 ).

[0029] Furthermore, an evaluation value 113 is calculated based on the detection result 111, the defective portion information 107, and the first function 109 (step S112). The operation of updating the defective detection parameter 105 is repeated so that the evaluation value 113 is maximized (step S114), thereby optimizing the defective detection parameter 105. The details of step S112 will be described later.

[0030] If a defective portion is detected in the defect detection step S110 , the object is judged as a defective product. Otherwise (if no defective portion is detected), the object is judged as a qualified product, and the judgment result is used as the detection result 111 .

[0031] In the evaluation value calculation step S112, first, the over-detection rate x and the unqualified detection rate y are calculated based on the detection result 111 and the unqualified part information 107. Figure 2 Explain how to calculate the over-detection rate x and the unqualified detection rate y.

[0032] Figure 2 This is a diagram showing an example of table 201. Table 201 shows the relationship between whether an object is actually a defective product (positive) or a conforming product (negative) and whether it is determined to be a defective product or a conforming product in the defect detection step S110 of this embodiment. Whether an object is actually a defective product (positive) or a conforming product (negative) can be determined based on the defective part information 107.

[0033] Correctly judging a defective product as a defective product is called a true positive, incorrectly judging a qualified product as a defective product is called a false positive, incorrectly judging a defective product as a qualified product is called a false negative, and correctly judging a qualified product as a qualified product is called a true negative. Figure 2 In Table 201, TP represents the number of true positives, FP represents the number of false positives, FN represents the number of false negatives, and TN represents the number of true negatives.

[0034] The number of true positives TP is the number of optimization images 103 that indicate unacceptable parts in the unacceptable part information 107 and are indicated as unacceptable in the detection result 111 .

[0035] The number of false positives FP is the number of optimization images 103 that do not indicate a failed part in the failed part information 107 and are indicated as failed in the detection result 111 .

[0036] The number of false negatives FN is the number of optimization images 103 that have failed parts indicated in the failed part information 107 and are not indicated as failed in the detection result 111 .

[0037] The number of true negatives TN is the number of optimization images 103 that do not indicate a failed portion in the failed portion information 107 and are not indicated as failed in the detection result 111 .

[0038] The over-detection rate x calculated in the evaluation value calculation step S112 is the ratio of qualified products mistakenly identified as defective products, and is expressed as FP / (TN+FP). Furthermore, the unqualified detection rate y is the ratio of unqualified products correctly identified as defective products, and is expressed as TP / (TP+FN).

[0039] The value calculated in evaluation value calculation step S112 can be either the over-detection rate x or the unacceptable detection rate y. When only the over-detection rate x is calculated, the first function is w(x), and when only the unacceptable detection rate y is calculated, the first function is w(y). When both the over-detection rate x and the unacceptable detection rate y are calculated in evaluation value calculation step S112, the first function is w(x, y).

[0040] In addition, it is also possible to calculate index values ​​other than the over-detection rate x and the unqualified detection rate y. For example, the suitability rate (TP / (TP+FP)) indicating the proportion of objects judged as unqualified products that are actually unqualified products may be calculated instead of the over-detection rate x. In addition, the missed detection rate (FN / (TP+FN)), which is the proportion of unqualified products mistakenly judged as qualified products (missed detection of unqualified products), may be calculated instead of the unqualified detection rate y. In the following description, the method of calculating both the over-detection rate x and the unqualified detection rate y is used for explanation.

[0041] Returning to the description of evaluation value calculation step S112, after calculating the over-detection rate x and the unacceptable detection rate y, the evaluation value e is calculated using the over-detection rate x, the unacceptable detection rate y, and the first function w(x, y). The evaluation value e is a value that evaluates the performance of detecting unacceptable areas based on the over-detection rate x and / or the unacceptable detection rate y, and is calculated, for example, using equation (1).

[0042] [Formula 1] e=w(x,y)…(1)

[0043] Alternatively, in the defect detection step S110, different detection results may be output for the acceptance thresholds {T_i (i is a positive integer from 1 to N; N is the number of thresholds)} used to determine whether the inspected object is defective or conforming. However, {T_i} is a set of acceptance thresholds. Hereinafter, brackets "{}" are used to represent a set of elements.

[0044] In this case, in evaluation value calculation step S112, the over-detection rate {x_i} and the unqualified detection rate {y_i} are calculated for each threshold value based on the detection results for each threshold value, and the evaluation value e is calculated using the over-detection rate {x_i}, the unqualified detection rate {y_i}, and the first function w(x, y). In this case, the evaluation value e is calculated, for example, using formula (2).

[0045] [Formula 2]

[0046] Alternatively, the ROC curve can be plotted using the over-detection rate {x_i} and the unqualified detection rate {y_i} calculated for each pass / fail threshold, and the evaluation value e can be calculated by weighting the ROC curve with the first function w(x, y). In this case, the evaluation value e is calculated, for example, using formula (3).

[0047] [Formula 3]

[0048] Here, ROC(x) in equation (3) is a ROC curve (a function representing the failure detection rate relative to the over-detection rate x).

[0049] [1st function w] The first function w will be described in detail. In the present embodiment, the first function w has a larger value in a region where the over-detection rate x is lower and / or a region where the defective detection rate y is higher.

[0050] As described above, the first function w is used to calculate the evaluation value e, prioritizing areas with low overdetection rates x and areas with high failure detection rates y. While the first function w is an arbitrary function, by prioritizing areas with low overdetection rates x and high failure detection rates y and increasing the value in areas with low overdetection rates x and high failure detection rates y, it is possible to obtain failure detection parameters suitable for practical applications.

[0051] Figure 3 This is a schematic diagram showing an example of the first function w(x, y) (301) in this embodiment. Figure 3 In the figure, a white area 302 (an area with a low over-detection rate x and a high under-detection rate y) is an area with w(x, y) = 1.0, and a shaded area 303 is an area with w(x, y) = 0.0.

[0052] like Figure 3 As shown in the example of , when the first function w is used, which has a large value in the area where the over-detection rate x is low and the unqualified detection rate y is high, Figure 12A 、 12B When the evaluation value e is calculated using the formula (3) for the ROC curves of the first and second failure detection parameters, the evaluation value for the first failure detection parameter is 0.58, and the evaluation value for the second failure detection parameter is 0.45. This shows that the first failure detection parameter, which has a larger evaluation value, is more suitable for practical use.

[0053] [Setting screen for the first function w] A method of acquiring the first function w in the first function acquisition step S108 will be described. In this embodiment, in the first function acquisition step S108, a GUI (Graphical User Interface) for setting the first function w is displayed on the GUI device.

[0054] The first function w is basically any function, but is preferably a function whose value increases in a region with a low over-detection rate and a region with a high defective detection rate.

[0055] However, depending on the purpose of the inspection, the focus on areas with low over-inspection rates and high non-conformance rates may differ. For example, when the object is nearly finished, since defective products are not likely to be shipped, the focus is likely to be on areas with a non-conformance rate of 0.98 to 1.0. Furthermore, in the early stages of manufacturing, where the impact of missed defects is relatively small, the focus is likely to be on areas with an over-inspection rate of 0.0 to 0.03 in order to reduce the frequency of inspections by inspectors.

[0056] Therefore, in this embodiment, a GUI is displayed for setting the first function w. This allows the first function w to be changed according to the purpose of the inspection, and the evaluation value e can be calculated based on the set first function w to optimize the failure detection parameters.

[0057] Figure 4 This is a schematic diagram of an example of a GUI used to acquire the first function w in the first function acquisition step S108 of this embodiment. Examples of GUI devices that display this GUI include a display or projector. Furthermore, examples of input devices that allow users to perform input operations such as pressing buttons while viewing the GUI include keyboards, mice, and touch panels.

[0058] like Figure 4 As shown, the GUI has: buttons 401 and 402, which are used to select whether to select and set the first function w from a predetermined preset or to input a formula to set the first function w; area 403, which is used to select a preset; area 404, which is used to input a formula; area 405, which represents the value of the first function w; button 406, which is used to complete the setting of the first function w; and button 407, which cancels the setting of the first function w.

[0059] When button 401 for selecting from presets is clicked and a preset is selected in area 403, the value of the function displayed in area 403 is displayed in area 405. In area 405, areas with larger function values ​​are displayed in a color closer to white, and areas with smaller function values ​​are displayed in a color closer to black.

[0060] By clicking button 402 for inputting a formula and inputting a formula into area 404, the value of the function displayed in area 404 is displayed in area 405. The user can set the first function w according to the purpose of the inspection while confirming the value of the first function w displayed in area 405. The setting of the first function w can be completed by pressing button 406, and the setting of the first function w can be canceled by pressing button 407.

[0061] [Unqualified test parameters] The failure detection parameters 105 are described in detail. In this embodiment, the failure detection parameters 105 are hyperparameters related to the network structure of the failure location determination engine that determines a failure location from an input image and / or weight coefficients of the network of the failure location determination engine. Furthermore, in this embodiment, the failure detection parameters 105 are optimized using the image 103 of the object 101 and the failure location information 107 in the parameter optimization step S104.

[0062] In recent years, with the advent of deep network models, the performance of machine learning has dramatically improved. In machine learning-based defect detection, inspectors visually teach the defective areas of an object image beforehand, learning the relationship between the object image and this teaching result. However, even if the inspector performs the teaching correctly, if the network model structure or the network weighting coefficients are not sufficiently optimized, it is impossible to accurately identify defective areas in the object image.

[0063] Therefore, in this embodiment, hyperparameters related to the network structure or network weight coefficients in the defective part determination engine that determines defective parts from input images are optimized as defect detection parameters 105. This makes it possible to obtain a defective part determination engine suitable for actual use.

[0064] Figure 5 is a schematic diagram showing an example of a neural network used in this embodiment. As a defective part determination engine in this embodiment, a neural network having Figure 5 The three-layer neural network shown in Figure 1 is a 3-layer neural network. Here, Y represents the input image, F1(Y) and F2(Y) represent the intermediate data, and F(Y) represents the result of identifying the defective part. The intermediate data and the estimation results are calculated using the following equations (4) to (6).

[0065] [Formula 4] F1(Y)=max(0,U1*Y+B1)…(4)

[0066] [Formula 5] F2(Y)=max(0,U2*F1(Y)+B2)…(5)

[0067] [Formula 6] F(Y)=U3*F2(Y)+B3…(6)

[0068] Here, "*" represents a convolution operation. Here, U1 represents A1 filters of size A0×f1×f1, A0 represents the number of channels in the input image, and f1 represents the size of the spatial filter. By convolving the input image with the filter of size A0×f1×f1 A1 times, an A1-dimensional feature map is obtained. B1 is an A1-dimensional vector and is the bias component corresponding to the A1 filters. Similarly, U2 represents A2 filters of size A1×f2×f2, B2 is an A2-dimensional vector, and U3 represents A0 filters of size A2×f3×f3, B3 is an A0-dimensional vector. A0 mentioned above is a value determined by the number of channels in the input image. Furthermore, f1, f2, A1, and A2 are hyperparameters related to the network structure of the failed part determination engine, and U1, U2, U3, B1, B2, and B3 are the weight coefficients of the failed part determination engine network. These are the objects to be optimized as the failure detection parameters 105 in the parameter optimization step S104 of this embodiment.

[0069] As the network structure of the engine for determining the unqualified parts, in addition Figure 5 In addition to the structure shown, for example, the network structure described in the following paper (Ronneberger et al.) can also be used.

[0070] Olaf Ronneberger, Philipp Fischer, Thomas Brox, "U-Net: ConvolutionalNetworks for Biomedical Image Segmentation", arXiv preprintarXiv:1505.04597(2015)

[0071] The number of layers, the number and size of convolution filters, and the type of activation function in the network model are hyperparameters related to the network structure of the unqualified part determination engine, and the weight coefficients between nodes are the weight coefficients of the network of the unqualified part determination engine. These are the objects to be optimized as unqualified detection parameters 105 in the parameter optimization step S104 in this embodiment.

[0072] In the embodiment described above, Figure 1As shown, the process includes a parameter optimization phase 121 and an inspection phase 122. The same operator can perform the parameter optimization phase 121 and the inspection phase 122. In addition, the parameter optimization phase 121 and the inspection phase 122 can also be performed by different operators. In this case, for example, an operator performs the parameter optimization phase 121, thereby updating the unqualified detection parameters in a manner that the evaluation value e becomes the maximum. Moreover, an operator different from the operator who performed the parameter optimization phase 121 can also perform the inspection phase using the updated parameters. Thus, it is possible to use unqualified detection parameters that are optimized for actual use to perform actual inspections of objects 115 such as products.

[0073] <Second embodiment> use Figures 6 and 7 The visual inspection method of the second embodiment is described. In the first embodiment, a method for optimizing failure detection parameters using a first function for over-detection rate and / or failure detection rate is used to obtain failure detection parameters suitable for practical applications. In particular, a method for optimizing hyperparameters or network weighting coefficients related to the network structure in the failure area determination engine that determines failure areas from input images, which serve as failure detection parameters, is described.

[0074] In the second embodiment, a method is described in which a suspected qualified product image estimation engine is used to estimate a suspected qualified product image corresponding to an input image, a suspected qualified product image is estimated based on an image of an object, and parameters when comparing the image of the object and the suspected qualified product image are used as unqualified detection parameters for optimization.

[0075] In addition, in the processing sequence of the appearance inspection method in the second embodiment, the image acquisition step S102, the defective part acquisition step S106, and the first function acquisition step (S108) in the parameter optimization stage 121 and the inspection stage 122 are the same as those in the first embodiment. Figure 1 The same as in , so the description is omitted.

[0076] The visual inspection method in the second embodiment further performs a suspected conforming product image estimation engine learning step S602, which uses the optimized conforming product image 601 of the object to train a suspected conforming product image estimation engine 603 that estimates suspected conforming product images based on an input image. The defective product detection step S608 includes a suspected conforming product image estimation step S604, which uses the suspected conforming product image estimation engine 603 to estimate a suspected conforming product image 605 based on the object image 103; and an image comparison step S606, which detects defective portions of the object by comparing the object image 103 with the suspected conforming product image 605. The parameters optimized in the parameter optimization step S104 are the parameters used in the image comparison step S606.

[0077] In machine learning-based defect detection, it's preferable to prepare a large number of images of both good and bad products during learning. However, in industrial product manufacturing lines, collecting a small number of defective product images is relatively easy, but collecting a large number of defective product images is often difficult or time-consuming. Therefore, in this embodiment, learning is performed using only images of good products, and when optimizing defect detection parameters, evaluation values ​​are calculated using both images of good and bad products. This eliminates the need to collect a large number of defective product images and allows for optimization of defect detection parameters using only images of good products.

[0078] Figure 6 is a diagram showing an example of the processing sequence of the parameter optimization stage in this embodiment. Figure 6 In the middle, omit Figure 1 Same object 101, steps S102, S106, S108.

[0079] First, the suspected qualified product image estimation engine 603 is trained using a plurality of optimized qualified product images 601, which are images of qualified products, among the optimized images 103 (step S702). The details of the suspected qualified product image estimation engine 603 will be described later. When an image of an object is input into the suspected qualified product image estimation engine 603, the suspected qualified product image estimation engine 603 estimates and outputs a suspected qualified product image corresponding to the input image. The suspected qualified product image estimation engine 603 is not trained with images of unqualified products. Therefore, when an image of an unqualified product is input into the suspected qualified product image estimation engine 603, an image different from the input image of the unqualified product is output. Therefore, by comparing the input image and the output image, unqualified parts can be detected.

[0080] Next, the plurality of optimization images 103 are input to the suspected non-defective product image estimation engine 603 , and a suspected non-defective product image 605 corresponding to the optimization image 103 is estimated for each optimization image 103 (step S604 ).

[0081] Next, the optimized image 103 is compared with its corresponding suspected good product image 605 using the defect detection parameters 607 (step S606). By performing this comparison on all optimized images 103, defective areas are detected in each optimized image 103, and a detection result 111 indicating the defective areas is obtained for each optimized image 103 (step S608).

[0082] Next, similarly to the first embodiment, the over-detection rate x and the unqualified detection rate y are obtained from the unqualified portion information 107, the detection result 111, and the first function 109, and the evaluation value 113 is calculated from the over-detection rate x and the unqualified detection rate y (step S112).

[0083] Then, the operation of updating the failure detection parameter 607 so that the evaluation value 113 becomes the maximum is repeated (step S114), thereby optimizing the failure detection parameter.

[0084] use Figure 7 The image comparison step S606 will be described in detail. Figure 7 This is a diagram showing an example of the processing procedure of the image comparison step S606 in this embodiment.

[0085] In the image comparison step S606 , for example, a difference image 702 having a difference value for each pixel between the optimization image 103 and the suspected non-defective product image 605 estimated therefrom is calculated as a pixel value (step S701 ).

[0086] Next, a part (a candidate unacceptable part) 704 having a pixel value greater than a difference threshold 708 in the difference image 702 is detected (step S703 ), and an abnormality score 706 is calculated for the candidate unacceptable part 704 (step S705 ).

[0087] Next, when the abnormality score 706 is greater than the pass / fail determination threshold 709 , the object is determined to be a defective product, and a test result 111 indicating whether it is a pass / fail product is obtained (step S707 ).

[0088] The difference threshold 708 and the pass / fail threshold 709 are optimized in the parameter optimization step S104 in this embodiment as the failure detection parameter 607. As the abnormality score 706, for example, the sum of the pixel values ​​of the difference image in the failure candidate part 704 can be used.

[0089] The details of the suspected good product image estimation engine 603 are described below. As the suspected good product image estimation engine 603, for example, a Figure 5 The three-layer neural network shown or the network structure described in the above paper (Ronneberger et al.).

[0090] In the learning step S602 of the suspected qualified product image estimation engine, the optimized qualified product image 601 is used as input, and the weight coefficients between the nodes in the suspected qualified product image estimation engine are optimized in the same manner as the estimated image and the input image. The network structure described in the paper (Ronneberger et al.) is called an encoder-decoder model that temporarily compresses (encodes) the high-dimensional feature quantities of an image into low-dimensional feature quantities and then restores (decodes) them into high-dimensional feature quantities. By learning in a manner that the estimated image is consistent with the input qualified product image, the original qualified product image is restored when the qualified product image is input, and the unqualified part cannot be restored when the unqualified product image is input. Therefore, by comparing the input image and the estimated image, it can be expected that the difference in the unqualified part will become larger.

[0091] <Third embodiment> use Figures 8 to 10 A description will be given of an appearance inspection method according to a third embodiment. Figure 8 3 is a diagram showing an example of the processing sequence of the appearance inspection method according to the third embodiment. Figure 8 In the middle, omit Figure 1 Same object 101, steps S102, S106.

[0092] In the first and second embodiments, a method for obtaining a failure detection parameter suitable for practical application by optimizing the failure detection parameter using a first function for the over-detection rate and / or the failure detection rate is described. In the third embodiment, a method for obtaining a failure detection parameter suitable for practical application by optimizing the failure detection parameter using a second function for the over-detection rate and / or the failure detection rate for each type (category) of failure is described.

[0093] In addition, in the processing sequence of the appearance inspection method of the third embodiment, the image acquisition step S102 and the defective portion acquisition step S106 in the parameter optimization stage 121, the inspection stage 122 and the first embodiment are respectively Figure 1 The same, so the description is omitted.

[0094] In the appearance inspection method of the third embodiment, the parameter optimization step S104 includes: a classification step S803 of classifying the image 103 of the object according to each unqualified category c (c is a positive integer from 1 to M; M is the number of categories); and a second function acquisition step S801 of acquiring a second function v_c (802) which is an element of each unqualified category of the first function and is an element of calculating the evaluation value e (213) based on the over-detection rate x and / or the unqualified detection rate y. In the evaluation value calculation step S805, the evaluation value e (213) is calculated based on the element evaluation value d_c taken by the second function v_c (802) for at least one over-detection rate x and / or unqualified detection rate y obtained for each unqualified category c based on the detection result 111 in the unqualified detection step S110.

[0095] There are many different types of defects that occur in the manufacturing lines of industrial products. Since the impact of each type of defect on the object varies, the required pass-test rate and failure detection rate for each defect type are generally different. For example, defects such as omissions in machining where holes were not originally drilled have a significant negative impact on the performance of the object, so a very high failure detection rate (approximately 0.98 to 1.0) is required. On the other hand, fine scratches such as cutting marks often have little impact on the performance of the object, so a low pass-test rate is required.

[0096] However, in the past, when different items were prioritized depending on the type of failure, appropriate failure detection parameters could not be obtained. Therefore, in this embodiment, a second function for each failure type is used as an element of the first function in the first and second embodiments. This allows appropriate failure detection parameters to be obtained even when different items were prioritized depending on the type of failure.

[0097] In this embodiment, first, a second function v_c (802) is obtained for each failure category c (c is a positive integer from 1 to M; M is the number of categories) (step S801).

[0098] Next, the plurality of optimization images 103 are classified into each defective category ( S803 ), and a classification result 804 indicating the classification result is obtained.

[0099] Then, the defective detection parameters 105 are used to detect defective locations in the plurality of optimization images 103 , and the detection result 111 is obtained for each optimization image 103 (step S110 ).

[0100] Next, based on the detection result 111, the defective portion information 107, the classification result 804, and the second function v_c (802), the factor evaluation value d_c (807) is calculated for each defective category c (step S806). The details of step S806 will be described later.

[0101] Next, the evaluation value e(113) is calculated using the element evaluation value {d_c} (step S808), and the failure detection parameter 105 is updated so that the evaluation value e(113) becomes the maximum (step S114). Then, by repeating steps S806, S808, and S114, the failure detection parameter is optimized.

[0102] In the factor evaluation value calculation step S806, for each unqualified category c (c is a positive integer from 1 to M; M is the number of categories), based on the detection result 111 and the unqualified part information 107, the over-detection rate {x_(i, c) (i is a positive integer from 1 to N; N is the number of thresholds)} and the unqualified detection rate {y_(i, c)} are calculated, and the factor evaluation value d_c is calculated by replacing e with the formula d_c in formulas (1) to (3) of the first embodiment.

[0103] In addition, when using formula (2) to calculate the factor evaluation value d_c, x_i and y_i are replaced by x_(i, c) and y_(i, c), respectively. In addition, when using formula (3) to calculate the factor evaluation value d_c, the over-detection rate {x_(i, c)} and the unqualified detection rate {y_(i, c)} calculated for each unqualified category c and the pass / fail threshold T_i are used to draw the ROC curve for each unqualified category c. The ROC curve is weighted by the second function v_c(x, y) to calculate the factor evaluation value d_c.

[0104] [The importance of each unqualified category c p_c] The method of calculating the evaluation value using the element evaluation value in the evaluation value calculation step S805 will be described in detail. The evaluation value calculation step S805 in this embodiment calculates the evaluation value e (113) based on the importance p_c (810) and the element evaluation value d_c of each failure category c.

[0105] As mentioned above, defects that occur on industrial product production lines vary widely. In practice, each defect type has a different level of importance, so it's important to prioritize detection and response for more important defects over less important ones. For example, by prioritizing response to frequently occurring defects over less frequently occurring ones, the defect rate can be reduced early.

[0106] However, prioritizing detection of highly important failure categories has been difficult. Therefore, in this embodiment, the importance p_c of each failure category c is used to weight the element evaluation value d_c to calculate the evaluation value. This increases the contribution of highly important failure categories to the evaluation value calculation, ultimately enabling prioritized detection of highly important failure categories.

[0107] In the parameter optimization phase 121 of this embodiment, the importance p_c (810) of each failure category c is obtained (step S809), and the evaluation value e is calculated using the importance {p_c} and the element evaluation value {d_c} (step S808). The evaluation value e is calculated, for example, using equation (7).

[0108] [Formula 7]

[0109] The importance p_c may use a value predetermined for each failure category, or may be determined by user input using a GUI or the like.

[0110] [Setting screen for the second function v_c for each failure category c] Figure 9 This is a schematic diagram illustrating an example of a GUI used to acquire the second function in the second function acquisition step S801 of this embodiment. Examples of GUI devices that display this GUI include a display or a projector. Furthermore, examples of input devices that allow a user to perform operations such as pressing buttons while viewing the GUI include a keyboard, a mouse, and a touch panel.

[0111] The GUI has: an area 901 for selecting a non-conforming category; buttons 902 and 903 for selecting, for the non-conforming category selected in area 901, whether to select and set the second function from a predetermined preset or to enter a formula to set the second function; an area 904 for selecting a preset; an area 905 for entering a formula; an area 906 for indicating the value of the second function; a button 907 for completing the setting of the second function; and a button 908 for canceling the setting of the second function.

[0112] When a failure category is selected in area 901, a second function for the selected failure category can be set. By clicking button 902 for selecting from presets and selecting a preset in area 904, the value of the function displayed in area 904 is displayed in area 906. In area 906, areas with larger function values ​​are displayed in a color closer to white, while areas with smaller function values ​​are displayed in a color closer to black.

[0113] By clicking button 903 for inputting a formula and entering a formula in area 905, the value of the function displayed in area 905 is displayed in area 906. The user can set the second function according to the failure type while confirming the value of the second function displayed in area 906. The setting of the second function can be completed by pressing button 907, and can be canceled by pressing button 908.

[0114] In addition, Figure 9 In the example, the second function is represented by the label “v”, but the second function may be represented by a label “v_c” that is different for each category c as described above.

[0115] [Unqualified category] The following describes the failure categories classified in the classification step: The classification step S803 in this embodiment classifies the image 103 of the object into different failure categories according to the degree of fatality caused by the failure.

[0116] As mentioned above, a wide variety of defects can occur on industrial product manufacturing lines. In practice, when such defects occur, those that have a significant negative impact on the performance of the production line or the object are treated as highly critical defects. For these critical defects, a high failure detection rate (approximately 0.98 to 1.0) is required. On the other hand, for less critical defects, a low over-detection rate is required.

[0117] However, existing visual inspection methods cannot generate suitable failure detection parameters for practical applications when a mixture of highly critical failures and less critical failures exists. Therefore, in this embodiment, object images are classified into different failure categories based on their criticality, and a second function, specific to each failure category, is used to calculate evaluation values ​​and optimize failure detection parameters. This allows, for example, to generate failure detection parameters that prioritize a high failure detection rate for highly critical failures and a low overdetection rate for less critical failures.

[0118] Figure 10 This is a schematic diagram showing an example of a GUI for classifying the image of the object according to the degree of lethality in the classification step S803 of this embodiment.

[0119] The GUI has areas 1005 to 1008 for classifying each of the images 1001 to 1004 of the object into a non-conforming category, a button 1012 for completing the classification, and a button 1013 for canceling the classification.

[0120] exist Figure 10In the example shown in FIG1 , image 1001 includes defective defect 1009, image 1002 includes damaged defect 1010, and image 1014 includes foreign matter defect 1011. In areas 1005 to 1008, each image of the object is classified by selecting a defective category. Classification of the defective category is terminated by pressing button 1012, and classification is canceled by pressing button 1013.

[0121] exist Figure 10 In this example, image 1001 is classified as a fatal failure, indicating a failure with a high degree of fatality, and image 1002 is classified as a non-fatal failure, indicating a failure with a low degree of fatality. Furthermore, image 1003 is classified as a conforming product, indicating that the object is not a defective product, and image 1004 is classified as unclassified, indicating that the object is not classified into any defective category or conforming product.

[0122] exist Figure 10 In the example described above, two failure categories, fatal and non-fatal, are used as failure categories. However, it is also possible to classify failures based on failure categories other than these, such as failure size. Furthermore, classification into three or more failure categories is also possible. While the example describes classification of an image of an object using a GUI, a classification engine that automatically classifies input images using, for example, a convolutional neural network can also be used. Furthermore, it is also possible to automatically classify an image of an object based on pre-set rules.

[0123] [Computer System] Figure 11 This is a diagram showing an example of the hardware configuration of a computer system that executes the appearance inspection methods according to the first to third embodiments.

[0124] The computer system is composed of an imaging device 1101 and a computer 1102. As described in the first embodiment, examples of the imaging device 1101 include a CCD camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, and an X-ray inspection device.

[0125] The computer 1102 is a component for processing the appearance inspection method described in this embodiment, and has the following contents.

[0126] Processor 1103: Examples of processor 1103 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). However, other components are also acceptable as long as they can handle the external tube inspection method. Computer 1102 may include one or more processors 1103.

[0127] Memory resource 1104: Examples of memory resource 1104 include RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), and non-volatile memory (such as flash memory). These memory resources may also store a program (referred to as a visual inspection program) that causes the processor to execute the visual inspection methods described in the first through third embodiments. Computer 1102 may include one or more memory resources 1104.

[0128] GUI device 1105: As described above, examples of the GUI device 1105 include a display, a projector, and the like. However, any other device may be used as long as it can display a GUI.

[0129] Input device 1106: As mentioned above, examples of input device 1106 include a keyboard, a mouse, and a touch panel. However, any other device that can accept user operations may also be used. Furthermore, input device 1106 and GUI device 1105 may be integrated.

[0130] Communication interface 1107: Examples of communication interface 1107 include USB (Universal Serial Bus), ETHERNET (registered trademark), and Wi-Fi. Any other interface device may be used as long as it can directly receive images from camera 1101 or allow a user to send the images to computer 1102. Furthermore, a removable non-volatile storage medium (e.g., a flash memory, DVD (Digital Versatile Disc), CD (Compact Disk)-ROM, Blu-ray Disc, etc.) storing the images may be connected to communication interface 1107 to store the images in computer 1102.

[0131] The above is the hardware configuration of the computer. In addition, there may be multiple computers 1102 and multiple imaging devices 1101 constituting the computer system.

[0132] In addition, the above-mentioned appearance inspection program can also be stored in the computer 1102 through the following path.

[0133] The appearance inspection program is stored in a nonvolatile removable nonvolatile storage medium, and the medium is connected to the communication interface 1107 , thereby distributing the program to the computer 1102 .

[0134] The program distribution server distributes the appearance inspection program to the computer 1102. The program distribution server includes a memory resource storing the appearance inspection program, a processor for performing distribution processing for distributing the appearance inspection program, and a communication interface capable of communicating with the communication interface of the computer.

[0135] As described above, the embodiments described so far do not limit the scope of the claims, and not all elements and combinations thereof described in the embodiments are essential to the solution of the present invention.

[0136] For example, the various components, functions, processing units, and components of the computer system described above may be partially or entirely implemented by the processor 1103 executing a visual inspection program. However, they may also be implemented in hardware by designing them as integrated circuits. Furthermore, the computer system may be implemented by a user (operator) executing some or all of the functions or processes implemented by the programs.

[0137] Furthermore, a computer system may not include a GUI device 1105 and instead delegate some of the processing of user output and input to a processor system external to the system, such as a smartphone or tablet (referred to as an external processor system). In such cases, the computer system (or processor 1103, appearance inspection program) may also perform the following processing in order to execute the processing described above or other parts of the program.

[0138] Instead of outputting data to the user using the GUI device 1105 described above, data required for outputting data to the user is transmitted to the external processor system via the communication interface 1107. Examples of this data include the output data itself and data for generating output data in another processor system. Alternatively, this data may be a program or web data describing the processing for performing user output in the external processor system.

[0139] Data indicating user input or operation is received from an external processor system via the communication interface 1107 instead of receiving input or operation from the user using the input device 1106 described above.

[0140] From another perspective, outputting data to a user may include, in addition to actions performed by the computer system itself, causing another entity outside the computer system to output (use) the data. Furthermore, receiving input or an operation from a user may include, in addition to direct input by the user into the computer system's input device 1106, receiving input indirectly by the computer system. Explanation of symbols

[0141] 101, 115…Object, 103…Optimized image, 105…Failure detection parameters, 107…Failure location information, 109…First function, 111, 119…Detection results, 113…Evaluation value, 117…Inspection image, 121…Parameter optimization phase, 122…Inspection phase, 302, 303, 403–405, 901, 904–906, 1005–1008…Regions, 401, 402, 406, 407, 902, 903, 907, 908, 1012, 1013…Button, 601…Conformed product image, 601…Conformed product image for optimization, 603…Suspected conforming product image estimation engine, 605 …Suspected qualified product image, 607…Unqualified product detection parameters, 702…Difference image, 704…Unqualified candidate parts, 706…Abnormality score, 708…Difference threshold, 709…Qualified / unqualified threshold, 804…Classification result, 1001–1004, 1014…Image, 1009…Defective unqualified, 1010…Damage unqualified, 1011…Foreign matter unqualified, 1101…Camera device, 1102…Computer, 1103…Processor, 1104…Memory resources, 1105…GUI device, 1106…Input device, 1107…Communication interface, 1201, 1203…ROC curve, 1202, 1204…Region.

Claims

1. A computer system having one or more processors and one or more memory resources, The computer system is characterized in that The one or more processors execute: an image acquisition step of acquiring an image of the object; and a parameter optimization step of optimizing parameters for detecting unqualified parts in the image of the object, The parameter optimization step comprises: a defective portion acquiring step of acquiring defective portion information representing the defective portion in the image of the object; a first function acquisition step of acquiring a first function, wherein the first function calculates an evaluation value, wherein the evaluation value evaluates the performance of detecting the unqualified portion based on an over-detection rate and / or an unqualified detection rate; a non-conforming detection step of detecting the non-conforming portion of the object using the image and the parameters of the object; an evaluation value calculation step of calculating the evaluation value obtained by the first function for at least one of the over-detection rate and / or the unqualified detection rate obtained based on the detection result in the unqualified detection step and the unqualified part information; as well as The parameter updating step updates the parameter so that the evaluation value becomes maximum.

2. The computer system according to claim 1, wherein: The region with a lower over-detection rate and / or the region with a higher unqualified detection rate has a larger evaluation value obtained according to the first function.

3. The computer system according to claim 1, wherein: In the first function acquisition step, the one or more processors display a GUI for setting the first function on a GUI device, where the GUI is a graphical user interface.

4. The computer system according to claim 1, wherein: The parameters are hyperparameters related to the network structure of the defective part determination engine for determining defective parts from an input image and / or weight coefficients of the network of the defective part determination engine, In the parameter optimization step, the one or more processors optimize the parameters using the image of the object and the defective portion information.

5. The computer system according to claim 1, wherein: The one or more processors further execute a suspected qualified product image estimation engine learning step, wherein the suspected qualified product image estimation engine learning step uses the qualified product image of the object to enable the suspected qualified product image estimation engine to learn the suspected qualified product image based on the input image. The unqualified detection step comprises: a suspected qualified product image estimating step of using the suspected qualified product image estimating engine to estimate the suspected qualified product image based on the image of the object; as well as an image comparison step of detecting the unqualified portion of the object by comparing the image of the object with the image of the suspected qualified product; The parameters optimized in the parameter optimization step are the parameters used in the image comparison step.

6. The computer system according to claim 1, wherein: The parameter optimization step comprises: a classification step of classifying the image of the object into each unqualified category; and The second function acquisition step is to acquire the second function, which is an element of each of the unqualified categories of the first function, and is the second function for calculating the element of the evaluation value based on the over-detection rate and / or the unqualified detection rate, i.e., the element evaluation value. In the evaluation value calculation step, the one or more processors calculate the evaluation value based on the factor evaluation value taken by the second function for at least one of the over-detection rates and / or the unqualified detection rates obtained for each unqualified category based on the detection results in the unqualified detection step.

7. The computer system according to claim 6, wherein: In the evaluation value calculation step, the one or more processors calculate the evaluation value based on the importance of each failure category and the element evaluation value.

8. The computer system according to claim 6, wherein: In the classification step, the one or more processors classify the image of the object into different failure categories according to the degree of fatality caused by failure.

9. The computer system according to claim 1, wherein: The one or more processors further execute the step of detecting defective portions in the image of the product using the updated parameters.

10. A visual inspection method, which is performed by a computer system having one or more processors and one or more memory resources, The appearance inspection method is characterized by comprising: an image acquisition step of acquiring an image of the object; and a parameter optimization step of optimizing parameters for detecting unqualified parts in the image of the object, The parameter optimization step comprises: a defective portion acquiring step of acquiring defective portion information representing the defective portion in the image of the object; a first function acquisition step of acquiring a first function, wherein the first function calculates an evaluation value, wherein the evaluation value evaluates the performance of detecting the unqualified portion based on an over-detection rate and / or an unqualified detection rate; a non-conforming detection step of detecting the non-conforming portion of the object using the image and the parameters of the object; an evaluation value calculation step of calculating the evaluation value obtained by the first function for at least one of the over-detection rate and / or the unqualified detection rate obtained based on the detection result in the unqualified detection step and the unqualified part information; as well as The parameter updating step updates the parameter so that the evaluation value becomes maximum.

11. The appearance inspection method according to claim 10, characterized in that: The region with a lower over-detection rate and / or the region with a higher unqualified detection rate has a larger evaluation value obtained according to the first function.

12. The appearance inspection method according to claim 10, characterized in that: In the first function acquisition step, a GUI for setting the first function is displayed on a GUI device.

13. The appearance inspection method according to claim 10, characterized in that: The parameters are hyperparameters related to the network structure of the defective part determination engine for determining defective parts from an input image and / or weight coefficients of the network of the defective part determination engine, In the parameter optimization step, the parameters are optimized using the image of the object and the defective portion information.

14. The appearance inspection method according to claim 10, characterized in that: The method further comprises a suspected qualified product image estimation engine learning step, wherein the suspected qualified product image estimation engine learning step uses the qualified product image of the object to enable the suspected qualified product image estimation engine to learn the suspected qualified product image based on the input image. The unqualified detection step comprises: a suspected qualified product image estimating step of using the suspected qualified product image estimating engine to estimate the suspected qualified product image based on the image of the object; as well as an image comparison step of detecting the unqualified portion of the object by comparing the image of the object with the image of the suspected qualified product; The parameters are parameters used in the image comparison step.

15. A visual inspection program, characterized in that: A computer system is caused to execute the appearance inspection method according to any one of claims 10 to 14.

Images