Inspection condition determination system

Abstract

Inspection conditions in an inspection system can be appropriately determined. A computer (200), which is provided with a processor (202) and determines an inspection condition for an inspection system for an inspection object, the inspection accuracy, the false alarm rate, the inspection time, the inspection reproducibility, the learning time, the device cost, the operation cost, the device robustness, the inspection versatility, the service life, the ease of assembly, the required assembly accuracy, the robustness with respect to the environment, the ease of maintenance, and the definition of the determination basis for the identification processing are accepted for the inspection in the inspection system. At least two or more element evaluation indexes in the ease of recognition of a detection site and a user request relating to a target of the element evaluation indexes are input, and an evaluation index based on a plurality of element evaluation indexes relating to an inspection in an inspection system is generated on the basis of the user request. An evaluation based on the evaluation index is performed on the inspection results under the plurality of inspection conditions, and an appropriate inspection condition is determined from the plurality of inspection conditions.

Description

Technical Field

[0001] The present invention relates to a technology for determining inspection conditions in an inspection system for inspecting an inspection object. Background Art

[0002] For example, there is known an inspection system that performs appearance inspection on products or the like manufactured in a manufacturing process.

[0003] As a method for setting inspection conditions in an inspection system, patent document 1 discloses the following method for setting lighting conditions when inspecting an object, which method is characterized in that it includes: illuminating the object using a light source that can change the lighting parameters of the lighting conditions when specifying the object to be photographed, photographing the object using an image sensor with multiple lighting parameters, and acquiring photographed images corresponding to the multiple lighting parameters, wherein the object has known label data; generating an inferred image of the object when the object is illuminated with variable lighting parameters based on an image data set obtained by associating the photographed images with the lighting parameters corresponding to the photographed images; and applying the inferred image of the object corresponding to the variable lighting parameters and the corresponding label data to the learning of the machine learning model, and based on the comparison result of the inference result of the machine learning model with the label data of the object, simultaneously optimizing both the lighting parameters and the inspection algorithm parameters, thereby setting both the lighting conditions and the inspection algorithm parameters of the machine learning model.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: International Publication No. 2019 / 171121 Summary of the invention

[0007] In the technology described in Patent Document 1, only the inspection accuracy is considered, and both the lighting parameters and the parameters of the inspection algorithm can be optimized simultaneously.

[0008] However, in an inspection system, it is not always preferable to determine conditions by considering only the inspection accuracy rate.

[0009] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technology capable of appropriately determining inspection conditions in an inspection system.

[0010] In order to achieve the above-mentioned purpose, an inspection condition determination system of a scheme is an inspection condition determination system that has a processor and determines the inspection conditions in the inspection of an inspection system for a specified inspection object. The above-mentioned processor accepts input of at least two or more factor evaluation indicators of the inspection in the above-mentioned inspection system, including inspection accuracy, false alarm rate, inspection time, inspection reproducibility, learning time, device cost, operating cost, device robustness, inspection versatility, life, ease of assembly, required assembly accuracy, robustness to the environment, ease of maintenance, clarity of judgment basis for identification processing, and ease of identification of detection parts, as well as user requirements related to the targets of the above-mentioned factor evaluation indicators. Based on the above-mentioned user requirements, an evaluation indicator based on multiple factor evaluation indicators for the inspection in the above-mentioned inspection system is generated, the inspection results under multiple inspection conditions are evaluated based on the above-mentioned evaluation indicators, and appropriate inspection conditions are determined from multiple inspection conditions.

[0011] Effects of the Invention

[0012] According to the present invention, it is possible to appropriately determine the inspection conditions in the inspection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is an overall structural diagram of an inspection system of an implementation method.

[0014] Figure 2 This is a block diagram of a computer for determining inspection conditions according to one embodiment.

[0015] Figure 3 This is a diagram for explaining inspection conditions according to one embodiment.

[0016] Figure 4 This is a flowchart of the first inspection condition optimization process according to one embodiment.

[0017] Figure 5 This is a flowchart of the second inspection condition optimization process according to one embodiment.

[0018] Figure 6 is a flow chart of an optical model calibration process according to one embodiment.

[0019] Figure 7 This is a diagram showing a GUI screen according to one embodiment.

[0020] Figure 8 This is a diagram for explaining inspection conditions when the inspection area is divided according to one embodiment.

[0021] Fig. 9 It is a diagram for explaining changes in an inspection image according to one embodiment.

[0022] Fig.10This is a flowchart of an inspection condition redetermination process according to an embodiment.

[0023] Fig.11 It is a diagram for explaining the manufacturing process of one embodiment. DETAILED DESCRIPTION

[0024] The embodiments will be described with reference to the drawings. The embodiments described below do not limit the technical solutions of the claims, and each element described in the embodiments and all combinations thereof are not necessarily essential to the solution of the invention.

[0025] Figure 1 It is an overall configuration diagram of the inspection system according to the first embodiment.

[0026] Inspection system 100 includes lighting 110 , imaging sensor 120 , manipulation mechanism 140 , and computer 150 . Lighting 110 and computer 150 are connected via communication line 111 , imaging sensor 120 and computer 150 are connected via communication line 121 , and manipulation mechanism 140 and computer 150 are connected via communication line 141 .

[0027] The illumination 110 irradiates inspection light. When the inspection object 130 is irradiated with light by the illumination 110 , defects existing in the inspection object 130 can be made visible.

[0028] The imaging sensor 120 is a two-dimensional camera or the like that captures an image (digital image) of the inspection object 130 . The imaging sensor 120 can capture an image of a defect existing in the inspection object 130 .

[0029] The manipulation mechanism 140 changes the relative positions of the illumination 110 and the imaging sensor 120 and the inspection object 130. The manipulation mechanism 140 can move at least one of the illumination 110 and the imaging sensor 120 and the inspection object 130.

[0030] The computer 150 includes a memory 151 , a processor 152 , an input device 153 , and a display device 154 .

[0031] The memory 151 is a storage device for storing various information. The memory 151 may be a non-volatile or volatile storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory). In addition, the memory 151 may also be a rewritable storage medium such as a flash memory, a hard disk or an SSD (Solid State Drive), or may be a USB (Universal Serial Bus) memory, a memory card, etc.

[0032] The memory 151 stores a recognition program 161 and a control program 162. The recognition program 161 is executed by the processor 152 to perform a process (recognition process) for identifying a defect based on the image (digital data) captured by the imaging sensor 120. In this embodiment, the recognition program 161 can change at least a part of the recognition engine that executes the recognition process and the parameters used in the recognition process. The control program 162 is executed by the processor 152 to control the lighting 110, the imaging sensor 120, and the operating mechanism 140 according to the inspection conditions.

[0033] The processor 152 performs various processes according to the program stored in the memory 151. The processor 152 may be, for example, a microprocessor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a quantum processor, or other semiconductor devices capable of performing calculations.

[0034] The input device 153 is, for example, a mouse or a keyboard, and receives information input by the user. The display device 154 is, for example, a display, and displays and outputs a user interface including various information.

[0035] In the inspection system 100, as inspection conditions, at least a part of conditions related to illumination by the illumination 110 (illumination conditions), conditions related to imaging by the imaging sensor 120 (photography conditions), and conditions related to actions by the manipulation mechanism 140 (manipulation conditions) can be changed. The illumination conditions include, for example, at least one of the type of illumination, the configuration of illumination, the intensity of illumination, the illumination time, and the lighting cycle. The imaging conditions include at least one of the type of imaging sensor, the configuration of imaging sensor, the aperture amount, the imaging time, and the imaging cycle. The manipulation conditions include at least one of the type of manipulation mechanism, the position and posture, and the trajectory to the target position and posture.

[0036] Next, the computer 200 that performs processing for determining optimal inspection conditions in the inspection system 100 will be described.

[0037] Figure 2 This is a diagram showing the configuration of a computer for determining inspection conditions according to the first embodiment.

[0038] The computer 200 is an example of an inspection condition determination system, and includes a memory 201, a processor 202, an input device 203, and a display device 204. The computer 200 is connected to the knowledge database 220 via a communication path. The computer 200 of this embodiment can acquire inspection images from the computer 150, and can also send inspection conditions to the computer 150.

[0039] The memory 201 is a storage device for storing various information. The memory 201 may be a non-volatile or volatile storage medium such as a RAM or a ROM. In addition, the memory 201 may also be a rewritable storage medium such as a flash memory, a hard disk, or an SSD, or may be a USB memory, a memory card, or the like.

[0040] The memory 201 stores a recognition program 211 , an inspection condition optimization program 212 , an optical simulation program 213 , an optical model correction program 214 , an inspection image change detection program 215 , and a plurality of inspection condition optimization programs 216 .

[0041] The recognition program 211 is executed by the processor 202 to perform a process for identifying defects based on the inspection image. The inspection condition optimization program 212 is executed by the processor 202 to accept a request for inspection (customer request: user request) from a user (customer) and perform a process for detecting the best inspection conditions that meet the customer's request (inspection condition optimization process). The optical simulation program 213 is executed by the processor 202 to perform a simulation of the inspection using an optical model that can reproduce the inspection state of the inspection object in the inspection system 100. The optical model correction program 214 is executed by the processor 202 to perform a process for correcting the optical model (optical model correction process). The inspection image change detection program 215 is executed by the processor 202 to perform a process for detecting changes in the inspection image. The multiple inspection condition optimization programs 216 are executed by the processor 202 to determine the best inspection conditions in the manufacturing process using multiple inspection systems.

[0042] The processor 202 performs various processes according to the program stored in the memory 201. The processor 202 may be, for example, a microprocessor, a CPU, a GPU, an FPGA, a quantum processor, or other semiconductor devices capable of performing calculations.

[0043] The input device 203 is, for example, a mouse or a keyboard, and receives information input by the user. The display device 204 is, for example, a display, and displays and outputs a user interface including various information.

[0044] The knowledge database 220 manages information on cases inspected in the inspection system 100, such as images of inspection objects (inspection images) and information on inspection conditions at the time. In addition, although the knowledge database 220 is provided outside the computer 200, the present invention is not limited thereto, and the knowledge database 220 may also be provided inside the computer 200.

[0045] Next, customer requests accepted by the inspection condition optimization program 212 will be described.

[0046] Customer requirements include multiple evaluation indicators such as inspection accuracy f1, false alarm rate f2, inspection time f3, inspection reproducibility f4, learning time f5, device cost f6, operating cost f7, device robustness f8, inspection versatility f9, life span f10, ease of assembly f11, required assembly accuracy f12, robustness to the environment f13, ease of maintenance f14, clarity of judgment basis for identification processing f15 and ease of identification of the inspection part f16.

[0047] The inspection accuracy f1 is the ratio of qualified products and unqualified products correctly identified relative to the number of inspection objects. Here, if the number of qualified products identified as qualified products is set to TN, the number of unqualified products identified as unqualified products is set to TP, the number of qualified products mistakenly identified as unqualified products is set to FP, and the number of unqualified products mistakenly identified as qualified products is set to FN, the inspection accuracy f1 can be expressed as the following formula (1).

[0048]

Number 1

[0049]

[0050] The false alarm rate f2 is the ratio of the number of conforming products that are erroneously identified as defective products relative to the total number of conforming products. The false alarm rate f2 can be expressed as the following formula (2).

[0051]

Number 2

[0052]

[0053] Here, although the inspection accuracy rate f1 and the false alarm rate f2 are cited as examples of indicators related to the inspection accuracy, the present invention is not limited to these, and other indicators using some of TN, TP, FP, and FN may be used.

[0054] The inspection time f3 is the time involved in the inspection, for example, the time required for inspecting an inspection object, which is the sum of the movement time of the operating mechanism 140, the shooting time of the shooting sensor 120, the data transmission time, and the time for the identification processing program to determine defects.

[0055] The inspection reproducibility f4 is the reproducibility of the inspection when the same inspection object is inspected repeatedly.

[0056] The learning time f5 is the time required for learning when a learning inspection algorithm is used as the recognition program 161 , and is 0 when a non-learning inspection algorithm is used for the recognition program 161 , for example, a method of detecting defects by comparing the feature value of an image with a threshold value.

[0057] The equipment cost f6 is the cost involved in constructing the inspection system 100 .

[0058] The operating cost f7 is a cost incurred for operating the inspection system 100. The operating cost f7 includes electricity costs and maintenance costs of the inspection system 100.

[0059] The device robustness f8 is an index indicating the difficulty of occurrence of a failure in the inspection system 100 .

[0060] The inspection versatility f9 is an index indicating whether the inspection system 100 is applicable to various inspection objects.

[0061] The life f10 is the life of the inspection system 100 .

[0062] The ease of assembly f11 is an index indicating the ease of assembly of the inspection system 100 .

[0063] The required assembly accuracy f12 is an index indicating the assembly accuracy required to achieve inspection accuracy, etc. The higher the required assembly accuracy, the more errors caused by assembly will adversely affect inspection accuracy, etc., and the higher the device assembly cost will be.

[0064] The robustness f13 to the environment is an index indicating whether the inspection result will not change due to external factors, temperature and humidity changes, vibrations, and other environments, and whether the inspection system 100 will not fail.

[0065] The ease of maintenance f14 is an index indicating the ease of cleaning and adjustment of the inspection system 100 .

[0066] The clarity f15 of the basis for the recognition process is an indicator indicating whether the recognition program 161 of the inspection system 100 is a recognition program that can explain the basis for the recognition process. In general, recognition programs using deep learning often lack basis for the recognition process and have low clarity.

[0067] The recognition ease f16 of the detected part is an indicator indicating whether the recognition program 161 can recognize which part of the image has a defect when a defect is detected. In addition, depending on the type of recognition engine of the recognition program 161, it may not be possible to recognize which part of the image has a defect, in which case the recognition ease is reduced.

[0068] The target value of the customer index (fi (i=1 to 16)) is displayed, for example, on the GUI screen 900 (see Figure 7 ) is inputted from the user, but the target value of the factor evaluation index of a part of the customer index includes a qualitative index. Therefore, in this embodiment, the received customer index (fi (i = 1 to 16)) is quantified and converted into a comparative customer index (gi (i = 1 to 16)) that can be compared.

[0069] Then, based on the comparative customer index (gi (i=1 to 16)), an index Ftarget including a plurality of factor evaluation indexes as shown in equation (3) is generated and used in the optimization process.

[0070]

Number 3

[0071]

[0072] Here, wi is a weight corresponding to each element evaluation index gi. An input of wi can be accepted through the GUI screen 900, and resetting of wi can also be accepted through the GUI screen 900.

[0073] Figure 3 This is a diagram for explaining the inspection conditions of the first embodiment.

[0074] The inspection conditions of the inspection system 100 include conditions related to lighting (illumination conditions), conditions related to photographing (photographing conditions), conditions related to manipulation (manipulation conditions), and conditions related to recognition (recognition conditions).

[0075] Lighting conditions include lighting type, lighting configuration, lighting intensity, lighting time, lighting cycle, etc. Shooting conditions include shooting sensor type, shooting sensor configuration, aperture amount, shooting time, shooting cycle, etc. Manipulation conditions include manipulation mechanism type, target position and posture, path to reach target position and posture, etc. Recognition conditions include recognition engine type, recognition processing parameters, etc.

[0076] In this embodiment, each condition is Figure 3The symbols shown are represented. Here, T represents the type of object that is the object of the condition. P represents the three-dimensional position and posture of the object. P corresponds to, for example, a vector (x, y, z, i, j, k) that combines (x, y, z) representing the three-dimensional position and a unit vector (i, j, k) representing the posture. S represents a period associated with the object. H represents time associated with the object. I represents the illumination intensity (e.g., watts), A represents the aperture amount of the shooting sensor (the intensity of the aperture, such as a value of 1 to 0), K represents the trajectory of the operating mechanism 140 (e.g., a set of vectors (x, y, z, i, j, k) of the three-dimensional coordinates and posture of each position in the trajectory), and M represents a parameter of the recognition processing (a set of parameter values).

[0077] The superscripts (1) to (4) on the upper right of the letters represent values ​​related to lighting, shooting, manipulation, and recognition, respectively. In addition, the subscript i on the lower right of the letters represents the number of objects of the same type configured. In addition, regarding K, there are i and j as the subscripts on the lower right, i represents the starting point of the trajectory, and j represents the end point of the trajectory. In addition, regarding T and M related to recognition, there are i and l as the subscripts on the lower right, i represents the part of the image captured, and l represents the number of recognition processing performed on the image.

[0078] In the inspection condition optimization process performed by the inspection condition optimization program 212, a combination of appropriate inspection conditions (T, P, S, H, I, A, K, M) is calculated and output based on the index Ftarget. Here, the appropriate inspection condition is determined according to the adopted index, for example, an inspection condition in which the value of the index Ftarget satisfies one of the minimum, maximum, minimum, maximum, a predetermined value, a maximum value with a gentle slope (stable), and a minimum value with a gentle slope.

[0079] Next, the processing operation of the inspection condition optimization process performed by the computer 200 will be described.

[0080] First, an inspection condition optimization process (first inspection condition optimization process) for optimizing inspection conditions based on actual inspection results obtained by the inspection system 100 will be described.

[0081] Figure 4 This is a flowchart of the first inspection condition optimization process according to one embodiment.

[0082] The inspection condition optimization program 212 (strictly speaking, the processor 202 that executes the inspection condition optimization program 212) receives customer indicators (f1~f16) corresponding to customer requirements, target values ​​of each factor evaluation indicator, and weight settings for each factor evaluation indicator via the GUI screen 900 (S501).

[0083] Next, the inspection condition optimization program 212 generates the evaluation index (Formula (3)) used in the processing based on the comparison evaluation index (g1~g16) corresponding to the customer index (f1~f16) and the weight setting, and converts the target value of each factor evaluation index into the target value of the comparison evaluation index (comparison target value) (S502).

[0084] Next, the inspection condition optimization program 212 receives the study conditions set as the constraint conditions related to the inspection conditions in the optimization process (S503) through, for example, the GUI screen 900. Here, the study conditions include, for example, the type of lighting to be studied, the type of imaging sensor, the type of steering mechanism, the type of recognition engine, etc.

[0085] Next, the inspection condition optimization program 212 determines the inspection conditions to be studied according to the study conditions, and sets the inspection system 100 to the settings according to the determined inspection conditions (object inspection conditions) (S504). In addition, if there are conditions in the object inspection conditions that cannot be set by the inspection system 100 itself, the user of the inspection system 100 can also set them.

[0086] Next, the inspection condition optimization program 212 uses the inspection system 100 to drive the operating mechanism 140 according to the specified trajectory (if there is a trajectory condition in the object inspection condition, it is a trajectory corresponding to the condition) (S505), and obtains an image of the inspection object 130 according to the object inspection condition (S506).

[0087] Next, the inspection condition optimization program 212 determines whether all movement of the trajectory is completed and all photography by the photography sensor 120 is completed (S507). If not all movement of the trajectory is completed and all photography by the photography sensor is completed (S507: No), the process proceeds to step S505. Thus, the movement of the trajectory and photography by the photography sensor 120 are continued.

[0088] On the other hand, when all movements of the trajectory are completed and all shooting based on the shooting sensor 120 is completed (S507: Yes), the inspection condition optimization program 212 enables the recognition program 211 to use the captured images to evaluate the performance of the inspection (S508), and calculates the evaluation index Ftarget (S509) based on the results of the performance evaluation, etc.

[0089] Next, the inspection condition optimization program 212 determines whether the evaluation indicators for the inspection conditions corresponding to all the discussion conditions have been calculated (S510). If the evaluation indicators for the inspection conditions corresponding to all the discussion conditions have not been calculated (S510: No), the inspection condition optimization program 212 enters the processing into step S504 in order to execute the inspection conditions including the discussion conditions for which the evaluation indicators have not been calculated.

[0090] On the other hand, when the evaluation indexes for the inspection conditions corresponding to all the research conditions are calculated (S510: Yes), the inspection condition optimization program 212 determines the best inspection condition based on the values ​​of the evaluation indexes according to the plurality of inspection conditions, and outputs the determined result (e.g., displayed on the GUI screen 900) (S511). In addition, in step S511, the best inspection condition is determined and output, but the present invention is not limited to this, and an appropriate inspection condition that satisfies a predetermined standard (e.g., exceeds the target value of the evaluation index) may be determined and output.

[0091] Next, an inspection condition optimization process (second inspection condition optimization process) for optimizing inspection conditions based on inspection results obtained by simulating inspection in the inspection system 100 using an optical model will be described.

[0092] Figure 5 This is a flowchart of the second inspection condition optimization process according to one embodiment.

[0093] The inspection condition optimization program 212 (strictly speaking, the processor 202 that executes the inspection condition optimization program 212) receives customer indicators (f1~f16) corresponding to customer requirements, target values ​​of each factor evaluation indicator, and weight settings for each factor evaluation indicator via the GUI screen 900 (S601).

[0094] Next, the inspection condition optimization program 212 generates the evaluation index (Formula (3)) used in the processing based on the comparison evaluation index (g1~g16) corresponding to the customer index (f1~f16) and the weight setting, and converts the target value of each factor evaluation index into the target value of the comparison evaluation index (comparison target value) (S602).

[0095] Next, the inspection condition optimization program 212 receives the study conditions set as the constraints related to the inspection conditions in the optimization process (S603) through, for example, the GUI screen 900. Here, the study conditions include, for example, the type of lighting to be studied, the type of imaging sensor, the type of steering mechanism, the type of recognition engine, etc.

[0096] Next, the inspection condition optimization program 212 determines the inspection conditions to be studied according to the study conditions, and the optical simulation program 213 generates an optical model (S604) representing the inspection state of the inspection object 130 in the inspection system 100 based on the shape model 601 (three-dimensional shape model) of the inspection object 130, the surface texture 602 of the inspection object 130, and the inspection conditions.

[0097] Here, the shape model 601 is a 3D-CAD model as design information of the product, or a mesh or CAD data generated based on the results measured by a three-dimensional shape measuring device. The shape model 601 can be obtained from the memory 201 of the computer 200, for example, or received from the outside. In addition, the surface texture 602 is information (surface texture information) of the reflection characteristics of light determined by tiny bumps and materials not included in the shape model, expressed by BRDF (Bidirectional Reflectance Distribution Function), etc., and can be input by the user, or it can be data measured by a surface texture measuring machine. The surface texture 602 can be obtained from the memory 201 of the computer 200, for example, or received from the outside.

[0098] The optical simulation program 213 moves the inspection object in the optical model along a specified trajectory (if there is a trajectory condition in the object inspection condition, it is a trajectory corresponding to the condition) (S605), and uses the optical model to generate an image (inferred image) that is inferred to be a picture of the inspection object 130 (S606).

[0099] Next, the inspection condition optimization program 212 determines whether all the movement of the trajectory is completed and the generation of all the inferred images estimated to be taken by the imaging sensor is completed (S607). If not all the movement of the trajectory is completed and the generation of all the inferred images based on the imaging sensor is completed (S607: No), the process proceeds to step S605. Thus, the inferred image is generated when the trajectory movement is performed.

[0100] On the other hand, when all movements of the trajectory are completed and the generation of all inferred images is completed (S607: Yes), the inspection condition optimization program 212 causes the recognition program 211 to use the inferred images to evaluate the performance of the inspection (S608), and calculates the evaluation index Ftarget (S609) based on the results of the performance evaluation, etc.

[0101] Next, the inspection condition optimization program 212 determines whether the evaluation indicators for the inspection conditions corresponding to all the discussion conditions have been calculated (S610). If the evaluation indicators for the inspection conditions corresponding to all the discussion conditions have not been calculated (S610: No), the inspection condition optimization program 212 enters the processing into step S604 in order to execute the inspection conditions including the discussion conditions for which the evaluation indicators have not been calculated.

[0102] On the other hand, when the evaluation indexes for the inspection conditions corresponding to all the research conditions are calculated (S610: Yes), the inspection condition optimization program 212 determines the best inspection condition based on the values ​​of the evaluation indexes according to the plurality of inspection conditions, and outputs the determined result (e.g., displayed on the GUI screen 900) (S611). In addition, in step S611, the best inspection condition is determined and output, but the present invention is not limited to this, and an appropriate inspection condition that satisfies a predetermined standard (e.g., exceeds the target value of the evaluation index) may be determined and output.

[0103] In addition, Figure 4 , Figure 5 In the inspection condition optimization process shown in FIG. 1 , the inspection conditions are comprehensively changed to perform optimization, but the optimization method is not limited to this. For example, effective exploration such as experimental planning method and Bayesian optimization can also be performed. For example, the conditions of the linear part related to the device structure can also be divided (in Figure 3 The Benders decomposition method for searching for optimal conditions is used to determine the conditions of the nonlinear part (the condition represented by the symbol T in the figure) and the condition of the nonlinear part (parameter). In addition, the type and configuration of lighting and the type of recognition engine can be limited based on similar examples in the knowledge database 220. In this case, the similar examples can be found by the user or by the comparison function after machine learning.

[0104] Next, an optical model correction process for correcting the optical model will be described. Figure 5 The second inspection condition optimization process shown is performed before.

[0105] Figure 6 is a flow chart of an optical model calibration process according to one embodiment.

[0106] The optical model correction program 214 (strictly speaking, the processor 202 that executes the optical model correction program 214) receives the setting of correction conditions including the inspection conditions of the inspection system 100 used in the correction (for example, the current inspection conditions of the inspection system 100) and the change conditions of the optical model (for example, the change range of the reflectivity of the inspection object (0.2 to 0.8, etc.)) from the user via the GUI screen (S701).

[0107] Next, the optical model correction program 214 generates an evaluation index in the correction (correction evaluation index) (S702). Here, the correction evaluation index is a function that combines the feature quantities to be compared in the correction, for example, it can be the sum of the average value or variance of the difference between each pixel of the inferred image and the acquired image described later. The correction evaluation index can be defined by the user or selected or generated based on examples.

[0108] Next, the optical model correction program 214 executes a process of acquiring an estimated image using the optical model ( S711 to S714 ) and a process of actually acquiring an image using the inspection system 100 ( S721 to S724 ).

[0109] In the process of obtaining an inferred image using an optical model (S711 to S714), first, the optical model correction program 214 determines the conditions of the optical model to be studied (for example, a specified value within the range of change of the reflectivity of the inspection object) based on the correction conditions, and the optical simulation program 213 generates an optical model (S711) representing the inspection state of the inspection object 130 in the inspection system 100 based on the set conditions in the correction conditions and the determined conditions of the optical model.

[0110] The optical simulation program 213 moves the inspection object in the optical model along a specified trajectory (if there is a trajectory condition in the object inspection condition, it is a trajectory corresponding to the condition) (S712), and uses the optical model to generate an image (inferred image) that is inferred to be a picture of the inspection object 130 (S713).

[0111] Next, the optical model correction program 214 determines whether all the movement of the trajectory is completed and the generation of all the inferred images estimated to be captured by the imaging sensor is completed (S714). If not all the movement of the trajectory is completed and the generation of all the inferred images based on the imaging sensor is completed (S714: No), the process proceeds to step S712. Thus, the inferred image is generated when the trajectory movement is performed.

[0112] On the other hand, when all the movements of the trajectory are completed and the generation of all the estimated images is completed ( S714 : Yes), the optical model correction program 214 advances the process to step S731 .

[0113] In the process of actually acquiring an image using the inspection system 100 (S721 to S724), first, the optical model calibration program 214 sets the inspection system 100 to conform to the setting of the inspection conditions (object inspection conditions) included in the calibration conditions (S721). In addition, if there are conditions in the object inspection conditions that cannot be set by the inspection system 100 itself, the user of the inspection system 100 may also set them.

[0114] Next, the optical model correction program 214 uses the inspection system 100 to drive the operating mechanism according to the specified trajectory (if there is a trajectory condition in the object inspection condition, it is a trajectory corresponding to the condition) (S722), and obtains the image (actual image) of the inspection object 130 according to the object inspection condition (S723).

[0115] Next, the optical model calibration program 214 determines whether all movement of the trajectory is completed and all photography by the photography sensor is completed (S724). If not all movement of the trajectory is completed and all photography by the photography sensor is completed (S724: No), the process proceeds to step S722. Thus, the trajectory movement and photography by the photography sensor are continued.

[0116] On the other hand, when all the movement of the trajectory is completed and all the imaging by the imaging sensor is completed (S724: Yes), the optical model correction program 214 advances the process to step S731.

[0117] In step S731 , the optical model correction program 214 compares the feature quantities of the inferred image and the actual image, and calculates a correction evaluation index for the optical model used.

[0118] Next, the optical model correction program 214 determines whether all changes to the optical model corresponding to the change conditions included in the correction conditions have been performed ( S732 ).

[0119] As a result, when all changes to the optical model have not been made (S732: No), the optical model correction program 214 changes the conditions for the optical model based on the change conditions (S733), and proceeds to step S711 to obtain an estimated image for another optical model.

[0120] On the other hand, when all the changes to the optical model have been made (S732: Yes), the optical model calibration program 214 determines the parameters of the optical model that appropriately represents the inspection of the actual inspection object (e.g., the reflectivity of the inspection object) based on the evaluation index, and outputs the parameters of the optical model as calibration data (S734). Here, the parameters in the optical model that appropriately represents the inspection are determined based on the adopted calibration evaluation index, for example, the parameters whose values ​​of the calibration evaluation index satisfy one of the minimum, maximum, minimum, maximum, satisfying a specified value, a maximum value with a gentle slope (stable), and a minimum value with a gentle slope.

[0121] In addition, if you then execute Figure 5In the second inspection condition optimization process shown, the optical simulation program 213 generates an optical model based on the calibration data, and can generate an optical model that conforms to the inspection state in the actual inspection system 100, thereby generating a high-precision estimated image.

[0122] Next, the GUI screen 900 will be described.

[0123] Figure 7 This is a diagram showing a GUI screen according to one embodiment.

[0124] The GUI screen 900 includes an input area 910 and an output area 920. The input area 910 includes a customer requirement input area 911, a research condition input area 912, and a weight input area 913.

[0125] The customer requirement input area 911 is an area for inputting customer requirements, and includes fields for indicating the symbol of the factor evaluation index required by the customer, the customer requirement indicating the content of the factor evaluation index, and the target value for inputting the target value of the factor evaluation index. Here, by setting the target value as unnecessary, it is possible to set the evaluation index of the corresponding factor evaluation index to be not considered. That is, it is possible to make the value of the corresponding factor evaluation index in formula (3) not be considered.

[0126] The study condition input area 912 is an area for inputting conditions (study conditions) to be studied among the inspection conditions in the inspection system 100 .

[0127] The weight input area 913 is an area for inputting a weight value for each element evaluation index value. In the weight input area 913, a weight value and a setting of whether to set the weight value as a fixed value or a variable value can be input.

[0128] The processing result list 921 is displayed in the output area 920. The processing result list 921 includes entries of appropriate inspection conditions detected in the inspection condition optimization process. For example, when the weights wi of all element evaluation indicators are input as fixed values, one entry is displayed, and when the weights wi of the element evaluation indicators are input as variable values, entries of appropriate inspection conditions corresponding to each group of weights of each element evaluation indicator are displayed.

[0129] The entry of the processing result list 921 includes two fields: element evaluation index / weight 921a and inspection condition 921b. The value of each element evaluation index and the value of the weight for the element evaluation index are displayed in element evaluation index / weight 921a. The value of each inspection condition of the inspection system 100 (inspection condition information) is stored in inspection condition 921b.

[0130] In the above description, an example of inspecting the inspection object 130 as a whole is shown, but the inspection object 130 may have different lighting, imaging sensors or recognition programs for each part of the inspection object depending on the shape and surface properties. For example, if the inspection object is a part after a part of a casting is cut, the surface properties of the part where the casting surface is retained and the part where it is cut are different, and the image features are greatly different. Therefore, by changing the type of lighting, lighting angle, lighting intensity or changing the parameters of the recognition program, it is helpful to improve the inspection success rate.

[0131] Therefore, the inspection region of the inspection object 130 may be divided into a plurality of parts for inspection. The following describes a case where the inspection is performed by dividing the inspection region into a plurality of inspection regions.

[0132] Figure 8 1 is a diagram for explaining inspection conditions when the inspection area is divided according to one embodiment. Figure 8 This is an example of inspecting the inspection object 130 by dividing it into the plane surface 131 of the upper surface and the cylindrical surface 132. This is an example of inspecting by photographing the plane 131 using the lighting 110a and the imaging sensor 120a, and photographing the cylindrical surface 132 using the lighting 110b and the imaging sensor 120b.

[0133] The inspection conditions of the inspection system 100 include conditions related to lighting (illumination conditions), conditions related to photographing (photographing conditions), conditions related to manipulation (manipulation conditions), and conditions related to recognition (recognition conditions).

[0134] Lighting conditions include lighting type, lighting configuration, lighting intensity, lighting time, lighting cycle, etc. Shooting conditions include shooting sensor type, shooting sensor configuration, aperture amount, shooting time, shooting cycle, etc. Manipulation conditions include manipulation mechanism type, target position and posture, path to reach target position and posture, etc. Recognition conditions include recognition engine type, recognition processing parameters, etc.

[0135] All conditions passed Figure 8 Here, the symbols are explained in the same way as Figure 3 The differences between the symbols shown.

[0136] As a subscript at the lower right of the letter, m is added. m is a number indicating the surface (inspection target area) of the inspection object. In this example, when the surface of the inspection object is a plane 131, m is set to 1, and when the surface of the inspection object is a cylindrical surface 132, m is set to 2.

[0137] Next, the inspection condition optimization process when the inspection object 130 is divided into a plurality of parts for inspection will be described.

[0138] First, the inspection condition optimization program 212 determines a division plan for dividing the inspection object 130 into a plurality of inspection regions (partial inspection regions) based on the processing conditions of the inspection object 130 (division plan determination step).

[0139] Next, the inspection condition optimization program 212 sets each inspection area in the segmentation scheme as an object and executes Figure 4 or Figure 5 The inspection condition optimization process is performed to determine the best inspection condition, and the evaluation index value of each inspection area is summed up to calculate the overall evaluation index value (segmentation scheme evaluation index value) at the time of segmentation (segmentation scheme evaluation index calculation step).

[0140] Next, the inspection condition optimization program 212 determines inspection conditions for a plurality of segmentation schemes for the inspection object 130 and calculates each segmentation scheme evaluation index value by performing a segmentation scheme determination step and a segmentation scheme evaluation index calculation step.

[0141] Next, the inspection condition optimization program 212 determines an appropriate (for example, optimal) segmentation scheme and inspection condition that satisfy predetermined conditions based on the segmentation scheme evaluation indexes of the plurality of segmentation schemes, and outputs the determination result.

[0142] According to this process, it is possible to determine an appropriate division scheme of the inspection object 130 and an appropriate inspection condition for each inspection region in the division scheme.

[0143] Next, changes over time of an inspection image of an inspection object targeted by the inspection system 100 will be described.

[0144] Fig. 9 It is a diagram for explaining changes in an inspection image according to one embodiment. Fig. 9 (A) shows the state of photographing the inspection object, Fig. 9 (B) shows an inspection image immediately after the inspection system 100 starts operating. Fig. 9 (C) shows an inspection image after a certain amount of time has passed since the inspection system 100 was put into operation.

[0145] In the inspection system 100, it is assumed that Fig. 9 As shown in (A) of FIG. 1 , the inspection object 130 is irradiated with the illumination 110 from above, and an image is captured by the imaging sensor 120 .

[0146] After the inspection system 100 begins to operate, Fig. 9 As shown in (B) of FIG. 1 , the inspection image of the inspection object 130 is in a state in which the inspection object 130 is clearly captured.

[0147] Then, the inspection system 100 continues to operate. For example, if the lens of the imaging sensor 120 is damaged, the inspection image of the inspection object 130 is as follows: Fig. 9 There is a case where the scratch 1031 is reflected as shown in (C). In addition, if the lens is dirty, there is a concern that the entire image will become darker. In addition, even if the inspection object 130 is the same product, there is a concern that the color tone and brightness of the surface 1032 of the inspection object 130 may change over time due to changes in the materials used in the manufacture of the inspection object 130.

[0148] In such a state, the inspection conditions in the inspection system 100 may no longer be the optimal inspection conditions.

[0149] Therefore, in the inspection system 100 of the present embodiment, in such a case, a process of redetermining the inspection conditions (inspection condition redetermining process) is performed.

[0150] Fig.10 This is a flowchart of an inspection condition redetermination process according to an embodiment.

[0151] The inspection image change detection program 215 receives an allowable value for a change in a feature quantity in an inspection image (S1101). Here, the feature quantity may be a statistical quantity such as a brightness average or brightness variance in an inspection image or a predetermined region of the inspection image. In addition, the allowable value may be received from a user when the inspection system 100 is started or each time a predetermined number of inspections are performed. In addition, the allowable value may be a default value.

[0152] Next, the inspection image change detection program 215 calculates the feature amount of the image based on the inspection image of the inspection object captured and stored by the inspection system 100 ( S1102 ).

[0153] Next, the inspection image change detection program 215 determines whether the calculated feature amount exceeds the allowable value (S1103).

[0154] As a result, if the calculated feature value exceeds the allowable value (S1103: Yes), the inspection image change detection program 215 issues an alarm and executes the inspection condition optimization program 212 to execute the inspection condition optimization program 212. Figure 4 or Figure 5The inspection condition optimization process (S1104) is performed, and the inspection image change detection program 215 outputs the result (S1105). In addition, when the inspection condition optimization process is performed in the inspection condition re-determination process, part of the inspection conditions such as the type of illumination, the type of image sensor, and the operating mechanism in the inspection conditions can also be fixed to the current state. In this way, the inspection conditions of the structure that is already used in the inspection system 100 can be directly used, thereby preventing the occurrence of additional investment in the inspection system 100.

[0155] On the other hand, when the calculated feature value does not exceed the allowable value (S1103: No), the inspection image change detection program 215 outputs a result that there is no need to perform the inspection condition optimization process (S1105).

[0156] Next, an example in which the computer 200 optimizes various conditions in a manufacturing process including a plurality of processing steps and a plurality of inspection steps will be described.

[0157] Fig.11 It is a diagram for explaining the manufacturing process of one embodiment.

[0158] The manufacturing process includes inspection process 1, processing process 1, inspection process 2, processing process 2, inspection process 3, processing process 3, and inspection process 4. In the inspection process 1, the received blank is inspected by the inspection system 100-1, in the inspection process 2, the processed product processed in the processing process 1 is inspected by the inspection system 100-2, in the inspection process 3, the processed product processed in the processing process 2 is inspected by the inspection system 100-3, and in the inspection process 4, the processed product processed in the processing process 3 is inspected by the inspection system 100-4. Here, the process capability, which is information (manufacturing information) related to manufacturing in the processing process, is denoted as Ak, the yield is denoted as Tk, the manufacturing speed is denoted as Sk, and the manufacturing cost is denoted as Ck. In addition, k represents the process number of the processing process.

[0159] The inspection condition optimization process based on multiple inspection condition optimization programs 216 in this example is described. Figure 3 ( Figure 4 ), explaining the processing differences between the multiple inspection condition optimization programs 216 and the inspection condition optimization program 212. In addition, it is assumed that the process capability Ak, yield rate Tk, manufacturing cost Sk, and manufacturing cost Ck of each processing process are known or imagined, and are set or input.

[0160] In step S501 (S601), the multiple inspection condition optimization program 216 receives the customer index fij (i=1-16, j is the process number of the inspection process, and Fig.11 In the example of , j = 1 to 4), target values ​​and weights of evaluation indicators for each factor required by the customer are set (Wij). In addition, multiple inspection condition optimization programs 216 receive overall customer indicators Fi (i = 1 to 16) corresponding to customer requirements in the overall inspection process, target values ​​and weights of evaluation indicators for each factor required by the overall customer (Wi).

[0161] In step S502 (S602), the multiple inspection condition optimization programs 216 obtain the comparative evaluation index gij corresponding to the customer index fij (i=1~16, j=1~4), obtain the comparative evaluation index Gi corresponding to the overall customer index Fi (i=1~16), and generate the evaluation index (Formula (4)) used in the processing taking into account the inspection process and the processing process, and convert the target values ​​of the customer index and the overall customer index into the target value of the comparative evaluation index (comparison target value).

[0162]

Number 4

[0163]

[0164] Here, wak, wtk, wsk, and wck are weights of process capability Ak, yield rate Tk, manufacturing cost Sk, and manufacturing cost Ck, respectively. In addition, in formula (4), the evaluation index is set as the linear sum of the evaluation index of each processing step and the information of the manufacturing status of the processing step, but the present invention is not limited to this, for example, it can be set as a product, or it can be set as the sum of the reciprocals of each index.

[0165] In step S511 (S611), the plurality of inspection condition optimization programs 216 determine the best inspection conditions based on the values ​​of the evaluation index (formula (4)) according to the plurality of inspection conditions, and output the determined results.

[0166] By performing the process of optimizing the conditions in the manufacturing process, customer requirements for the inspection process and information on the processing process are collected as one evaluation index, so that appropriate inspection conditions can be determined in the entire manufacturing process.

[0167] In addition, the present invention is not limited to the above-described embodiment, and can be implemented with appropriate modifications within the scope not departing from the gist of the present invention.

[0168] For example, in the above-mentioned embodiment, the optimal inspection condition is determined, but the present invention is not limited to this, and the inspection condition satisfying the predetermined condition may be determined instead of the optimal inspection condition.

[0169] In addition, in the above embodiment, the computer 200 is enabled to execute Figure 3 and Figure 4 , but it is also possible to only perform the inspection condition optimization processing of one party.

[0170] In the above-described embodiment, the computer 150 and the computer 200 are different computers, but for example, the functions of the computer 200 may be incorporated into the computer 150 .

[0171] In addition, in the above-mentioned embodiments, a part or all of the processing performed by the processor may be performed by a hardware circuit. In addition, the program in the above-mentioned embodiments may be installed from a program source. The program source may be a program distribution server or a recording medium (e.g., a removable recording medium).

[0172] Description of Reference Numerals

[0173] 100…inspection system, 150…computer, 200…computer, 201…memory, 202…processor, 203…input device, 204…display device, 211…recognition program, 212…inspection condition optimization program, 213…optical simulation program, 214…optical model correction program, 215…inspection image change detection program, 216…multiple sets of inspection condition optimization programs.

Claims

1. An inspection condition determination system, comprising a processor, for determining inspection conditions in an inspection of an inspection system for a predetermined inspection object, wherein: The processor receives input of at least two or more factor evaluation indicators of the inspection accuracy, false alarm rate, inspection time, inspection reproducibility, learning time, device cost, operation cost, device robustness, inspection versatility, lifespan, ease of assembly, required assembly accuracy, robustness to the environment, ease of maintenance, clarity of judgment basis of identification processing, and ease of identification of the detection part, as well as user requirements related to the targets of the factor evaluation indicators, The processor generates an evaluation index based on a plurality of element evaluation indexes related to the inspection in the inspection system based on the user request, The processor evaluates the inspection results under a plurality of inspection conditions based on the evaluation index and determines an appropriate inspection condition from among the plurality of inspection conditions.

2. The inspection condition determination system according to claim 1, wherein: The processor displays and outputs inspection condition information related to the determined inspection condition.

3. The inspection condition determination system according to claim 2, wherein: The evaluation index is an index obtained by combining values ​​of a plurality of the element evaluation indexes by adjusting weights.

4. The inspection condition determination system according to claim 3, wherein: the processor adjusting the weights, The processor evaluates the inspection results under the plurality of inspection conditions based on the evaluation index after the weight is adjusted, and determines an appropriate inspection condition from the plurality of inspection conditions. The processor displays an inspection condition and a value of a factor evaluation index corresponding to the inspection condition.

5. The inspection condition determination system according to claim 4, wherein: The adjustment of the weight is accepted from the user.

6. The inspection condition determination system according to claim 1, wherein: The inspection system includes an illumination device for irradiating light to the inspection object, an imaging sensor for imaging the inspection object, and a manipulation mechanism for adjusting the relative position of the inspection object and the imaging sensor. The inspection conditions are at least a part of the lighting type, lighting configuration, lighting intensity, lighting time, lighting cycle, shooting sensor type, shooting sensor configuration, aperture amount, shooting time, shooting cycle, control mechanism type, target position posture, trajectory, recognition engine and recognition processing parameters.

7. The inspection condition determination system according to claim 6, wherein: The inspection conditions include at least one of an imaging sensor type, an imaging sensor configuration, an aperture amount, an imaging time, and an imaging cycle, and at least one of a manipulation mechanism type, a target position posture, and a trajectory.

8. The inspection condition determination system according to claim 1, wherein: The inspection system includes an illumination device for irradiating light to the inspection object and an imaging sensor for imaging the inspection object. the processor generating an optical model related to an inspection state of the inspection object based on a three-dimensional shape model of the inspection object, surface property information of the inspection object, inspection conditions related to the illumination, and inspection conditions related to the imaging sensor, The processor generates an inferred image inferred to be captured by the capturing sensor based on the optical model, The processor performs evaluation based on the evaluation index in the inspection result using the estimated image, and determines an appropriate inspection condition from among a plurality of inspection conditions.

9. The inspection condition determination system according to claim 8, wherein: The processor receives input of the three-dimensional shape model and the surface property information.

10. The inspection condition determination system according to claim 8, wherein: The processor obtains an actual image captured by the inspection system under the same inspection conditions as when the inferred image is generated, The processor determines feature quantities of the inferred image and the actual image, and corrects the optical model based on the feature quantities.

11. The inspection condition determination system according to claim 1, wherein: The processor generates a plurality of segmentation schemes for segmenting an inspection region of the inspection object into a plurality of partial inspection regions. For each segmentation scheme, the processor evaluates a plurality of inspection conditions based on the evaluation index for a plurality of local inspection areas, and determines an appropriate inspection condition from among the plurality of inspection conditions. The processor determines an appropriate partitioning scheme from among a plurality of partitioning schemes.

12. The inspection condition determination system according to claim 1, wherein: The processor obtains an actual image captured by the inspection system according to the determined inspection conditions, determines whether the inspection result obtained based on the actual image meets the target of the element evaluation index required by the user, and issues an alarm if the target is not met. It also evaluates the new inspection result obtained based on multiple inspection conditions based on the evaluation index and determines an appropriate inspection condition from the multiple inspection conditions.

13. The inspection condition determination system according to claim 1, wherein: The processor obtains multiple actual images captured by the inspection system according to the determined inspection conditions, determines whether the feature quantities of the multiple actual images exceed the prescribed allowable values, issues an alarm if the feature quantities exceed the allowable values, performs an evaluation based on the evaluation index on the new inspection results obtained based on the multiple inspection conditions, and determines appropriate inspection conditions from the multiple inspection conditions.

14. The inspection condition determination system according to claim 1, wherein: The processor accepts the input of the user's request for each of the plurality of inspection systems. The processor generates an evaluation index based on multiple element evaluation indicators regarding the inspection in the inspection system based on the user requirements, evaluates the inspection results under multiple inspection conditions based on the evaluation index, and determines an appropriate inspection condition from the multiple inspection conditions.

15. The inspection condition determination system according to claim 14, wherein: Multiple inspection systems are used in multiple inspection processes in the manufacturing process. The processor further accepts input of at least two of the factor evaluation indicators of inspection accuracy, false alarm rate, inspection time, inspection reproducibility, learning time, device cost, operation cost, device robustness, inspection versatility, lifespan, ease of assembly, required assembly accuracy, robustness to the environment, ease of maintenance, clarity of judgment basis of identification processing, and ease of identification of the detection part, as well as user requirements related to the targets of the factor evaluation indicators, regarding the inspection accuracy, false alarm rate, inspection time, inspection reproducibility, learning time, device cost, operation cost, device robustness, inspection versatility, lifespan, ease of assembly, required assembly accuracy, robustness to the environment, ease of maintenance, clarity of judgment basis of the identification processing, and ease of identification of the detection part, for the inspection of the plurality of inspection systems as a whole, Based on at least one of the manufacturing information including process capability, yield, manufacturing speed, and manufacturing cost of the manufacturing process, factor evaluation indicators for multiple inspection systems as a whole, and factor evaluation indicators for each inspection system, evaluation indicators for manufacturing and inspection in the manufacturing process are generated, and inspection results and manufacturing conditions under multiple inspection conditions are evaluated based on the evaluation indicators, and appropriate inspection conditions are determined from the multiple inspection conditions.

Images