Parameter adjustment assistance device and parameter adjustment assistance method

Abstract

The present invention relates to a parameter adjustment assistance device and a parameter adjustment assistance method, which simplify the parameter adjustment operation for a user with respect to an image sensor having a plurality of inspection processes. The parameter adjustment assistance device includes: an adjustment object setting unit that sets N items of inspection processes to be adjusted from among the plurality of inspection processes; an image acquisition unit that acquires an OK image and an NG image; a graph generation unit that generates graphs for the N items of inspection processes, respectively, depicting the OK image measurement value, the NG image measurement value, and the current threshold value, where the OK image measurement value is the value of a measurement parameter calculated from the OK image according to the current conditions of the set parameters, and the NG image measurement value is the value of the measurement parameter calculated from the NG image of the inspection process according to the current conditions of the set parameters; and an output unit that outputs the graph to a display device.

Description

Technical Field

[0001] The present invention relates to a technique for assisting in adjusting parameters used in an inspection process of an image sensor. Background Art

[0002] In a factory production line or the like, a system called an image sensor is often used to automate or streamline the inspection of manufactured products (see Patent Document 1). In an image sensor, a series of inspection processes are configured by combining multiple inspection processes. In order to obtain correct inspection results through an image sensor, it is necessary to appropriately set the parameters used in each inspection process. Conventionally, however, there has been only a method of setting the parameters of each inspection process individually, and the operation of adjusting (regulating) these parameters is not simple.

[0003] Patent Document 1: Japanese Unexamined Patent Application Publication No. 2008-015706 Summary of the Invention

[0004] Problems to be Solved by the Invention

[0005] The present invention has been made in view of the above actual situation, and an object thereof is to provide a technique for simplifying the operation of adjusting parameters performed by a user with respect to an image sensor having a plurality of inspection processes.

[0006] Means for Solving the Problems

[0007] The present disclosure includes a parameter adjustment assistance device that assists in adjusting parameters used in each inspection process for an image sensor capable of performing a plurality of inspection processes on an image. The parameter adjustment assistance device is characterized in that the inspection process calculates a value of a prescribed measurement parameter from the image, compares the value of the measurement parameter with a threshold value, and outputs a determination result of OK or NG. The parameters include setting parameters and the threshold value. The setting parameters define conditions for calculating the value of the measurement parameter. The parameter adjustment assistance device includes: an adjustment target setting unit that sets N items of inspection processes to be adjusted from among the plurality of inspection processes, where N is an integer of 2 or more; an image acquisition unit that acquires an OK image that should obtain an OK determination in all of the N items of inspection processes and an NG image that should obtain an NG determination in at least any one of the N items of inspection processes; a graph generation unit that generates the following graphs for each of the N items of inspection processes: in this graph, an OK image measurement value, which is a value of the measurement parameter calculated from the OK image according to the current conditions of the setting parameters, an NG image measurement value, which is a value of the measurement parameter calculated from the NG image of this inspection process according to the current conditions of the setting parameters, and the current threshold value are depicted; and an output unit that outputs the graph to a display device.

[0008] Alternatively, the OK image measurement value, the NG image measurement value, and the threshold value may be depicted in the chart in such a way that at least one of color or pattern is different.

[0009] Alternatively, the chart generation unit determines whether adjustment of the parameter for the inspection process is necessary based on the relationship among the OK image measurement value, the NG image measurement value, and the threshold value in the inspection process of the object to be adjusted, and depicts the determination result in the chart.

[0010] Alternatively, the chart generation unit further determines which one of the threshold value and the set parameter needs to be adjusted, and depicts the determination result in the chart.

[0011] Alternatively, the chart generation unit determines at least three states: no adjustment of the parameter is required, adjustment of the threshold value is required, and adjustment of the set parameter is required, and the three states are depicted in the chart in different colors.

[0012] Alternatively, the output unit outputs an adjustment window for performing the adjustment operation of the parameter to the display device.

[0013] Alternatively, a scatter diagram depicting the OK image measurement values of the respective OK images, the NG image measurement values of the respective NG images, and the threshold value is displayed in the adjustment window.

[0014] Alternatively, the adjustment window can switch the OK image measurement value and the NG image measurement value depicted in the scatter diagram to the OK image measurement value of only the OK images deviating from the OK range specified by the threshold value and the NG image measurement value of the NG images included in the OK range.

[0015] The present disclosure includes a parameter adjustment assistance method for an image sensor capable of performing multiple inspection processes on an image, which assists in adjusting parameters used in each inspection process. The parameter adjustment assistance method is characterized in that the inspection process calculates the value of a prescribed measurement parameter based on the image, compares the value of the measurement parameter with a threshold value, and outputs a determination result of OK or NG. The parameters include a setting parameter and the threshold value. The setting parameter defines the conditions for calculating the value of the measurement parameter. The parameter adjustment assistance method includes the following steps: setting N items of inspection processes to be adjusted from the multiple inspection processes, where N is an integer of 2 or more; obtaining an OK image that should obtain an OK determination in all of the N items of inspection processes and an NG image that should obtain an NG determination in at least any one of the N items of inspection processes; generating a graph for each of the N items of inspection processes, in which the value of the measurement parameter calculated based on the OK image according to the current conditions of the setting parameter, i.e., the OK image measurement value, the value of the measurement parameter calculated based on the NG image of the inspection process according to the current conditions of the setting parameter, i.e., the NG image measurement value, and the current threshold value are depicted; and outputting the graph to a display device.

[0016] The present disclosure includes a program for causing a processor to execute each step of the above parameter adjustment assistance method.

[0017] The present invention can be understood as a parameter adjustment assistance device having at least a part of the above units, or can be understood as an image sensor including the device. In addition, the present invention can also be understood as a parameter adjustment assistance method or an image sensor control method including at least a part of the above processing, or a program for implementing the method, or a recording medium non-temporarily recording the program. Furthermore, the above respective units and processes can be combined with each other as much as possible to constitute the present invention.

[0018] Advantageous Effects of the Invention

[0019] According to the present invention, regarding an image sensor having multiple inspection processes, it is possible to simplify the parameter adjustment operation performed by a user. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Figure 1 is a diagram schematically showing the appearance of an image sensor.

[0021] Figure 2 Figure 2 is a functional block diagram showing a structural example of an inspection function and a parameter adjustment assistance function.

[0022] Figure 3 Figure 3 ​​​​​​It is a diagram for explaining the processing flow of inspection and the setting of parameters.

[0023] Figure 4 Figure 4 It is a flowchart showing the process of parameter adjustment operation.

[0024] Figure 5 Figure 5 It is an example of the main screen of the parameter adjustment assist tool.

[0025] Figure 6 Figure 6 It is an example of the label setting tab of the image classification window.

[0026] Figure 7 Figure 7 It is an example of the label assignment tab of the image classification window.

[0027] Figure 8 Figure 8 It is an example of radar chart display.

[0028] Figure 9 Figure 9 It is an example of adjusting the threshold through the adjustment window.

[0029] Figure 10 Figure 10 It is an example of adjusting the set parameters through the adjustment window.

[0030] Figure 11 Figure 11 It is an example of adjusting the set parameters for the area centroid.

[0031] Figure 12 Figure 12 It is an example of adjusting the set parameters through the adjustment window.

[0032] Figure 13 Figure 13 It is an example of the radar chart after the parameter adjustment is completed. Detailed implementation mode

[0033] Refer to Figure 1 , and the image sensor of the implementation mode of the present invention will be described. Figure 1 It is a diagram schematically showing the appearance of the image sensor.

[0034] The image sensor 1 is, for example, arranged on the production line PL of a factory, etc., and is a device for various image-based processes. The image sensor 1 is also called an image processing system, a vision sensor, a vision system, etc.

[0035] ​​​​​​​​​​​​​​​​​​​​The image sensor 1 includes a photographing device 10, an image processing device 11, a display device 12, and an input device 13 as hardware resources. The photographing device 10, the display device 12, and the input device 13 are connected to the input / output I / F (interface) of the image processing device 11. Additionally, in Figure 1 the image sensor 1 in which the photographing device 10 and the image processing device 11 have a separate structure is shown, but it may also be an integrated processing type image sensor in which the photographing device and the image processing device are integrated.

[0036] The photographing device 10 is a device including an illumination unit, a lens unit, a photographing unit, etc., and is also called an industrial camera. The illumination unit is a device that illuminates a subject (the inspection object O), and is composed of, for example, an LED light source or the like. The lens unit is an optical device that forms an optical image of the subject on the photographing unit, and uses, for example, an optical system having functions such as focus adjustment, aperture, and zoom. The photographing unit is a device that generates and outputs image data through photoelectric conversion, and is composed of, for example, a photographing element such as a CCD or CMOS sensor.

[0037] As the main functions, the image processing device 11 has functions such as taking in image data from the photographing device 10, performing image processing on the image data, performing inspection processing based on the result of the image processing, performing data transmission and reception with an external device via the input / output I / F, generating data to be output to the external device, processing data received from the external device, and controlling the photographing device 10 and the input / output I / F. The image processing device 11 can be composed of, for example, an image processing computer including a processor (CPU, GPU, etc.), a memory (RAM, ROM, etc.), a storage device (a non-volatile storage device such as a hard disk, a solid state drive, etc.), an input / output I / F, etc. The functions and processes of the image processing device 11 described later can be realized by loading a program stored in the storage device into the memory and executing it by the processor. However, part or all of the functions of the image processing device 11 can also be realized by an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc., and can also be executed by other computers, cloud servers, etc.

[0038] Figure 2 is a functional block diagram showing a structural example of the inspection function and the parameter adjustment assistance function in the image processing device 11.

[0039] The image processing apparatus 11 includes an image input unit 20, an image storage unit 21, an inspection execution unit 22, a parameter storage unit 23, an inspection process setting unit 24, and a parameter adjustment assistance unit 25. The image input unit 20 takes in image data from the imaging device 10. The acquired image data is stored in the image storage unit 21. The inspection execution unit 22 performs inspections on the image data for multiple items and outputs OK / NG determination results for each item. The determination results of the inspections may also be saved in association with the image data. The set values of the parameters used in each inspection process are registered in the parameter storage unit 23. The inspection execution unit 22 refers to the set values of the parameters from the parameter storage unit 23 when performing the inspection process. The inspection process setting unit 24 provides a user interface (UI) for the processing flow of the inspection performed by the inspection execution unit 22 and the setting of the parameters used in individual inspection processes. The parameter adjustment assistance unit 25 provides a tool for assisting the user in adjusting (regulating) the parameters to appropriate values. The parameter adjustment assistance unit 25 includes an image acquisition unit 250, an adjustment target setting unit 251, a graph generation unit 252, and a UI output unit 253. The details of these functions will be described later.

[0040] (Setting of Inspection Process)

[0041] Refer to Figure 3 , and an example of the steps for setting the processing flow and parameters of the inspection using the UI of the inspection process setting unit 24 will be described.

[0042] In the image processing apparatus 11, a variety of modules for performing specific processes (hereinafter referred to as "processing units") are prepared. The processing units are roughly divided into processing units related to inspection processes and processing units related to processes other than inspection processes. As processing units for inspection processes, for example, various processing units such as "shape search" for checking whether a region of a specified shape exists in the image, "area centroid" for checking the area and position of a region of a specified color, and "edge position" for checking the position of the edge of the subject are prepared. In addition, as processing units other than inspection processes, for example, "camera image input" for taking in a captured image from the imaging device 10, "color extraction color filter" for extracting a specified color range from the image, etc. As Figure 3 shown, the user sets the inspections to be performed by appropriately combining the processing units using the UI of the inspection process setting unit 24. A series of processes created by combining multiple processing units is called a "processing flow".

[0043] The processing units for inspection processing have the following common design: calculate the value of a specified measurement parameter based on features extracted from an image, compare the value of the measurement parameter with a threshold value, and output a determination result of "OK" or "NG". For example, in the case of the above "shape search", the similarity (correlation value) between the registered model and the model detected from the image, the position (XY coordinates) of the detected model, the inclination of the detected model (relative angle with respect to the registered model), etc. are calculated as measurement parameters. Then, thresholds (the upper threshold value and the lower threshold value that define the OK range) are set for the similarity, position, and inclination respectively. If all the values of the similarity, position, and inclination are within the OK range, the determination result of "OK" is output. If even one deviates from the OK range, the determination result of "NG" is output. Additionally, in the case of "area centroid", the area of the region, the centroid position (XY coordinates) of the region, etc. are calculated as measurement parameters. Then, thresholds are set for the area and the centroid position respectively. If both the area and the centroid position are within the OK range, "OK" is output. If any one deviates from the OK range, a determination result such as "NG" is output. The "shape search" and "area centroid" exemplified here are methods for making OK / NG determinations for multiple measurement parameters, that is, methods for performing inspection processing for multiple items by one processing unit. However, there are also processing units that have only one type of measurement parameter (inspection processing for one item).

[0044] The indicators used as measurement parameters in each processing unit are predefined, and users cannot change them. However, users can set various conditions (hereinafter referred to as "setting parameters") when calculating the value of the measurement parameter and the thresholds for determination. For example, as setting parameters in "shape search", examples include the model to be searched, the position of the model, and the feature points on the model that are concerned in the calculation of the similarity. Additionally, as Figure 3 shown, as setting parameters in "area centroid", examples include the conditions of the color (the ranges of hue, chroma, and lightness respectively) extracted as the region for calculating the area and the centroid. Depending on the setting of these setting parameters, the value of the measurement parameter calculated from the image changes significantly. Additionally, whether the threshold is set appropriately affects the determination accuracy of OK / NG and the reliability of the determination result.

[0045] Therefore, after the user combines the processing units to create a processing flow, it is necessary to appropriately set the setting parameters and thresholds of each processing unit to obtain the desired inspection performance (determination result). However, the operation of adjusting (regulating) the setting parameters and thresholds to appropriate values ​​requires a high level of skills and knowledge. Moreover, when the number of processing units included in the processing flow is large and each processing unit has a large number of setting parameters and thresholds, it is not easy to adjust the setting parameters and thresholds in a well-balanced manner so as to obtain an appropriate determination result among all of them.

[0046] Therefore, in the image sensor 1 of the present embodiment, a parameter adjustment assisting tool for assisting the adjustment work of setting parameters and thresholds is provided by the parameter adjustment assisting unit 25 (the parameter adjustment assisting tool is an example of the parameter adjustment assisting device of the present invention).

[0047] (Parameter adjustment auxiliary tool)

[0048] Reference Figures 4 to 12 , the parameter adjustment auxiliary tool of this embodiment is described in detail. Figure 4 This is a flowchart showing the flow of parameter adjustment work using a parameter adjustment supporting tool. Figure 5 This is an example of the main screen of the parameter adjustment assist tool.

[0049] When the parameter adjustment assisting tool (hereinafter also referred to as “tool”) is started, the UI output unit 253 displays Figure 5 The main screen displays a list 50 of a plurality of processing units constituting the processing flow.

[0050] The user first specifies the location where the image (sample image) used in the adjustment operation is stored in the "Re-measurement Object Image" column 51 of the main screen (step S41). When the "main body recorded image" is specified, the image group stored in the image storage unit 21 (for example, the RAM disk or internal storage device of the image sensor 1) can be used, and when the "specified folder image" is specified, the image group stored in the external storage device can be used. When the storage location is specified, the sample image is taken in by the image acquisition unit 250. Although the adjustment operation can be performed using only one sample image, multiple sample images obtained by shooting different inspection objects can also be used for the adjustment operation. In addition, the multiple sample images can include both images of good products and images of defective products, and further, the images of defective products can include multiple defects (that is, the inspection and processing items that have NG judgments are different from each other). By using multiple images with changes to simultaneously adjust the parameters for multiple inspection and processing items, the appropriate parameter adjustment (Japanese: 追い込み) operation can be efficiently implemented.

[0051] Next, when the user presses the "Image Classification" button 52 on the main screen, the image classification window shown by Figure 6 is displayed by the adjustment object setting unit 251. The user specifies the inspection processes to be adjusted in the "Label Setting" tab 60 of the image classification window and performs the operation of setting labels for them (step S42). Specifically, the number of items of the inspection process to be adjusted is input in the "Setting Quantity" column 61. The number of items N can be set, for example, within the range of 3 to 20. When the number of items N is input, N labels are generated, so the user sets the "Label Name", "Object Unit", and "Measurement Parameter" for each label. The "Label Name" can be set arbitrarily. In the "Object Unit", the name of the processing unit that performs the inspection process to be adjusted is specified, and in the "Measurement Parameter", the name of the measurement parameter used in the inspection process to be adjusted is specified. Figure 6 An example in which five labels of "Circularity", "Printing Area", "Printing Presence / Absence", "Printing Quality", and "Printing Height" are set is shown. Figure 6

[0052] Figure 7 When the label setting is completed, the user performs the operation of assigning labels to the image through the "Label Assignment" tab 62. When the "Label Assignment" tab 62 of the image classification window is selected, it switches to Figure 7 . Figure 7The UI shown in FIG. 6 is a UI of the “label assignment” tab 62, which includes a label display 71, a preview button 72, a file list 73, and a label assignment column 74. The label display 71 displays a list of “judgment labels” and “defect type labels”. Judgment labels are labels indicating the judgment results of images, and three labels, “OK”, “NG”, and “classification retention”, are prepared in advance. Defect type labels are labels defined by the user through the “label setting” tab 60. The file names of the images specified in step S41 are displayed in the file list 73. When multiple images are specified in step S41, the file names of the multiple images are displayed in the file list 73. When the preview button 72 is pressed, a preview of the image selected in the file list 73 can be displayed. The user first assigns a judgment label while confirming the preview image. The “OK” label is assigned to images that should be judged as OK in all inspection processes that are the adjustment objects, the “NG” label is assigned to images that should be judged as NG in any inspection process, and the “classification retention” label is assigned to images that cannot be judged as either OK or NG. In the label assignment column 74, grids corresponding to each label are prepared, and the assignment and removal of the label can be switched each time the grid is pressed. For the image assigned with the "NG" label, the defect type label is then assigned. That is, the information in which inspection process should be judged as NG is associated with the image. In addition, multiple defect type labels can be assigned to one image. In the label display 71 and the label assignment column 74, each label can be displayed in a different color from each other. Thereby, it is easy to visually confirm the label assigned to each image, the label assignment operation becomes easy, and operational errors such as assigning the wrong label can be prevented. By performing the above operation, multiple OK images and multiple NG images can be prepared.

[0053] When labeling (i.e., image classification) is completed and the OK button is pressed, the screen returns to the main screen. In addition, if you want to save the content set on the "label setting" tab 60 and the "labeling" tab 62, you can save it in a file on the "data saving" tab 63.

[0054] Next, when the user presses the "Re-measurement" button 53 on the main screen, the inspection execution unit 22 performs a plurality of inspection processes on the classified images (images with labels) according to the processing flow, and calculates the measurement parameters and determines the threshold value in each inspection process (step S44). At this time, the current condition value and the current threshold value of the setting parameter stored in the parameter storage unit 23 are used.

[0055] After the re-measurement process in step S44 is completed, when switching to the "Data Details" tab 54 of the main screen, a chart representing the result of the re-measurement process is generated by the chart generation unit 252 (step S45), and the chart is displayed on the main screen by the UI output unit 253 (step S46). In the chart, for example, for the inspection processes to be adjusted (i.e., the inspection processes with defect type labels set), the "value of the measurement parameter calculated according to the current conditions of the set parameters from the OK image", the "value of the measurement parameter calculated according to the current conditions of the set parameters from the NG image", and the "current threshold value" can be depicted respectively. The specific form of the chart is not limited as long as it can clearly show the results of the inspection processes of multiple items at a glance. For example, a radar chart, a bar chart, a line chart, etc. can be used.

[0056] Figure 8 An example of a radar chart display is shown. Each axis of the radar chart corresponds to the inspection process with a defect type label set. For example, in the case where Figure 6 five defect type labels such as "circularity", "printing area", "printing presence / absence", "printing quality", and "printing height" are set as Figure 8 shown, the radar chart becomes a pentagon.

[0057] Eight points, namely "OK maximum value", "OK average value", "OK minimum value", "NG maximum value", "NG average value", "NG minimum value", "threshold upper limit value", and "threshold lower limit value", are depicted on each axis of the radar chart. The OK maximum value, OK average value, and OK minimum value respectively represent the maximum value, average value, and minimum value of the measurement parameter values calculated from multiple OK images. In addition, the NG maximum value, NG average value, and NG minimum value respectively represent the maximum value, average value, and minimum value of the measurement parameter values calculated from multiple NG images. In this way, by presenting the user with a chart in which these eight points, namely "OK maximum value", "OK average value", "OK minimum value", "NG maximum value", "NG average value", "NG minimum value", "threshold upper limit value", and "threshold lower limit value", are depicted in association with each inspection process (each axis), the tendency of the measurement parameter and the relationship with the threshold in each inspection process can be easily understood. Here, the measurement parameter values calculated from the OK image (OK maximum value, OK average value, OK minimum value), the measurement parameter values calculated from the NG image (NG maximum value, NG average value, NG minimum value), and the threshold (threshold upper limit value, threshold lower limit value) can be depicted in such a way that at least one of color or shape is different. In addition, the maximum value, average value, and minimum value can also be depicted in such a way that at least one of color or shape is different. By adopting such a display method, it is easy to visually distinguish OK, NG, and the threshold.

[0058] The scales of the axes of the radar chart can be standardized using the minimum and maximum values among the above 8 values. That is, the scales can be adjusted so that the minimum and maximum values among the 8 values reach both ends (the center and the outermost end) of the axes of the radar chart or near them. Thus, the plotting positions of the 8 points are extended as much as possible, and therefore the visual recognition (ease of observation) is improved.

[0059] On each axis of the radar chart, the "label name" and the "status of setting adjustment" of the defect type label are displayed. The status of setting adjustment has 3 modes: "setting completed", "threshold adjustment required", and "setting adjustment required". The chart generation unit 252 determines the status of setting adjustment based on the relationship between the value of the measurement parameter calculated from the OK image (hereinafter also simply referred to as the "OK image measurement value"), the value of the measurement parameter calculated from the NG image (hereinafter also simply referred to as the "NG image measurement value"), and the threshold value. In Figure 8 the example, the status of setting adjustment is represented by color icons (for example, green: setting completed, yellow: threshold adjustment required, red: setting adjustment required).

[0060] Here, "setting completed" is a state where the OK image measurement value and the NG image measurement value are separated, and the threshold value is set between the OK image measurement value and the NG image measurement value. If this state is obtained, it is possible to expect appropriate OK / NG determination in this inspection process, so it can be judged that the setting parameters and the setting of the threshold value related to this measurement parameter are completed (no adjustment is required). In Figure 8 the example, "circularity", "print quality", and "presence or absence of printing" are in the "setting completed" state.

[0061] "Threshold adjustment required" is a state where the OK image measurement value and the NG image measurement value are separated, but the threshold value is not set between the OK image measurement value and the NG image measurement value. In Figure 8 the example, "print height" is in the state of "threshold adjustment required". In this case, by adjusting the threshold value so that the threshold value is between the OK image measurement value and the NG image measurement value, it becomes the "setting completed" state.

[0062] "Setting adjustment required" is a state where the OK image measurement value and the NG image measurement value are not separated. In this case, since it cannot be solved only by adjusting the threshold value, adjustment of the setting parameters is required. In Figure 8 the example, "print area" is in the state of "setting adjustment required".

[0063] By displaying the state of the setting adjustment together with the graph in this way, for multiple inspection processes selected as the adjustment targets, the inspection processes with appropriate parameter settings (inspection processes that do not require adjustment) and the inspection processes that require parameter adjustment are distinguished. Furthermore, for the inspection processes that require parameter adjustment, the user can immediately and correctly judge whether it is only necessary to adjust the threshold value or the setting parameters must be adjusted. In addition, in the graph, not only the determination result (color icon) of the "state of setting adjustment" is displayed, but also the numerical relationship among the OK image measurement value, the NG image measurement value, and the threshold value is displayed. Therefore, it is possible to easily confirm the basis (evidence) for the determination of the "state of setting adjustment", and the credibility of the user can be improved. In addition, based on the numerical relationship among the OK image measurement value, the NG image measurement value, and the threshold value, the user can master how or to what extent to adjust the threshold value and the setting parameters, so that the subsequent parameter adjustment operation can be performed efficiently. Since it is possible to use not only one image but also multiple OK images and multiple NG images, and the "OK maximum value", "OK average value", "OK minimum value", "NG maximum value", "NG average value", and "NG minimum value" are shown in the graph, the user can comprehensively grasp the adjustment status of the parameters for multiple images, and the parameter adjustment operation can be performed efficiently.

[0064] When there are inspection processes (labels) for which it is determined that "threshold adjustment is required" or "setting adjustment is required" remaining (No in step S47), the user performs the adjustment operation of the threshold value and the adjustment operation of the setting parameters for this inspection process (step S48).

[0065] Refer to Figure 9 , and the operation in the case of "threshold adjustment is required" will be described. When the label name (for example, "printing height") displayed in the radar chart selected (pressed) Figure 8 is selected, an adjustment window for adjusting the parameters related to this label is displayed by the UI output unit 253 ( Figure 9 ).

[0066] The scatter diagram 90 and the histogram 91 show the details of the distribution of the OK image measurement values and the NG image measurement values related to the selected label. The horizontal axis of the scatter diagram 90 corresponds to each image (image No.), and the vertical axis corresponds to the measurement value (the value of the measurement parameter calculated from the image). The horizontal axis of the histogram 91 is the frequency (the number of images), and the vertical axis is the same as the vertical axis of the scatter diagram 90. Here, in the scatter diagram 90, the arrangement of the images on the horizontal axis can be determined in such a way that the group of OK images and the group of NG images are separated (in Figure 9In the example, the OK images are arranged first, and the NG images are arranged in the second half, and a line indicating the boundary between the OK image group and the NG image group is drawn). In addition, the scale on the vertical axis can be adjusted so that the minimum value and the maximum value of the measured values are appropriately separated. In addition, in the scatter diagram 90 and the histogram 91, the drawing colors can be made different between the OK images and the NG images. With such a design, it is easy to grasp the distribution, tendency of the OK image measurement values, and the distribution, tendency of the NG image measurement values.

[0067] In the text boxes 92B and 92U below the scatter diagram 90, the current set values of the threshold (lower limit) and the threshold (upper limit) are displayed. In addition, lines 93B and 93U indicating the threshold (lower limit) and the threshold (upper limit) are drawn in the scatter diagram 90 and the histogram 91.

[0068] Observation Figure 9 It can be seen that although the OK image measurement values and the NG image measurement values are sufficiently separated, both the OK image measurement values and the NG image measurement values fall within the OK range defined by the threshold (lower limit) and the threshold (upper limit). In this case, the user can adjust the threshold by changing the values in the text boxes 92B and 92U or by dragging the moving lines 93B and 93U. In Figure 9 In the example, by correcting the threshold (upper limit) to be between the OK image measurement value and the NG image measurement value, an appropriate determination result can be obtained. In addition, the threshold can also be adjusted automatically. When the "Auto Adjust" button 94 is pressed, an appropriate threshold is automatically calculated based on the OK image measurement values and the NG image measurement values. The threshold determination algorithm is arbitrary. For example, the threshold can also be determined in such a way that it is the midpoint between the representative value (average value, median value, minimum value, maximum value, etc.) of the OK image measurement values and the representative value (average value, median value, minimum value, maximum value, etc.) of the NG image measurement values. Or, a threshold determination algorithm such as discriminant analysis can also be used.

[0069] In addition, in Figure 9In this example, a dozen images are shown. However, in actual applications, it is also envisioned to use several hundred or several thousand images of this order of magnitude for parameter adjustment. If the number of images becomes large, it may lead to a decrease in the visual recognition of the scatter plot 90 and the histogram 91, resulting in overlooking images that have caused misjudgments or misinterpreting the appropriate configuration of the threshold value. In such a case, the "Adjust settings with misjudged images" button 95 is useful. If this button 95 is pressed, only the images that have caused misjudgments (i.e., OK images with measurement values deviating from the OK range and NG images with measurement values falling within the OK range) are extracted from the population of images, and only the information of the extracted misjudged images is displayed on the scatter plot 90 and the histogram 91. By switching to such a display, it is possible to observe the measurement values for multiple images while confirming the status of the measurement values for each image, making the problem areas clear, and thus enabling efficient and appropriate parameter adjustment.

[0070] The user can select one image by selecting a plotted point on the scatter plot 90 or by entering the image No. in the re-measurement image column 96. When an image is selected, the measurement value of the selected image is displayed on the scatter plot 90. In addition, the inspection process (re-measurement) of each processing unit is performed on the selected image, and its OK / NG determination result is displayed in the processing unit list 97, and the determination result and measurement value of the processing unit to be adjusted are displayed in the details display column 98. By using such a function, it is possible to easily confirm the measurement values and determination results for each image.

[0071] Refer to Figure 10 to describe the operations in the case of "Needs setting adjustment". Figure 10 This is an example of an adjustment window displayed when the "Printing area" label is selected in the Figure 8 radar chart.

[0072] Observing the scatter plot 90 and the histogram 91, it can be seen that the measurement values of OK images and NG images are not separated, and it cannot be solved only by adjusting the threshold value. In such a case, it is possible to focus on groups of OK images and NG images that obtain similar measurement values, or focus on images in OK images that obtain measurement values significantly different from other images, and adjust the set parameters. In the Figure 10 example, the OK image of No. 12 has the same measurement value as the NG image of No. 15 and is significantly smaller than other OK images in terms of measurement value. Therefore, it can be judged that adjusting the parameters corresponding to the OK image of No. 12 is a shortcut.

[0073] Therefore, the user selects the OK image of No. 12 on the scatter plot 90, or enters "12" in the re-measurement image column 96 to select the image of No. 12. Under the current conditions, "NG" is displayed in the determination result of the corresponding processing unit "area centroid", and measurement values such as "area: 0.0000" are displayed. Observing the scatter plot 90, the measurement values of the area in other OK images are approximately around 10,000. Therefore, it can be speculated that there is a problem with the set parameters used for calculating the area.

[0074] To correct the set parameters, select (press) the processing unit in the processing unit list 97. For example, when selecting "area centroid", as Figure 11 shown, a setting window for setting the set parameters of the area centroid is displayed. Since nothing is displayed in the preview on the right side of the setting window, it can be seen that the extraction of the area fails in the current color specification settings (hue, saturation, lightness). Therefore, while confirming the preview, the user adjusts the ranges of hue, saturation, and lightness so that the area to be inspected is exactly extracted. Figure 11 The lower part of shows that by changing the upper limit value of the lightness to 255, the area can be extracted. When the adjustment of the set parameters is completed, press the "OK button" to close the setting window.

[0075] As Figure 12 shown, when returning to the adjustment window again and pressing the "re-measure and execute together" button 99, the measurement parameters are recalculated using the adjusted set parameters. In the Figure 12 example, the measurement value of image No. 12 is 9631, and it can be seen that a value equivalent to that of other OK images can be obtained. If it is in this state, a threshold can be set to separate the measurement values of OK images and NG images. Therefore, continue to adjust the threshold to complete the adjustment operation.

[0076] In the steps described above, after adjusting the parameters for all of the labels of "threshold adjustment required" and "setting adjustment required", press the "re-measure and execute together" button 53 on the main screen to update the radar chart. As Figure 13 shown, if the status of all labels becomes "setting completed", the parameter adjustment operation is completed ( Figure 4 Yes in step S47 of ).

[0077] The above-described embodiments merely illustrate structural examples of the present invention. The present invention is not limited to the above specific embodiments, and various modifications can be made within the scope of its technical idea. For example, in the above embodiments, a radar chart is illustrated, but a bar chart, a line chart, etc. can also be used. In addition, instead of plotting representative values (maximum value, average value, minimum value) of OK image measurement values and NG image measurement values in a radar chart or the like, all the individual measurement values can be plotted. In addition, the UI and parameters illustrated in the above embodiments are merely examples, and can be appropriately modified. Further, in the above embodiments, a method in which the user himself / herself selects the inspection process to be adjusted is adopted, but the parameter adjustment assistance tool can also automatically select (set) the inspection process to be adjusted, or recommend to the user the inspection process that should be the adjustment target. For example, it is also possible to sequentially select N items of inspection processes from among the plurality of inspection processes included in the processing flow, which have a larger number of images with NG determination, as the investigation target (or recommendation target). Or, the inspection process to be adjusted can be determined in advance.

Claims

1. A parameter adjustment assistance device is used for an image sensor capable of performing multiple inspection processes on an image, and assists in adjusting parameters used in each inspection process. The parameter adjustment assistance device is characterized in that, the inspection process calculates the value of a prescribed measurement parameter based on the image, compares the value of the measurement parameter with a threshold value, and outputs a determination result of OK or NG; the parameters include setting parameters and the threshold value, and the setting parameters define the conditions when calculating the value of the measurement parameter; the parameter adjustment assistance device includes: an adjustment target setting unit that sets N items of inspection processes as adjustment targets from the multiple inspection processes, wherein, N is an integer of 2 or more; an image acquisition unit that acquires an OK image that should obtain an OK determination in all of the N items of inspection processes and an NG image that should obtain an NG determination in at least any one of the N items of inspection processes; a graph generation unit that generates the following graphs for each of the N items of inspection processes: In this graph, the value of the measurement parameter calculated according to the OK image under the current conditions of the setting parameters, that is, the OK image measurement value, the value of the measurement parameter calculated according to the NG image of this inspection process under the current conditions of the setting parameters, that is, the NG image measurement value, and the current threshold value are depicted; and an output unit that outputs the graph to a display device, the graph is a radar graph having N axes corresponding to the N items of inspection processes respectively, and the OK image measurement value, the NG image measurement value, and the threshold value of the corresponding inspection process are depicted on each axis.

2. The parameter adjustment assistance device according to claim 1, characterized in that, the OK image measurement value, the NG image measurement value, and the threshold value are depicted in the graph in such a way that at least one of color or shape is different.

3. The parameter adjustment assistance device according to claim 1, characterized in that, the graph generation unit judges whether the parameters of this inspection process need to be adjusted based on the relationship among the OK image measurement value, the NG image measurement value, and the threshold value in the inspection process of the adjustment target, and depicts the judgment result in the graph.

4. The parameter adjustment assistance device according to claim 3, characterized in that, the graph generation unit also judges which one of the threshold value and the setting parameters needs to be adjusted, and depicts the judgment result in the graph.

5. The parameter adjustment assistance device according to claim 4, characterized in that, the graph generation unit judges at least three states: no adjustment of the parameters is required, adjustment of the threshold value is required, and adjustment of the setting parameters is required, and the three states are depicted in the graph in different colors.

6. The parameter adjustment assistance device according to any one of claims 1 to 5, characterized in that, the output unit outputs an adjustment window for performing the parameter adjustment operation to the display device.

7. The parameter adjustment assistance device according to claim 6, characterized in that, In the adjustment window, a scatter diagram depicting the OK image measurement values of the respective OK images, the NG image measurement values of the respective NG images, and the threshold value is displayed.

8. The parameter adjustment assisting device according to claim 7, wherein, the adjustment window can switch the OK image measurement values and the NG image measurement values depicted in the scatter diagram to only the OK image measurement values of the OK images deviating from the OK range defined by the threshold value and the NG image measurement values of the NG images included in the OK range.

9. A parameter adjustment assisting method for assisting in the adjustment of parameters used in respective inspection processes for an image sensor capable of performing a plurality of inspection processes on an image, the parameter adjustment assisting method being characterized in that in the inspection process, the value of a prescribed measurement parameter is calculated based on the image, and the value of the measurement parameter is compared with a threshold value to output a determination result of OK or NG, the parameters include setting parameters and the threshold value, and the setting parameters define the conditions for calculating the value of the measurement parameter, the parameter adjustment assisting method includes the following steps: setting, from among the plurality of inspection processes, the inspection processes of N items to be adjusted, wherein, N is an integer of 2 or more; acquiring an OK image that should obtain an OK determination in all of the inspection processes of the N items and an NG image that should obtain an NG determination in at least any one of the inspection processes of the N items; generating, for each of the inspection processes of the N items, a chart in which the value of the measurement parameter calculated according to the OK image under the current conditions of the setting parameters, i.e., the OK image measurement value, the value of the measurement parameter calculated according to the NG image of the inspection process under the current conditions of the setting parameters, i.e., the NG image measurement value, and the current threshold value are depicted; and outputting the chart to a display device, the chart being a radar chart having N axes respectively corresponding to the inspection processes of the N items, and the OK image measurement value, the NG image measurement value, and the threshold value of the corresponding inspection process being depicted on each axis.

10. A recording medium that non-temporarily records a program for causing a processor to execute each step of the parameter adjustment assisting method according to claim 9.

Images