Parameter adjustment support device and parameter adjustment support method

Abstract

To provide a technique for simplifying a parameter adjustment work by a user, concerning an image sensor with a plurality of inspection processes.SOLUTION: A parameter adjustment support device comprises: an adjustment object setting unit which sets N items of inspection processes to be adjusted, from among a plurality of inspection processes; an image acquisition unit which acquires an OK image and an NG image; a graph generation unit which generates, with respect to each of the N items of inspection processes, a graph drawing an OK image measurement value being a value of a measurement parameter calculated in accordance with the present condition of the setting parameter from the OK image, an NG image measurement value being the value of the measurement parameter calculated in accordance with the present condition of the setting parameter from the NG image of the inspection process, and a present threshold; and an output unit which outputs the graph to a display device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a technique for assisting in the adjustment of parameters used in the inspection process of an image sensor. [Background technology]

[0002] Systems known as image sensors are widely used in factory production lines and the like to automate or reduce the labor required for inspecting products (see Patent Document 1). With image sensors, a series of inspection flows are configured by combining multiple inspection processes. To obtain accurate inspection results using an image sensor, it is necessary to appropriately set the parameters used in each inspection process. However, until now, the only way to do this was to set the parameters for each inspection process individually, making it difficult to adjust (finish) the parameters. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-015706 Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made in consideration of the above-described circumstances, and its purpose is to provide a technique for simplifying the parameter adjustment work by a user for an image sensor having multiple inspection processes. [Means for solving the problem]

[0005] The present disclosure relates to an image sensor capable of executing a plurality of inspection processes on an image, and to a parameter adjustment support device for supporting adjustment of parameters used in each inspection process, wherein the inspection process calculates values ​​of predetermined measurement parameters from an image, compares the values ​​of the measurement parameters with thresholds, and outputs a judgment result of OK or NG, the parameters include setting parameters that specify conditions for calculating the values ​​of the measurement parameters, and the thresholds, and the adjustment target setting unit sets N items (N is an integer of 2 or more) of inspection processes to be adjusted from among the plurality of inspection processes, and a parameter adjustment support device that supports adjustment of parameters used in each inspection process when an OK judgment is obtained in all of the N items of inspection processes. and an NG image that should be judged as NG in at least one of the N inspection processes; a graph generation unit that generates a graph depicting, for each of the N inspection processes, an OK image measurement value, which is the value of the measurement parameter calculated from the OK image according to the current conditions of the setting parameters, an NG image measurement value, which is the value of the measurement parameter calculated from the NG image of the inspection process according to the current conditions of the setting parameters, and the current threshold value; and an output unit that outputs the graph to a display device.

[0006] The OK image measurement value, the NG image measurement value, and the threshold value may be plotted on the graph in a manner in which at least one of the colors or shapes is different.

[0007] The graph generation unit may determine whether or not the parameters of the inspection process to be adjusted are necessary based on the relationship between the OK image measurement value, the NG image measurement value, and the threshold value in the inspection process to be adjusted, and plot the result of the determination on the graph.

[0008] The graph generating unit further determines whether adjustment of the threshold value or the setting parameter is necessary. The results of the determination may be plotted on the graph.

[0009] The graph generation unit may determine at least three states: no adjustment of the parameter is required, adjustment of the threshold is required, and adjustment of the setting parameter is required, and the three states may be drawn on the graph in different colors.

[0010] The output unit may output an adjustment window for performing an adjustment operation of the parameter to the display device.

[0011] The adjustment window may display a scatter diagram plotting the OK image measurement value of each OK image, the NG image measurement value of each NG image, and the threshold value.

[0012] The adjustment window may be capable of switching the OK image measurement values ​​and NG image measurement values ​​to be drawn on the scatter plot to only the OK image measurement values ​​of OK images that fall outside the OK range defined by the threshold value and the NG image measurement values ​​of NG images that fall within the OK range.

[0013] The present disclosure relates to an image sensor capable of performing multiple inspection processes on an image, and includes a parameter adjustment support method for supporting the adjustment of parameters used in each inspection process, wherein the inspection process calculates the value of a predetermined measurement parameter from an image, compares the value of the measurement parameter with a threshold value, and outputs a judgment result of OK or NG, the parameters including setting parameters that specify the conditions for calculating the value of the measurement parameter and the threshold value, and the parameter adjustment support method includes the steps of: setting N items (N is an integer of 2 or more) of inspection processes to be adjusted from the multiple inspection processes; acquiring OK images that should be judged OK in all of the N items of inspection processes and NG images that should be judged NG in at least one of the N items of inspection processes; generating a graph for each of the N items of inspection processes, depicting OK image measurement values, which are the values ​​of the measurement parameters calculated from the OK images according to the current conditions of the setting parameters, NG image measurement values, which are the values ​​of the measurement parameters calculated from the NG images of the inspection process according to the current conditions of the setting parameters, and the current threshold value; and outputting the graph to a display device.

[0014] The present disclosure includes a program for causing a processor to execute each step of the parameter adjustment support method.

[0015] The present invention may be understood as a parameter adjustment support device having at least some of the above means, or as an image sensor equipped with the same. The present invention may also be understood as a parameter adjustment support method or an image sensor control method including at least some of the above processing, or as a program for realizing such a method or a recording medium on which such a program is non-temporarily recorded. The above means and processing may be combined with each other as much as possible to constitute the present invention. [Effects of the Invention]

[0016] According to the present invention, it is possible to simplify the parameter adjustment work by the user for an image sensor having a plurality of inspection processes. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a diagram schematically illustrating the appearance of an image sensor. [Figure 2] FIG. 2 is a functional block diagram showing an example of the configuration of the inspection function and the parameter adjustment support function. [Figure 3] FIG. 3 is a diagram for explaining the processing flow of the inspection and the setting of parameters. [Figure 4] FIG. 4 is a flowchart showing the flow of the parameter adjustment work. [Figure 5] Figure 5 shows an example of the main screen of the parameter adjustment support tool. [Figure 6] Figure 6 shows an example of the Label Settings tab of the Image Classification window. [Figure 7] Figure 7 is an example of the Labeling tab of the Image Classification window. [Figure 8] Figure 8 is an example of a radar chart display. [Figure 9] FIG. 9 shows an example of adjusting the threshold value using an adjustment window. [Figure 10] FIG. 10 shows an example of adjusting setting parameters using an adjustment window. [Figure 11] FIG. 11 shows an example of adjusting the setting parameters for the areal center of gravity. [Figure 12] FIG. 12 shows an example of adjusting setting parameters using an adjustment window. [Figure 13] FIG. 13 is an example of a radar chart after parameter adjustment is complete. DETAILED DESCRIPTION OF THE INVENTION

[0018] An image sensor according to an embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a diagram schematically showing the appearance of the image sensor.

[0019] The image sensor 1 is a device that is installed, for example, on a production line PL of a factory and is used for various processes that use images. The image sensor 1 is also called an image processing system, a vision sensor, a vision system, etc.

[0020] The image sensor 1 includes, as hardware resources, an imaging device 10, an image processing device 11, a display device 12, and an input device 13. The imaging device 10, the display device 12, and the input device 13 are connected to an input / output I / F of the image processing device 11. Note that, although the image sensor 1 in FIG. 1 has a structure in which the imaging device 10 and the image processing device 11 are separate, the image sensor may be an integrated processing type in which the imaging device and the image processing device are integrated.

[0021] The imaging device 10 is a device that includes an illumination unit, a lens unit, an imaging unit, etc., and is also called an industrial camera. The illumination unit is a device that illuminates a subject (inspection object O) and is composed of, for example, an LED light source. The lens unit is an optical device that forms an optical image of the subject on the imaging unit and uses, for example, an optical system with functions such as focus adjustment, aperture, and zoom. The imaging unit is a device that generates and outputs image data by photoelectric conversion and is composed of, for example, an imaging element such as a CCD or CMOS sensor.

[0022] The image processing device 11 has the following main functions: capturing image data from the imaging device 10, image processing of the image data, inspection processing based on the results of the image processing, sending and receiving data to and from an external device via an input / output I / F, generating data to be output to an external device, processing data received from an external device, and controlling the imaging device 10 and the input / output I / F. The image processing device 11 can be configured, for example, by an image processing computer equipped with a processor (CPU, GPU, etc.), memory (RAM, ROM, etc.), storage (non-volatile storage device such as a hard disk or solid-state drive), an input / output I / F, etc. The functions and processing of the image processing device 11, which will be described later, can be realized by loading a program stored in the storage into memory and executing it with a processor. However, some or all of the functions of the image processing device 11 can be realized by using an ASIC (Application Specific Integrated Circuit) or FPGA. This can be realized by using a Field Programmable Gate Array (FPGA) or by using other computers or clusters. It may also be executed on a cloud server.

[0023] FIG. 2 is a functional block diagram showing an example of the configuration of the inspection function and parameter adjustment support function in the image processing device 11.

[0024] The image processing device 11 includes an image input unit 20, an image storage unit 21, an inspection execution unit 22, and a parameter The system includes a storage unit 23, an inspection process setting unit 24, and a parameter adjustment support unit 25. The image input unit 20 imports image data from the imaging device 10. The imported image data is stored in the image storage unit 21. The inspection execution unit 22 performs inspections on the image data for multiple items and outputs a judgment result of OK / NG for each item. The inspection judgment results may be stored in association with the image data. The parameter storage unit 23 stores setting values ​​of parameters used in each inspection process. The inspection execution unit 22 references the setting values ​​of parameters from the parameter storage unit 23 when executing an inspection process. The inspection process setting unit 24 provides a user interface (UI) for setting the processing flow of the inspection performed by the inspection execution unit 22 and parameters used in each inspection process. The parameter adjustment support unit 25 provides tools to support the user in adjusting (driving in) parameters to appropriate values. The parameter adjustment support unit 25 includes an image acquisition unit 250, an adjustment target setting unit 251, a graph generation unit 252, and a UI output unit 253. Details of these functions will be described later.

[0025] (Inspection process settings) An example of a procedure for setting the processing flow and parameters of an inspection using the UI of the inspection processing setting unit 24 will be described with reference to FIG.

[0026] The image processing device 11 is equipped with multiple types of modules (hereinafter referred to as "processing units") that perform specific processes. The processing units can be broadly divided into those related to inspection processes and those related to processes other than inspection processes. For inspection processes, various processing units are provided, such as a "shape search" that checks whether an area of ​​a specific shape exists within an image, an "area centroid" that checks the area and position of an area of ​​a specific color, and an "edge position" that checks the position of an object's edge. For processing units other than inspection processes, for example, a "camera image input" that imports a captured image from the imaging device 10 and a "color extraction filter" that extracts a specific color range from an image are provided. As shown in Figure 3, the user can set the desired inspection by appropriately combining processing units using the UI of the inspection process setting unit 24. A series of processes created by combining multiple processing units is called a "processing flow."

[0027] The processing units of the inspection process share a common design: they calculate the values ​​of specified measurement parameters based on features extracted from an image, compare the values ​​of the measurement parameters with thresholds, and output a judgment result of either "OK" or "NG." For example, in the case of the aforementioned "shape search," measurement parameters calculated include the similarity (correlation value) between the registered model and the model detected in the image, the position (XY coordinates) where the model was detected, and the tilt of the detected model (relative angle to the registered model). Thresholds (upper and lower thresholds defining the OK range) are then set for each of the similarity, position, and tilt. If all values ​​of similarity, position, and tilt are within the OK range, the judgment result is "OK." If any one value is outside the OK range, the judgment result is "NG." In the case of "area centroid," measurement parameters calculated include the area of ​​the region and the position of the center of gravity (XY coordinates). Thresholds are then set for each of the area and center of gravity. If both the area and center of gravity are within the OK range, the judgment result is "OK." If either is outside the OK range, the judgment result is "NG." The "shape search" and "area centroid" examples given here are designed to make OK / NG judgments for multiple types of measurement parameters, i.e., to perform inspection processing for multiple items with one processing unit, but there are also processing units that only have one type of measurement parameter (inspection processing for one item).

[0028] The indices used as measurement parameters in each processing unit are predefined and cannot be changed by the user. However, the user can set various conditions (hereafter referred to as "setting parameters") when calculating the values ​​of the measurement parameters and the threshold values ​​used for judgment. For example, the setting parameters for "shape search" are the search target, These include the model to be evaluated, the model's position, and the feature points on the model that are of interest when calculating similarity. As shown in Figure 3, the setting parameters for "area centroid" include the color conditions (hue, saturation, and brightness ranges) extracted as the area for calculating the area and centroid. The values ​​of the measurement parameters calculated from the image vary greatly depending on the settings of these setting parameters. Furthermore, whether the thresholds are set appropriately affects the accuracy of the OK / NG judgment and the reliability of the judgment results.

[0029] Therefore, after creating a processing flow by combining processing units, the user must appropriately set the setting parameters and thresholds of each processing unit so as to obtain the desired inspection performance (determination result). However, the task of adjusting (finding) the setting parameters and thresholds to appropriate values ​​requires advanced skills and know-how. Moreover, when a processing flow includes a large number of processing units, or when each processing unit has a large number of setting parameters and thresholds, it is not easy to adjust the setting parameters and thresholds in a balanced manner so as to obtain appropriate determination results for all of them.

[0030] Therefore, in the image sensor 1 of this embodiment, the parameter adjustment support unit 25 provides a parameter adjustment support tool to support the adjustment work of setting parameters and thresholds (this parameter adjustment support tool is an example of a parameter adjustment support device according to the present invention).

[0031] (Parameter adjustment support tool) The parameter adjustment support tool of this embodiment will be described in detail with reference to Fig. 4 to Fig. 12. Fig. 4 is a flowchart showing the flow of parameter adjustment work using the parameter adjustment support tool. Fig. 5 is an example of the main screen of the parameter adjustment support tool.

[0032] When the parameter adjustment support tool (hereinafter also simply referred to as "tool") is started, the main screen of Fig. 5 is displayed by the UI output unit 253 (step S40). The main screen displays a list 50 of multiple processing units that make up the processing flow.

[0033] First, the user specifies the location where the images (sample images) to be used in the adjustment process are stored in the “Remeasurement Target Images” field 51 on the main screen (step S41). If “Main Unit Logging Images” is specified, images stored in the image storage unit 21 (e.g., the RAM disk or internal storage of the image sensor 1) are used. If “Specified Folder Images” is specified, images stored in external storage can be used. Once the storage location is specified, the image acquisition unit 250 captures the sample image. While adjustments can be performed using only one sample image, it is preferable to use multiple sample images obtained by photographing different inspection targets. It is also preferable that the multiple sample images include both images of non-defective and defective products, and that the images of defective products include multiple types of defects (i.e., images of inspection process items that result in different NG judgments). Simultaneous parameter adjustments for multiple inspection process items using a variety of images allows for efficient parameter refinement.

[0034] Next, when the user presses the "Image Classification" button 52 on the main screen, the adjustment target setting unit 251 displays the image classification window shown in FIG. 6. The user specifies the inspection processes to be adjusted in the "Label Setting" tab 60 of the image classification window and sets labels for them (step S42). Specifically, the number of items for the inspection processes to be adjusted is input in the "Number of Settings" field 61. The number of items N can be set, for example, in the range of 3 to 20. When the number of items N is input, N labels are generated, and the user sets "Label Name", "Target Unit", and "Measurement Parameters" for each label. The "Label Name" can be set arbitrarily. The "Target Unit" is the name of the processing unit that performs the inspection process to be adjusted. Specify a name, and then specify the name of the measurement parameter used in the inspection process to be adjusted in "Measurement parameter." Figure 6 shows an example in which five labels have been set: "Circularity," "Print area," "Presence or absence of printing," "Print quality," and "Print height."

[0035] After completing the label setting, the user assigns labels to the images using the “Labeling” tab 62 (step S43). Selecting the “Labeling” tab 62 in the image classification window switches to the UI shown in FIG. 7. The UI of the “Labeling” tab 62 includes a label display 71, a preview button 72, a file list 73, and a label assignment field 74. The label display 71 displays a list of “judgment labels” and “defect type labels.” The judgment labels represent the judgment results of the images, and three types of labels are pre-defined: “OK,” “NG,” and “Classification pending.” The defect type labels are labels defined by the user using the “Label Setting” tab 60. The file list 73 displays a list of file names of the images specified in step S41. If multiple images are specified in step S41, the file list 73 displays the file names of the multiple images. Pressing the preview button 72 displays a preview of the image selected in the file list 73. The user first assigns judgment labels while checking the preview image. An "OK" label is assigned to an image that should be judged as OK by all the inspection processes to be adjusted. An "NG" label is assigned to an image that should be judged as NG by any of the inspection processes. An "Classification Pending" label is assigned to an image that cannot be judged as either OK or NG. The label assignment field 74 has a cell corresponding to each label, and pressing that cell toggles between assigning and removing a label. Images that have been assigned an "NG" label are then assigned a defect type label. In other words, information about which inspection process should result in an NG judgment is associated with the image. It is also possible to assign multiple defect type labels to a single image. It is preferable that each label be displayed in a different color in the label display 71 and the label assignment field 74. This makes it easier to see the labels assigned to each image, facilitating the label assignment process and preventing operational errors such as assigning incorrect labels. By performing the above steps, multiple OK images and multiple NG images can be prepared.

[0036] Once labeling (i.e., image classification) is complete, pressing the OK button will return you to the main screen. If you wish to save the settings made in the "Label Settings" tab 60 or the "Labeling" tab 62, you can save them to a file in the "Save Data" tab 63.

[0037] Next, when the user presses the "Execute batch remeasurement" button 53 on the main screen, the inspection execution unit 22 performs multiple inspection processes on each of the classified images (labeled images) according to the processing flow, and calculates measurement parameters and performs threshold judgment for each inspection process (step S44). At this time, the current condition values ​​and current thresholds of the setting parameters stored in the parameter storage unit 23 are used.

[0038] After the remeasurement process in step S44 is completed, switching to the "Data Details" tab 54 on the main screen causes the graph generation unit 252 to generate a graph showing the results of the remeasurement process (step S45), and the UI output unit 253 to display the graph on the main screen (step S46). The graph may, for example, depict the "measurement parameter values ​​calculated from a OK image according to the current conditions of the set parameters," "measurement parameter values ​​calculated from a NG image according to the current conditions of the set parameters," and "current threshold value" for each of the inspection processes to be adjusted (i.e., the inspection processes for which defect class labels have been set). The specific format of the graph is not important, but it is preferable that the results of inspection processes for multiple items can be confirmed at a glance. For example, a radar chart, a bar graph, a line graph, or the like may be used.

[0039] An example of a radar chart display is shown in Figure 8. Each axis of the radar chart represents the defect This supports inspection processes that set type labels. For example, if five defect type labels, "circularity," "print area," "presence of printing," "print quality," and "print height," are set as shown in Fig. 6, the radar chart will be pentagonal, as shown in Fig. 8.

[0040] Eight points are plotted on each axis of the radar chart: "OK maximum value," "OK average value," "OK minimum value," "NG maximum value," "NG average value," "NG minimum value," "upper threshold value," and "lower threshold value." The OK maximum value, OK average value, and OK minimum value represent the maximum, average, and minimum values ​​of the measurement parameter calculated from multiple OK images, respectively. The NG maximum value, NG average value, and NG minimum value represent the maximum, average, and minimum values ​​of the measurement parameter calculated from multiple NG images, respectively. By presenting the user with a graph in which the eight points - "OK maximum value," "OK average value," "OK minimum value," "NG maximum value," "NG average value," "NG minimum value," "upper threshold value," and "lower threshold value" - are associated with each inspection process (each axis), the user can easily understand the trends of the measurement parameters in each inspection process and their relationship with the thresholds. Here, the values ​​of the measurement parameters calculated from the OK image (OK maximum value, OK average value, OK minimum value), the values ​​of the measurement parameters calculated from the NG image (NG maximum value, NG average value, NG minimum value), and the thresholds (upper threshold value, lower threshold value) may be depicted in a manner in which at least one of the colors or shapes differ. Also, the maximum value, average value, and minimum value may be depicted in a manner in which at least one of the colors or shapes differ. By adopting such a display manner, it becomes easier to visually distinguish between OK, NG, and the threshold value.

[0041] The scale of each axis of the radar chart should be normalized using the minimum and maximum of the eight values ​​mentioned above. In other words, the scale should be adjusted so that the minimum and maximum of the eight values ​​are at or near the ends of each axis of the radar chart (the center and outermost edges of the chart). This spreads the plotting positions of the eight points as far as possible, improving visibility (ease of viewing).

[0042] Each axis of the radar chart displays the "label name" of the defect type label and the "status of setting adjustment." There are three statuses of setting adjustment: "setting complete," "threshold adjustment required," and "setting adjustment required." The graph generation unit 252 determines the status of setting adjustment based on the relationship between the value of the measurement parameter calculated from the OK image (hereinafter also simply referred to as "OK image measurement value") and the value of the measurement parameter calculated from the NG image (hereinafter also simply referred to as "NG image measurement value") and the threshold. In the example of FIG. 8, the status of setting adjustment is represented by a color icon (e.g., green: setting complete, yellow: threshold adjustment required, red: setting adjustment required).

[0043] Here, "settings complete" refers to a state in which the OK image measurement values ​​and NG image measurement values ​​are separated and the threshold is set between the OK image measurement values ​​and NG image measurement values. If this state is achieved, it can be expected that an appropriate OK / NG judgment will be made in the inspection process, so it can be determined that the setting of the setting parameters and thresholds for the measurement parameters is complete (no adjustment required). In the example of Figure 8, three items, "circularity," "print quality," and "presence of printing," are in the "settings complete" state.

[0044] "Threshold adjustment required" is a state in which the OK image measurement value and the NG image measurement value are separated, but the threshold is not set between the OK image measurement value and the NG image measurement value. In the example in Figure 8, "print height" is in the "threshold adjustment required" state. In this case, adjusting the threshold so that it is between the OK image measurement value and the NG image measurement value will result in the "setting complete" state.

[0045] "Settings adjustment required" means that the OK image measurement values ​​and NG image measurement values ​​are not separated. In this case, adjusting the threshold value alone will not solve the problem, so the setting parameters must be adjusted. In the example in Figure 8, "Print area" is in the "Settings adjustment required" state.

[0046] By displaying the setting adjustment status along with a graph, users can distinguish between parameters for multiple inspection processes selected for adjustment that have appropriate settings (no adjustment necessary) and those that require parameter adjustment. Furthermore, for parameters that require parameter adjustment, users can quickly and accurately determine whether adjusting the thresholds is sufficient or whether the setting parameters must be adjusted. The graph also displays not only the "setting adjustment status" judgment results (color icons) but also the numerical relationship between the OK image measurement values, the NG image measurement values, and the thresholds. This allows users to easily confirm the basis (evidence) for the "setting adjustment status" judgment, enhancing user satisfaction. Furthermore, users can understand how and to what extent thresholds and setting parameters should be adjusted from the numerical relationship between the OK image measurement values, the NG image measurement values, and the thresholds, thereby enabling efficient parameter adjustment work. The graph can display the "OK maximum value," "OK average value," "OK minimum value," "NG maximum value," "NG average value," and "NG minimum value" for multiple OK and NG images, rather than just a single image, allowing users to grasp the overall parameter adjustment status for multiple images and efficiently perform parameter adjustment work.

[0047] If there are any inspection processes (labels) that have been determined to require "threshold adjustment" or "setting adjustment" (NO in step S47), the user adjusts the thresholds or setting parameters of the inspection processes (step S48).

[0048] The operation when "threshold adjustment is required" will be described with reference to Fig. 9. When a label name (for example, "print height") displayed on the radar chart in Fig. 8 is selected (pressed), the UI output unit 253 displays an adjustment window for adjusting parameters related to the label (Fig. 9).

[0049] The scatter plot 90 and histogram 91 show detailed distributions of OK and NG image measurement values ​​for the selected label. The horizontal axis of the scatter plot 90 corresponds to individual images (image numbers), and the vertical axis corresponds to measurement values ​​(measurement parameter values ​​calculated from the images). The horizontal axis of the histogram 91 represents frequency (number of images), and the vertical axis is the same as the vertical axis of the scatter plot 90. In the scatter plot 90, it is recommended to arrange the images on the horizontal axis so that OK and NG image groups are separated (in the example of Figure 9, OK images are arranged first and NG images are arranged later, and a line is drawn to indicate the boundary between the OK and NG image groups). It is also recommended to adjust the scale of the vertical axis so that the minimum and maximum measurement values ​​are adequately separated. It is also recommended to use different colors for OK and NG images in the scatter plot 90 and histogram 91. This approach makes it easier to understand the distribution and trends of OK and NG image measurement values.

[0050] The current settings of the threshold (lower limit) and threshold (upper limit) are displayed in text boxes 92B and 92U below the scatter plot 90. In addition, lines 93B and 93U indicating the threshold (lower limit) and threshold (upper limit) are drawn on the scatter plot 90 and histogram 91.

[0051] As can be seen from Figure 9, although the OK image measurement values ​​and the NG image measurement values ​​are well separated, both the OK image measurement values ​​and the NG image measurement values ​​fall between the OK range defined by the threshold (lower limit) and threshold (upper limit). In such cases, the user can adjust the threshold by changing the values ​​in the text boxes 92B and 92U or by dragging and moving the lines 93B and 93U. In the example of Figure 9, correcting the threshold (upper limit) so that it is between the OK image measurement values ​​and the NG image measurement values ​​will result in an appropriate judgment result. Note that the threshold can also be adjusted automatically. Pressing the "Auto Adjust" button 94 will automatically calculate an appropriate threshold based on the OK image measurement values ​​and the NG image measurement values. Threshold The algorithm for determining the threshold is arbitrary. For example, the threshold may be determined to be exactly midway between the representative value (average value, median value, minimum value, maximum value, etc.) of the OK image measurement values ​​and the representative value (average value, median value, minimum value, maximum value, etc.) of the NG image measurement values. Alternatively, a threshold determination algorithm such as discriminant analysis may be used.

[0052] While Figure 9 shows an example of a dozen or so images, in actual operation, it is expected that parameter adjustment will be performed using hundreds or even thousands of images. A huge number of images can reduce the visibility of the scatter plot 90 and histogram 91, potentially leading to overlooking images with misjudgments or misjudging the appropriate placement of thresholds. In such cases, the "Adjust Settings with Misjudged Images" button 95 is useful. Pressing this button 95 extracts only images with misjudgments (i.e., OK images with measurement values ​​outside the OK range and NG images with measurement values ​​within the OK range) from the image population, and only information about the extracted misjudgments is displayed in the scatter plot 90 and histogram 91. Switching to this display allows you to check the status of measurement values ​​for each image while viewing the measurement values ​​for multiple images, clearly identifying problem areas and enabling efficient and appropriate parameter adjustment.

[0053] The user can select one image by selecting a plot point on the scatter diagram 90 or by entering an image number in the remeasurement image field 96. Once an image is selected, the measurement values ​​of the selected image are displayed in the scatter diagram 90. Furthermore, inspection processing (remeasurement) is performed for each processing unit on the selected image, and the OK / NG judgment results are displayed in the processing unit list 97, while the judgment results and measurement values ​​of the processing unit to be adjusted are displayed in the detailed display field 98. By using these functions, it is easy to check the measurement values ​​and judgment results for each image.

[0054] The operation when "setting adjustment is required" will be described with reference to Fig. 10. Fig. 10 is an example of an adjustment window that is displayed when the "print area" label is selected in the radar chart of Fig. 8.

[0055] Looking at the scatter plot 90 and histogram 91, we can see that the measurement values ​​for OK and NG images are not separated, and that adjusting the threshold alone will not solve the problem. In such cases, it is a good idea to focus on a group of OK and NG images that have similar measurement values, or on an image among the OK images that has a measurement value that is significantly different from the others, and adjust the setting parameters accordingly. In the example of Figure 10, the OK image No. 12 has the same measurement value as the NG image No. 15, and its measurement value is clearly smaller than those of the other OK images. Therefore, it can be concluded that adjusting the parameters corresponding to the OK image No. 12 is the quickest solution.

[0056] Therefore, the user selects the OK image No. 12 on the scatter plot 90, or enters "12" in the remeasurement image field 96 and selects the No. 12 image. Under the current conditions, the judgment result for the corresponding processing unit "area centroid" is displayed as "NG," and the measurement value is displayed as "Area: 0.0000." Looking at the scatter plot 90, the measurement values ​​for the areas of the other OK images are roughly 10,000, so it can be inferred that there is a problem with the setting parameters for calculating the area.

[0057] To modify the setting parameters, simply select (press) a processing unit in the processing unit list 97. For example, if "area centroid" is selected, a setting window for setting the setting parameters for the area centroid will be displayed, as shown in Figure 11. Since nothing is displayed in the preview on the right side of the setting window, it can be seen that the current color specification settings (hue, saturation, brightness) have failed to extract the area. Therefore, while checking the preview, the user can adjust the ranges of hue, saturation, and brightness so that the area they want to inspect is extracted exactly. The bottom part of Figure 11 shows that the area can now be extracted by changing the upper brightness limit to 255. Once you have finished adjusting the setting parameters, press the "OK button" to close the setting window.

[0058] As shown in Figure 12, if you return to the adjustment window and press the "Execute Batch Remeasurement" button 99, the measurement parameters will be recalculated using the adjusted setting parameters. In the example of Figure 12, the measurement value for image No. 12 is 9631, which is equivalent to the value obtained for the other OK images. In this state, it is possible to set a threshold value that will separate the OK image measurement values ​​from the NG image measurement values. Therefore, you can continue by adjusting the threshold value to complete the adjustment work.

[0059] After adjusting the parameters for all labels that "need threshold adjustment" and "need setting adjustment" using the procedure described above, press the "Execute batch remeasurement" button 53 on the main screen to update the radar chart. If the status of all labels is "Settings complete" as shown in Figure 13, the parameter adjustment work is complete (YES in step S47 of Figure 4).

[0060] The above-described embodiment merely exemplifies a configuration example of the present invention. The present invention is not limited to the above-described specific embodiment, and various modifications are possible within the scope of the technical concept. For example, while the above-described embodiment illustrates a radar chart, a bar graph, a line graph, or the like may also be used. Furthermore, instead of plotting representative values ​​(maximum, average, and minimum) of OK and NG image measurement values ​​on a radar chart, all individual measurement values ​​may be plotted. Furthermore, the UI and parameters illustrated in the above-described embodiment are merely exemplary and may be modified as appropriate. Furthermore, the above-described embodiment employs a method in which the user selects the inspection process to be adjusted. However, a parameter adjustment support tool may automatically select (set) the inspection process to be adjusted or recommend the inspection process to be adjusted to the user. For example, of multiple inspection processes included in a processing flow, N inspection processes may be selected as the inspection targets (or the recommendation targets) in descending order of the number of NG-rated images. Alternatively, the inspection processes to be adjusted may be predetermined.

[0061] <Additional Notes> 1. A parameter adjustment support device for supporting adjustment of parameters used in each inspection process of an image sensor capable of performing multiple inspection processes on an image, comprising: The inspection process calculates values ​​of predetermined measurement parameters from the image, compares the values ​​of the measurement parameters with thresholds, and outputs a judgment result of OK or NG; The parameters include setting parameters that define conditions for calculating the values ​​of the measurement parameters and the threshold values, an adjustment target setting unit that sets N items (N is an integer equal to or greater than 2) of inspection processes to be adjusted from among the plurality of inspection processes; an image acquisition unit that acquires OK images that should be judged as OK in all of the N inspection processes and NG images that should be judged as NG in at least one of the N inspection processes; a graph generating unit that generates a graph depicting, for each of the N inspection processes, OK image measurement values, which are values ​​of the measurement parameters calculated from an OK image in accordance with the current conditions of the setting parameters, NG image measurement values, which are values ​​of the measurement parameters calculated from an NG image of the inspection process in accordance with the current conditions of the setting parameters, and the current threshold value; an output unit that outputs the graph to a display device; A parameter adjustment support device comprising:

[0062] 2. A parameter adjustment support method for supporting adjustment of parameters used in each inspection process for an image sensor capable of performing multiple inspection processes on an image, comprising: The inspection process calculates values ​​of predetermined measurement parameters from the image, compares the values ​​of the measurement parameters with thresholds, and outputs a judgment result of OK or NG; The parameters include setting parameters that define conditions for calculating the values ​​of the measurement parameters and the threshold values, A step of setting inspection processes for N items (N is an integer of 2 or more) to be adjusted from among the plurality of inspection processes; acquiring an OK image that should be judged as OK in all of the N inspection processes and an NG image that should be judged as NG in at least one of the N inspection processes; generating a graph plotting OK image measurement values, which are values ​​of the measurement parameters calculated from OK images according to the current conditions of the setting parameters, NG image measurement values, which are values ​​of the measurement parameters calculated from NG images of the inspection processes according to the current conditions of the setting parameters, and the current threshold values ​​for each of the N inspection processes; outputting the graph to a display device; A parameter adjustment support method comprising: [Explanation of symbols]

[0063] 1: Image sensor 10: Imaging device 11: Image processing device 12:Display device 13: Input device

Claims

1. A parameter adjustment support device for supporting adjustment of parameters used in each inspection process of an image sensor capable of performing a plurality of inspection processes on an image, comprising: The inspection process calculates values ​​of predetermined measurement parameters from the image, compares the values ​​of the measurement parameters with thresholds, and outputs a judgment result of OK or NG; The parameters include setting parameters that define conditions for calculating the values ​​of the measurement parameters and the threshold values, an adjustment target setting unit that sets N items (N is an integer of 3 or more) of inspection processes to be adjusted from the plurality of inspection processes; an image acquisition unit that acquires OK images that should be judged as OK in all of the N inspection processes and NG images that should be judged as NG in at least one of the N inspection processes; a graph generating unit that generates a graph plotting OK image measurement values, which are values ​​of the measurement parameters calculated from OK images in accordance with the current conditions of the setting parameters, NG image measurement values, which are values ​​of the measurement parameters calculated from NG images in accordance with the current conditions of the setting parameters, and the current threshold values ​​for each of the N inspection processes; an output unit that outputs the graph to a display device; Equipped with The parameter adjustment support device is characterized in that the graph is a radar chart having N axes corresponding to the N items of inspection processing, and on each axis, OK image measurement values, NG image measurement values, and threshold values ​​for the corresponding inspection processing are drawn.

2. The OK image measurement value, the NG image measurement value, and the threshold value are plotted on the graph in a manner in which at least one of the colors or shapes is different.

2. The parameter adjustment support device according to claim 1.

3. The graph generating unit determines whether the parameters of the inspection process to be adjusted are necessary based on the relationship between the OK image measurement value, the NG image measurement value, and the threshold value in the inspection process to be adjusted. The result of the determination is plotted on the graph.

3. The parameter adjustment support device according to claim 1 or 2.

4. The graph generating unit further determines whether adjustment of the threshold value or the setting parameter is necessary, and plots the result of the determination on the graph.

4. The parameter adjustment support device according to claim 3.

5. the graph generation unit determines at least three states: that the parameter does not need to be adjusted; that the threshold needs to be adjusted; and that the setting parameter needs to be adjusted; The three states are depicted in different colors on the graph.

5. The parameter adjustment support device according to claim 4.

6. The output unit outputs an adjustment window for performing the parameter adjustment work to the display device.

6. The parameter adjustment support device according to claim 1, wherein:

7. The adjustment window displays a scatter diagram in which the OK image measurement value of each OK image, the NG image measurement value of each NG image, and the threshold value are plotted.

7. The parameter adjustment support device according to claim 6.

8. The adjustment window can switch the OK image measurement values ​​and NG image measurement values ​​to be plotted on the scatter diagram to only the OK image measurement values ​​of OK images that fall outside the OK range defined by the threshold value and the NG image measurement values ​​of NG images that fall within the OK range.

8. The parameter adjustment support device according to claim 7.

9. A parameter adjustment support method for supporting adjustment of parameters used in each inspection process of an image sensor capable of performing a plurality of inspection processes on an image, comprising: The inspection process calculates values ​​of predetermined measurement parameters from the image, compares the values ​​of the measurement parameters with thresholds, and outputs a judgment result of OK or NG; The parameters include setting parameters that define conditions for calculating the values ​​of the measurement parameters and the threshold values, a step of setting N items (N is an integer of 3 or more) of inspection processes to be adjusted from among the plurality of inspection processes; acquiring a good image that should be judged as good in all of the N inspection processes and an NG image that should be judged as bad in at least one of the N inspection processes; generating a graph plotting OK image measurement values, which are values ​​of the measurement parameters calculated from OK images according to the current conditions of the setting parameters, NG image measurement values, which are values ​​of the measurement parameters calculated from NG images of the inspection process according to the current conditions of the setting parameters, and the current threshold values ​​for each of the N inspection processes; outputting the graph to a display device; Including, The parameter adjustment support method is characterized in that the graph is a radar chart having N axes corresponding to the N inspection processes, and on each axis, the OK image measurement value, the NG image measurement value, and a threshold value for the corresponding inspection process are plotted.

10. A program for causing a processor to execute each step of the parameter adjustment support method according to claim 9.

Images