Programming device, setting adjustment method, and program

Abstract

The present invention is a programming device comprising: a program generation part that provides a function to generate an image processing program for executing image processing of an image captured by a visual sensor; and a setting part that receives settings related to the image processing. The setting part displays, in a display screen, a plurality of adjustment purposes related to the image processing in a selectable manner, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes so that the priority corresponding to the selected one adjustment purpose can be recognized.

Description

Programming device, setting adjustment method, and program

[0001] The present disclosure relates to a programming device and a method and program for adjusting settings that runs on the programming device.

[0002] There is known an image processing system configured to process an image of an object captured by a camera and detect or inspect the object. In this regard, Patent Documents 1 and 2 describe configuration examples of an image processing device that can present a setting screen for setting measurement conditions.

[0003] JP 2008-123034 A JP 2016-120565 A

[0004] Image processing programs used in the image processing devices described above tend to have many setting items so that they can accommodate as wide a variety of objects as possible. Therefore, it is generally a difficult task requiring skill for an operator of such image processing programs to find and properly set the necessary setting items according to the purpose. There is a need for technology that enables easy and efficient programming, such as setting and adjusting the setting items of an image processing program.

[0005] One aspect of the present disclosure is a programming device comprising: a program creation unit that provides a function for creating an image processing program that performs image processing on an image captured by a visual sensor; and a setting unit that accepts settings related to the image processing, wherein the setting unit displays, on a display screen, a plurality of adjustment purposes related to the image processing in a selectable manner, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a manner that allows the priority according to the selected adjustment purpose to be recognized.

[0006] These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the invention illustrated in the accompanying drawings.

[0007] 1 is a diagram illustrating a configuration of an image processing system including a programming device according to an embodiment. FIG. 1 is a functional block diagram of the programming device and the image processing device. FIG. 2 is a diagram illustrating an example of a setting screen as a user interface for programming. FIG. 3 is a diagram illustrating an example of an image processing program set and displayed in a processing flow display area. FIG. 4 is a diagram illustrating examples of setting items associated with each functional module constituting the image processing program. FIG. 5 is a diagram illustrating an example of an adjustment purpose displayed selectably in an adjustment purpose selection area. FIG. 6 is a diagram illustrating an example of a display mode of setting items displayed in the setting area when the adjustment purpose "I want to increase processing speed" is selected. FIG. 7 is a diagram illustrating an example of a processing flow display area in which the processing flow of the image processing program is displayed, highlighting functional modules including setting items to which priorities according to the adjustment purpose have been assigned. FIG. 8 is a flowchart illustrating the basic operations of processing for adjusting setting items of the image processing program.

[0008] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings, like components or functional parts are designated by like reference numerals. The scales of these drawings have been changed appropriately to facilitate understanding. Furthermore, the embodiment shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the illustrated embodiment.

[0009] FIG. 1 is a diagram illustrating the configuration of an image processing system 100 including a programming device according to an embodiment. As illustrated in FIG. 1, the image processing system 100 includes a programming device 10, an image processing device 20 connected to the programming device 10, and a visual sensor 30 and an illumination device 31 connected to the image processing device 20. The image processing system 100 is configured to capture an image of an object 90 using the visual sensor 30 and perform various image processing operations on the captured image, such as detecting and inspecting the object 90. In the image processing system 100, the programming device 10 provides a function for creating various programs related to capturing images using the visual sensor 30 and image processing of the captured images. The image processing device 20 controls the visual sensor 30 and the illumination device 31 and performs image processing based on commands from the programming device 10.

[0010] In this specification, programming an image processing program includes not only creating a program using command statements, functional modules, icons, etc., but also completing the function of the image processing program by setting and adjusting setting items.

[0011] Various information processing devices such as a tablet terminal can be used as the programming device 10. The programming device 10 may have a hardware configuration as a general computer including a processor 11, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 12, a display unit 13, an operation unit 14, an input / output interface, a network interface, etc. (See FIG. 2 ). The display unit 13 may be a flat panel display (such as a liquid crystal display). The operation unit 14 may be a touch panel input device integrated with the display.

[0012] The visual sensor 30 may be a two-dimensional camera that captures grayscale or color images, or a stereo camera or three-dimensional sensor that can capture distance images. The image processing device 20 stores a model pattern of the object and can perform image processing to detect the object by pattern matching between the object in the captured image and the model pattern.

[0013] The image processing device 20 may be configured as a device dedicated to image processing, or may be configured as a general-purpose information processing device such as a PC (personal computer). The image processing device 20 may have a hardware configuration as a general computer including a processor 21, memory (ROM, RAM, non-volatile memory, etc.), a storage unit 22, an input / output interface, a network interface, etc. (See FIG. 2). The image processing device 20 may further include a display unit and an operation unit as hardware components.

[0014] Although FIG. 1 shows an example in which the image processing device 20 is configured as a device separate from the programming device 10, the function of the image processing device 20 may be incorporated into the programming device 10.

[0015] FIG. 2 shows a functional block diagram of the programming device 10 and the image processing device 20.

[0016] 2, the programming device 10 includes a program creation unit 111, a program execution unit 112, a setting unit 113, a processing time acquisition unit 114, and a state analysis unit 115. These functional blocks may be realized by the processor 11 of the programming device 10 executing software. Also shown in Fig. 2 is a storage unit 12 of the programming device 10. The storage unit 12 may be configured, for example, by a non-volatile memory or a hard disk drive.

[0017] The program creation unit 111 provides a function for creating various programs, such as programs related to capturing images using the visual sensor 30 and image processing of the captured images. For example, the program creation unit 111 displays a program creation screen for creating a program and accepts user input for programming. The program creation unit 111 may be configured to accept programming based on input of text-based statements, as well as programming based on functional modules or icons. The program creation unit 111 also has a function for registering the created program in the storage unit 12.

[0018] The program execution unit 112 provides a function for executing a created program. The program execution unit 112 sends commands based on the program to the image processing device 20 to cause it to capture images and perform image processing.

[0019] The setting unit 113 displays various setting items related to image processing and provides a function for accepting input of setting values ​​for the setting items. The setting unit 113 has a function for displaying selectable adjustment purposes for image processing, so that an operator can efficiently set parameters for image processing, and for displaying setting items related to the selected adjustment purpose in response to the operator's selection of the adjustment purpose. Furthermore, the setting unit 113 has a function for displaying one or more setting items related to one adjustment purpose selected from multiple adjustment purposes, so that the operator can recognize the priority according to the selected adjustment purpose. This allows the operator to recognize the priority according to the adjustment purpose for the setting items related to the selected adjustment purpose, and allows efficient setting.

[0020] Information (hereinafter also referred to as priority information) associating, for each adjustment purpose, setting items related to the adjustment purpose with priorities for the setting items related to the adjustment purpose according to the adjustment purpose may be set in advance and stored in the storage unit 12. As an example, the priority information has a data structure as shown in Table 1 below. Note that, in the priority information shown in Table 1 below, adjustment purpose A is associated with setting items A1 and A2 that are related to it, and with priorities for those setting items A1 and A2 that are related to adjustment purpose A. Also, in the priority information shown in Table 1 below, adjustment purpose B is associated with setting items B1 and B2 that are related to it, and with priorities for those setting items B1 and B2 that are related to adjustment purpose B.

[0021]

[0022] Using such priority information, the setting unit 113 can display one or more setting items related to one adjustment purpose selected from multiple adjustment purposes so that the priority according to the selected adjustment purpose can be recognized.

[0023] The processing time acquisition unit 114 can acquire from the processing time measurement unit 123 the processing time actually required for each functional unit related to image processing when image processing is executed in the image processing device 20. For example, the processing time acquisition unit 114 can acquire the time required for processing to search for the angle of an object as one functional unit in pattern matching for detecting the object.

[0024] The state analysis unit 115 cooperates with the image processing device 20 and has a function of acquiring and analyzing the state related to the execution of the image processing program.

[0025] The storage unit 12 stores programs, various settings related to image processing programs, priority information, and the like.

[0026] As shown in Fig. 2, the image processing device 20 includes a visual sensor control unit 121, an image processing unit 122, and a processing time measurement unit 123. These functional blocks may be realized by the processor 21 of the image processing device 20 executing software. Fig. 2 also shows a storage unit 22 of the image processing device 20. The storage unit 22 may be configured, for example, by a non-volatile memory or a hard disk drive.

[0027] The visual sensor control unit 121 controls the visual sensor 30 in accordance with commands sent from the programming device 10. The image processing unit 122 has the function of executing various image processing operations on images captured by the visual sensor 30. The processing time measurement unit 123 provides a function of measuring the actual processing time for each function corresponding to each setting item when image processing is performed by the image processing unit 122. The function of the processing time measurement unit 123 can be realized, for example, by incorporating into the image processing program a function that obtains the processing start time and processing end time for each function from an internal timer and calculates the processing time.

[0028] The storage unit 22 stores various information such as image data, various setting information related to image processing, and model data of the object.

[0029] 3 shows an example of a setting screen 200 as a user interface for programming. The setting screen 200 also functions as a program creation screen. The setting screen 200 is provided by the program creation unit 111 and the setting unit 113. The program creation unit 111 and the setting unit 113 display the setting screen 200 on the display unit 13. The program creation unit 111 and the setting unit 113 also accept input to the setting screen 200 via the operation unit 14.

[0030] As shown in FIG. 3, the setting screen 200 includes an image display area 211 , an adjustment object selection area 212 , a processing flow display area 213 , and a setting area 214 .

[0031] The image display area 211 is an area for displaying an image captured by the visual sensor 30. For example, the programming device 10 (program execution unit 112) executes an image processing program and acquires an image G of the object 90 captured by the visual sensor 30 from the image processing device 20, and displays the image G in the image display area 211. The operator can check the image used for image processing (inspection, etc.) on the image display area 211. Image G is an example of an image capturing three objects 90 in the situation shown in FIG. 1 .

[0032] The adjustment purpose selection area 212 is an area for displaying a list of adjustment purposes related to the image processing program and accepting selection of an adjustment purpose. The setting unit 113 displays a list of multiple selectable adjustment purposes in the adjustment purpose selection area 212. The setting unit 113 accepts a user operation for selecting one adjustment purpose from the list of adjustment purposes.

[0033] The processing flow display area 213 is an area for setting and displaying the processing flow of the image processing program. The program creation unit 111 can provide a function for an operator to perform programming by arranging icons representing functional modules along the processing flow in the processing flow display area 213. The operator can perform programming by selecting desired functional icons from a list of functional module icons prepared in advance and arranging them in the processing flow display area 213.

[0034] The setting area 214 is an area for displaying setting items related to the image processing program and for inputting or adjusting the setting items. The setting unit 113 has a function of displaying setting items related to a functional module selected in the processing flow display area 213 and accepting settings for those setting items. Furthermore, the setting unit 113 has a function of displaying, in the setting area 214, one or more setting items related to one adjustment purpose selected in the adjustment purpose selection area 212 so that the priority according to the selected adjustment purpose can be recognized.

[0035] 4 shows an example of an image processing program P1 set and displayed in the processing flow display area 213. The image processing program P1 is a program for inspecting an object. As shown in FIG. 4, the image processing program P1 includes an imaging function M1, a pattern matching function M2, a length measurement function M3, and a pass / fail determination function M4.

[0036] The imaging function M1 corresponds to the function of capturing an image of an object using the visual sensor 30. The pattern matching function M2 corresponds to the function of detecting an object by pattern matching from the captured image using a model pattern of the object. The length measurement function M3 corresponds to the function of measuring the length of an inspection target portion on an object recognized by pattern matching. An example of the length measurement function M3 is a process for measuring the length between opposing sides when the object is box-shaped. The pass / fail judgment function M4 corresponds to the function of making a pass / fail judgment on the length measured by the length measurement function M3.

[0037] The operator can perform programming by arranging the functional modules in order in the processing flow display area 213 and setting and adjusting the necessary setting items for each functional module. The setting unit 113 has a function of displaying the setting items of the functional module selected in the processing flow display area 213 in the setting area 214 and accepting settings for those setting items.

[0038] 5 shows an example of setting items associated with each functional module constituting the image processing program P1. The setting unit 113 displays the setting items associated with the functional module selected in the processing flow display area 213 in the setting area 214 and accepts the setting. Information associating each functional module with its associated setting items is stored in advance in the storage unit 12.

[0039] As shown in Figure 5, the imaging function M1 includes the following setting items associated with it: camera selection, exposure time, lighting intensity, number of times to capture images, imaging area, and image reduction ratio. The screen of setting items displayed in the setting area 214 in response to the selection of the imaging function M1 will be referred to as the setting item screen 301. When the imaging function M1 is selected in the processing flow display area 213, the setting unit 113 has the function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the following setting values ​​are input for these setting items: Camera selection: Camera 1 Exposure time: 200 ms Lighting intensity: 100% Number of times to capture images: 1 time Imaging area: 1280 x 1024 pixels Image reduction ratio: 1 / 2

[0040] The pattern matching function M2 includes the following setting items associated with it: model setting, score threshold, contrast threshold, size threshold, angle threshold, upper limit of number of detections, and detection time limit. The screen of setting items displayed in the setting area 214 in response to the selection of the pattern matching function M2 will be referred to as the setting item screen 302. The setting unit 113 has the function of displaying these setting items in the setting area 214 when the pattern matching function M2 is selected in the processing flow display area 213, and accepting settings for these setting items. FIG. 5 shows an example in which the following setting values ​​have been entered for these setting items: Model setting: set Score threshold: 75% Contrast threshold: 50 Size threshold: 90 to 120% Angle threshold: -30 to +30 degrees Upper limit of number of detections: 5 Detection time limit: 30 seconds

[0041] The length measurement function M3 includes input, measurement region setting, contrast threshold, and length threshold as its associated setting items. The screen of setting items displayed in the setting area 214 in response to the selection of the length measurement function M3 will be referred to as the setting item screen 303. When the length measurement function M3 is selected in the processing flow display area 213, the setting unit 113 has the function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the following setting values ​​have been entered for these setting items: Input: Pattern match Measurement region setting: Set Contrast threshold: 50 Length threshold: 100 to 100 pixels

[0042] The pass / fail judgment function M4 includes input, judgment criteria, and judgment as its related setting items. The screen of setting items displayed in the setting area 214 in response to the selection of the pass / fail judgment function M4 will be referred to as the setting item screen 304. When the pass / fail judgment function M4 is selected in the processing flow display area 213, the setting unit 113 has the function of displaying these setting items in the setting area 214 and accepting settings for these setting items. Figure 5 shows an example in which the following setting values ​​are entered for these setting items: Input: Length measurement Judgment criteria: 50 > input > 25 Judgment: Pass if within range

[0043] In this way, the program creation unit 111 and the setting unit 113 can provide basic programming functions, such as setting an image processing program in the processing flow display area 213 and displaying setting items in the setting area 214 to accept setting input.

[0044] Next, we will explain the function of the setting unit 113 to display one or more setting items related to one adjustment purpose selected in the adjustment purpose selection area 212 so that the priority according to the selected adjustment purpose can be recognized.

[0045] 6 shows examples of adjustment purposes displayed in a selectable manner in the adjustment purpose selection area 212. The adjustment purposes displayed here are set based on the results of a prior analysis of what kind of adjustments workers involved in image processing generally desire. In this example, the multiple adjustment purposes include: 1. "Standard" 2. "Increase processing speed" 3. "Take measures to prevent false detection" 4. "Take measures to prevent non-detection" 5. "Improve detection accuracy" 6. "Limit adjustments"

[0046] The adjustment purpose "standard" is an item selected when the operator desires standard parameter settings for the image processing program. When the adjustment purpose "standard" is selected, the setting unit 113 may, for example, set predetermined standard values ​​as values ​​for each setting item for the image processing program set in the processing flow display area 213 and display them in the setting area 214.

[0047] The adjustment purpose "increase processing speed" is an item selected when the operator desires to increase the processing speed of the image processing program. When the adjustment purpose "increase processing speed" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

[0048] The adjustment purpose "I want to take measures against false detection" is an item selected when the operator wishes to improve false detections in the image processing program. Here, false detection refers to an event in which the image processing program detects an object that is not a target object as a target object. When the adjustment purpose "I want to take measures against false detection" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

[0049] The adjustment purpose "I want to take measures against non-detection" is an item that is set when the operator wishes to improve the non-detection of the image processing program. Here, "non-detection" refers to an event in which the image processing program is unable to detect an object that should have been detected. When the adjustment purpose "I want to take measures against non-detection" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

[0050] The adjustment purpose "improve detection accuracy" is an item that is set when the operator desires more accurate detection. When the adjustment purpose "improve detection accuracy" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

[0051] The adjustment purpose "I want to limit adjustment" is an item selected when the operator wishes to perform settings in a state where the setting items to be adjusted are limited and only necessary setting items are subject to adjustment. When the adjustment purpose "I want to limit adjustment" is selected, the setting unit 113 displays one or more setting items related to this adjustment purpose in the setting area 214 so that the priority according to this adjustment purpose can be recognized.

[0052] 7 shows an example of the display of setting items displayed in the setting area 214 when the adjustment purpose "increase processing speed" is selected in a situation where the image processing program P1 is set in the processing flow display area 213. Here, it is assumed that a standard value is set in advance for each setting item, and the operator selects this adjustment purpose "increase processing speed" in order to increase the processing speed. As shown in the example of FIG. 7, the setting area 214 displays, in a recognizable manner, setting items that should be set with priority in accordance with the adjustment purpose "increase processing speed" out of the setting items related to each functional module of the image processing program P1.

[0053] As shown in FIG. 7 , when the imaging function M1 is selected in the processing flow display area 213, the setting item screen 301 displayed in the setting area 214 is in the display mode indicated by the reference symbol 301a in FIG. Of the setting items for the imaging function M1, namely, camera selection, exposure time, lighting intensity, number of captures, imaging area, and image reduction ratio, the exposure time, number of captures, imaging area, and image reduction ratio affect the processing speed. The imaging area is a setting item that specifies the area of ​​the imageable area to be captured as an image. Setting a smaller imaging area can shorten the processing time. The image reduction ratio is a setting item that specifies the degree to which the image is compressed. Here, the smaller the image reduction ratio, the smaller the image and the shorter the time required for image processing. Therefore, on the setting item screen 301a for the imaging function M1, the exposure time, number of captures, imaging area, and image reduction ratio are highlighted among the setting items, namely, camera selection, exposure time, lighting intensity, number of captures, imaging area, and image reduction ratio. The highlighting may be achieved, for example, by changing the size, thickness, color, font, etc. of the characters.

[0054] Furthermore, among these four setting items, exposure time, number of captures, capture area, and image reduction ratio, the exposure time and image reduction ratio are particularly closely related to the image processing speed, so the "exposure time" and "image reduction ratio" have the highest priority among these four setting items. Therefore, the "exposure time" and "image reduction ratio" are displayed with even more emphasis than the other four setting items. The "exposure time" and "image reduction ratio" may be further emphasized by, for example, changing the size, thickness, color, font, etc. of the characters.

[0055] As shown in FIG. 7 , when the pattern matching function M2 is selected in the processing flow display area 213, the setting item screen 302 displayed in the setting area 214 is in the display mode indicated by the reference symbol 302a in FIG. Among the setting items of the pattern matching function M2, namely, model setting, score threshold, contrast threshold, size threshold, angle threshold, upper limit of detection count, and detection time limit, the size threshold and angle threshold affect the processing speed. Pattern matching involves searching an image for features identical to the taught feature (model feature). The search is performed while rotating and resizing the model feature. The "angle threshold" specifies the allowable angle range for feature rotation during this search. For example, if the angle threshold is set to ±0 degrees, feature rotation is not taken into account, thereby speeding up the search. The "size threshold" specifies the allowable range for feature size during this search. For example, if the size threshold is set to 100%, only features of the same size as the model feature are searched, thereby speeding up the processing. Thus, among the setting items of the pattern matching function M2, the size threshold and angle threshold affect the processing speed. Therefore, on the setting item screen 302a for the pattern matching function M2, these two setting items, the size threshold and the angle threshold, are highlighted in comparison with the other setting items.

[0056] In this way, the operator can recognize the setting items that should be set preferentially and their priorities according to the adjustment purpose, such as "to increase processing speed," and can efficiently make appropriate settings according to the adjustment purpose.

[0057] Note that, when the processing time of the entire image processing program is taken into consideration, if there is a difference in effectiveness between the setting items to be emphasized for the imaging function M1 and the setting items to be emphasized for the pattern matching function M2, the degree of emphasis may be set taking this superiority into further consideration. For example, when the processing time of the entire image processing program is taken into consideration, the order of effectiveness (priority) for improving the processing speed with respect to the exposure time, the number of images taken, the imaging area, the image reduction rate, the size threshold, and the angle threshold is as follows: (exposure time, image reduction rate) > (number of images taken, imaging area) > (size threshold, angle threshold). In this case, the setting unit 113 may display the exposure time and the image reduction rate with the strongest emphasis, then the number of images taken and the imaging area with the second strongest emphasis, and the size threshold and the angle threshold with the third strongest emphasis.

[0058] For the length measurement function M3, there is no setting item that should be set with priority in accordance with the adjustment purpose of "wanting to increase the processing speed," so no special emphasis is placed on the setting item screen 303 displayed in association with the length measurement function M3. For the pass / fail determination function M4, there is no setting item that should be set with priority in accordance with the adjustment purpose of "wanting to increase the processing speed," so no special emphasis is placed on the setting item screen 304 displayed in association with the pass / fail determination function M4.

[0059] In addition, the setting item screens 301a, 302a, 303, and 304 shown in Figure 7 may be displayed simultaneously in window format in the setting area 214 in response to the selection of ``I want to increase the processing speed'' in the adjustment purpose selection area 212 when the image processing program P1 is set in the processing flow display area 213.

[0060] In addition to the above examples, other possible modes for recognizably displaying setting items that should be set with priority include: - highlighting setting items that should be set with priority by adding underlines, borders, or other images; - indicating the level of priority with the number of marks displayed next to the setting items; - hiding or weakly displaying setting items other than setting items that should be set with priority. Note that "weak display" here means displaying with reduced emphasis by using a lighter color or smaller font size, etc.

[0061] If the adjustment purpose is "to take measures against false detection," the following setting items may be highlighted and displayed as setting items that should be set with priority according to this adjustment purpose: "Score threshold" of the pattern matching function "Contrast threshold" of the pattern matching function "Model setting" of the pattern matching function "Size threshold" of the pattern matching function "Exposure time" of the imaging function

[0062] The pattern matching setting "Score Threshold" specifies the threshold for the degree of match calculated for objects in an image during pattern matching. Increasing the pattern matching score threshold reduces false positives. The pattern matching setting "Contrast Threshold" specifies the contrast threshold for image features (e.g., the outline of an object) against the background. Increasing the contrast threshold for pattern matching allows only clearly visible features to be subjected to pattern matching, thereby reducing false positives. The pattern matching setting "Model Settings" is a setting for teaching model features. For example, re-executing model settings and increasing the number of model features (e.g., increasing the number of edge points representing model features) can reduce false positives. Reducing the pattern matching size threshold also reduces false positives. Regarding the exposure time of the imaging function, for example, increasing the exposure time to eliminate unnecessary features from the image can also reduce false positives.

[0063] Of the above five setting items that are effective in preventing false detection, the "score threshold" and the "contrast threshold" are particularly effective in reducing false detection, and therefore the setting unit 113 may assign a higher priority to the "score threshold" and the "contrast threshold" among the above five setting items than to the other setting items. Therefore, the setting unit 113 may further emphasize and display the "score threshold" and the "contrast threshold" among the above five setting items.

[0064] In this way, even with regard to the adjustment purpose of "taking measures against false detections," the operator can recognize the setting items that should be set preferentially according to the adjustment purpose and their priorities, and can efficiently make appropriate settings according to the adjustment purpose.

[0065] If the adjustment purpose is "to take measures against non-detection," the following setting items may be highlighted and displayed as setting items that should be set with priority according to this adjustment purpose: "Score threshold" of the pattern matching function "Contrast threshold" of the pattern matching function "Size threshold" of the pattern matching function "Exposure time" of the imaging function

[0066] Lowering the "score threshold" of the pattern matching function has the effect of reducing undetected results. Lowering the "contrast threshold" of the pattern matching function has the effect of reducing undetected results. Increasing the width of the "size threshold" of the pattern matching function has the effect of reducing undetected results. Adjusting the "exposure time" of the imaging function so that the required features of the object appear has the effect of reducing undetected results.

[0067] Of the four setting items that are effective in preventing non-detection, the "score threshold" and the "contrast threshold" are particularly effective in reducing non-detection, and therefore the setting unit 113 may assign a higher priority to the "score threshold" and the "contrast threshold" than to the other setting items. Therefore, the setting unit 113 may further emphasize and display the "score threshold" and the "contrast threshold" among the four setting items that are effective in preventing non-detection.

[0068] In this way, even with regard to the adjustment purpose of "taking measures to deal with undetected conditions," the operator can recognize the setting items that should be set preferentially according to the adjustment purpose and their priorities, and can efficiently make appropriate settings according to the adjustment purpose.

[0069] If the adjustment purpose is "to improve detection accuracy," the following setting items may be highlighted and displayed as setting items that should be set with priority according to this adjustment purpose: "Model setting" of the pattern matching function; "Contrast threshold" of the pattern matching function; and "Image reduction ratio" of the imaging function.

[0070] For example, re-teaching model features in the "model setting" of the pattern matching function so that only stable features are included as model features is effective in improving detection accuracy. Increasing the "contrast threshold" of the pattern matching function has the effect of improving detection accuracy. Increasing the "image reduction ratio" of imaging to closer to 1 increases the image resolution, which is effective in improving detection accuracy.

[0071] Of the three setting items that are effective in improving detection accuracy, "Model Setting" and "Contrast Threshold" are particularly effective in improving detection accuracy, and therefore the setting unit 113 may assign higher priority to "Model Setting" and "Contrast Threshold" than to the other setting items. Therefore, the setting unit 113 may further emphasize and display "Model Setting" and "Contrast Threshold" among the three setting items that are effective in improving detection accuracy.

[0072] The adjustment purpose "I want to limit adjustments" is selected when the operator wants to limit the setting items so that only necessary setting items are configurable. For example, in a situation where an operator changes only the pass / fail judgment criteria in the daily operation of the image processing system 100, the operator may want to operate only this setting item and exclude other setting items from the setting items. In this case, for example, when the adjustment purpose "I want to limit adjustments" is selected, the following display examples are possible: - Only the "judgment criteria" for pass / fail judgment is displayed as a configurable setting item; - Only the "judgment criteria" for pass / fail judgment is highlighted; - Setting items other than the "judgment criteria" for pass / fail judgment are displayed weakly (grayed out, etc.). Such an operation can also be considered an example of an operation in which setting items related to a selected adjustment purpose are displayed so that the priority according to the selected adjustment purpose can be recognized.

[0073] When the adjustment purpose "want to limit adjustment" is selected, two or more setting items may be highlighted as configurable setting items. When there are two or more setting items that can be set in relation to the adjustment purpose "want to limit adjustment", priorities according to this adjustment purpose may be assigned to the two or more setting items.

[0074] This adjustment purpose "I want to limit adjustments" allows the setting know-how of an experienced worker to be passed on to an inexperienced worker as priority information. For example, if priority information is created so that setting items that do not need to be adjusted are not set when an image processing program is executed, an inexperienced worker can select the adjustment purpose "I want to limit adjustments" to perform adjustments while excluding setting items that do not need to be adjusted from the settings.

[0075] The setting unit 113 may have a function for displaying a functional module in the process flow display area 213 so that the user can recognize it if there is a setting item for the functional module that has been assigned a priority according to the selected adjustment purpose. FIG. 8 shows an example in which the process flow display area 213 displaying the process flow of the image processing program P1 highlights a functional module that includes a setting item that has been assigned a priority according to the adjustment purpose. In this example, when the adjustment purpose is "to increase processing speed," the "imaging function M1" and the "pattern matching function M2" include setting items that have been assigned a priority, and these "imaging function M1" and "pattern matching function M2" are highlighted. In this example, the processing blocks of the imaging function M1 and the pattern matching function M2 are highlighted by thickening their borders. Other methods for highlighting processing blocks in the process flow display area 213 (e.g., highlighting by font size or color) may also be used.

[0076] The setting unit 113 may more strongly emphasize functional blocks that include setting items that should be set with higher priority among the functional blocks that should be highlighted in the processing flow display area 213. Fig. 8 shows an example in which the processing block of the imaging function M1 is highlighted more than the processing block of the pattern matching function M2 because the imaging function M1 includes setting items that have a higher priority than the pattern matching function M2. Here, the processing block of the imaging function M1 is displayed with a thicker frame than the processing block of the pattern matching function M2.

[0077] In the above-described embodiment, the setting unit 113 sets priorities of one or more setting items related to the selected adjustment purpose according to preset priority information. The setting unit 113 may have a function of switching the priorities set according to the priority information depending on various situations related to the setting or execution of the image processing program. Note that switching the priority may include assigning a priority to a setting item that did not have a priority (i.e., a setting item that had a priority of zero) or canceling the priority of a setting item that had had a priority. In this regard, the setting unit 113 may have one or more of the following functions (F1) to (F3): (F1) A function of switching the priority of a setting item based on the actual processing time for each function of the image processing program; (F2) A function of switching the priority of a setting item depending on the current setting value of each setting item of the image processing program; and (F3) A function of switching the priority of a setting item depending on the execution status of the image processing program.

[0078] An example of the above function (F1) will be described. As an example, assume that the selected adjustment purpose for the image processing program P1 is "to increase processing speed." In this case, the processing time acquisition unit 114 can acquire the processing time for each function when the image processing program P1 is executed from the image processing device 20 (processing time measurement unit 123). For example, assume that the processing time acquisition unit 114 determines that angle search is dominant in pattern matching (or in the entire image processing) based on the measured processing time for each function, such as angle search and size search, in pattern matching. In this case, the setting unit 113 may set the "angle threshold" of the pattern matching function M2 as the highest priority setting item, or may change the priority of the "angle threshold" to a value higher than its current value.

[0079] In this way, based on the measured processing time for each function, the setting unit 113 may: - assign the highest priority to the setting item corresponding to the function with the longest processing time, - increase the priority of the setting item corresponding to the function with a longer processing time than other functions, or - decrease the priority of the setting item corresponding to the function with a shorter processing time than other functions.

[0080] A first example of the above function (F2) will be described. When the adjustment purpose is "to prevent false detection," the setting items prioritized by the priority information include the pattern matching score threshold and the contrast threshold. Setting the score threshold high is effective in preventing false detection, but if the current score threshold is already set to a high value, there is no room left to adjust the score threshold. In such a situation, the setting unit 113 may lower the priority of the score threshold or cancel the priority assigned to the score threshold. That is, the setting unit 113 may lower the emphasis level of the score threshold or may not perform highlighting. Furthermore, in a situation where there is no room left to adjust the score threshold, the importance of adjusting the contrast threshold increases. Therefore, the setting unit 113 may, for example, change the priority of the contrast threshold to a higher value, thereby making the priority of the contrast threshold higher than the priority of the score threshold.

[0081] A first example of the function (F3) will be described. Assume that the adjustment purpose is "to take measures against undetected objects." For example, when the captured image is dark overall, the setting unit 113 may assign priority to the exposure time and the illumination intensity so that they are adjusted. The state analysis unit 115 may determine whether the captured image is dark overall by acquiring and analyzing an image obtained by executing an image processing program from the image processing device 20 (image processing unit 122). The state analysis unit 115 may determine the overall brightness of the image based on, for example, a histogram of the luminance values ​​of each pixel in the captured image. Note that in this case, when the exposure time has already reached its settable upper limit, the setting unit 113 may additionally apply the function (F2) to assign priority only to the illumination intensity, so that only the illumination intensity is highlighted.

[0082] A second example of the above function (F3) will be described. Assume that the detection results of the object's position are found to be unstable despite repeated detection without moving the object. In this case, the setting unit 113 may prioritize the model setting items or change the priority of the model setting to a higher value to prompt re-registration of the pattern matching function model for the adjustment purpose of "improving detection accuracy." The stability of the object's position detection results can be determined by the state analysis unit 115 acquiring object detection results from the image processing device 20 (image processing unit 122) when the image processing program is executed multiple times and analyzing the detected position of the object based on those detection results. For example, the state analysis unit 115 may determine that the object's position detection results are unstable if the variation in the object's position detected multiple times is equal to or greater than a certain value, even when the object's position is constant.

[0083] With this function, the priority state (degree of emphasis) of the setting items displayed in the setting area 214 can be switched to a state that reflects the settings and execution status of the image processing program.

[0084] 9 is a flowchart showing the basic operation of a process (setting adjustment method) for adjusting setting items of an image processing program provided by the programming device 10. This setting item adjustment process is executed under the control of the processor 11 of the programming device 10. First, the setting unit 113 displays a list of adjustment objects selectably in a first area (adjustment object selection area 212) on the setting screen 200 (step S1).

[0085] Next, the setting unit 113 displays one or more setting items related to the selected adjustment purpose in a second area (setting area 214) so ​​that the priority according to the selected adjustment purpose can be recognized (step S2).The setting unit 113 then accepts adjustment operations for the one or more setting items in the second area (setting area 214) (step S3).With this configuration, as described above, the operator can easily and efficiently make adjustments that match the adjustment purpose.

[0086] In the above-described embodiment, the setting unit 113 displays in the setting area 214 setting items that should be set with priority in accordance with the purpose of adjustment in a recognizable manner. In addition to this function, the setting unit 113 may have a function to display guidance information for the setting items that should be set with priority. Information that associates the setting items with the guidance information may be stored in advance in the storage unit 12, and the setting unit 113 may be configured to display the guidance information based on this information.

[0087] For example, when the adjustment purpose is "increased processing speed," in a scene where the setting item screen 301a shown in FIG. 7 is displayed, the following examples may be possible: - a message saying "It is recommended to shorten the exposure time" may be displayed as guidance information next to the setting item "exposure time," or - a message saying "It is recommended to further reduce the image" may be displayed as guidance information next to the setting item "image reduction ratio."

[0088] If the purpose of adjustment is to "prevent false detection," examples of messages that may be displayed as guidance information include: - Regarding the score threshold, "We recommend that you increase the score threshold." - Regarding the contrast threshold, "We recommend that you increase the contrast threshold."

[0089] If the purpose of adjustment is "to take measures against undetected objects," examples of messages that may be displayed as guidance information include: - Regarding the score threshold, "It is recommended that you lower the score threshold." - Regarding the contrast threshold, "It is recommended that you lower the contrast threshold."

[0090] If the purpose of adjustment is to "improve detection accuracy," there may be examples where guidance information is displayed that includes messages such as: - Regarding model settings, "We recommend that you re-teach so that the model features increase." - Regarding image reduction rate, "We recommend that you set the image reduction rate closer to 1."

[0091] Such guidance information may be constantly displayed next to the setting item, for example, in a balloon format, or may be displayed in a pop-up format when the cursor is placed over the setting item. By displaying guidance information in this manner, the operator can adjust the setting item more easily and efficiently.

[0092] As described above, according to this embodiment, the operator can easily and efficiently make settings according to the adjustment purpose by simply performing the simple operation of selecting the adjustment purpose when programming the image processing program.

[0093] The functional allocation in the functional block diagram of the programming device 10 and the image processing device 20 shown in Figure 2 is an example, and various examples of functional allocation are possible. For example, there may be a configuration in which some of the functions of the programming device 10 shown in Figure 2 are arranged in the image processing device 20. There may also be a configuration in which the entire functions of the image processing device 20 are arranged in the programming device 10 (i.e., a configuration in which the programming device 10 and the image processing device 20 are integrated).

[0094] The image processing system 100 can also be applied to a robot system configured to detect an object using a visual sensor and handle it using a robot. In this case, the functions of the image processing device 20 and the programming device 10 may be incorporated into the robot control device. Alternatively, the functions of the image processing device 20 and the programming device 10 may all be incorporated into a teaching device connected to the robot control device.

[0095] The functional blocks in the functional block diagram of the programming device shown in Fig. 2 may be realized by one or more processors of the programming device executing various software programs stored in a storage device, or may be realized by a configuration mainly composed of hardware such as an ASIC (Application Specific Integrated Circuit). The functional blocks in the functional block diagram of the image processing device shown in Fig. 2 may be realized by one or more processors of the image processing device executing various software programs stored in a storage device, or may be realized by a configuration mainly composed of hardware such as an ASIC (Application Specific Integrated Circuit).

[0096] The program for executing various processes such as the setting item adjustment process of Figure 9 in the above-mentioned embodiment can be recorded on various computer-readable recording media (for example, semiconductor memory such as ROM, EEPROM, flash memory, magnetic recording media, optical disks such as CD-ROM and DVD-ROM).

[0097] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0098] The following supplementary notes are further provided with respect to the above-described embodiment and modified examples. (Supplementary Note 1) A programming device (10) comprising: a program creation unit (111) that provides a function for creating an image processing program that executes image processing on an image captured by a visual sensor (30); and a setting unit (113) that accepts settings related to the image processing, wherein the setting unit (113) selectably displays a plurality of adjustment purposes related to the image processing on a display screen, and displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a manner that enables the priority corresponding to the selected adjustment purpose to be recognized. (Supplementary Note 2) The setting unit (113) displays one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned in a manner that makes the one or more setting items to which the priority has not been assigned more emphasized than setting items to which the priority has not been assigned. (Supplementary Note 3) The programming device (10) according to Supplementary Note 1 or 2, wherein the setting unit (113) displays, for one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned, setting items with a higher priority corresponding to the selected adjustment purpose so as to be more strongly emphasized. (Supplementary Note 4) The programming device (10) according to any one of Supplements 1 to 3, further comprising a storage unit (12) that stores, for each of the plurality of adjustment purposes, priority information associating at least one setting item related to the adjustment purpose with a priority corresponding to the adjustment purpose for the at least one setting item, and the setting unit (113) assigns a priority corresponding to the selected adjustment purpose to one or more setting items related to the selected adjustment purpose based on the priority information. (Supplementary Note 5) The programming device (10) according to Supplementary Note 4, wherein the setting unit (113) switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information based on the setting or execution state of the image processing program.(Supplementary Note 6) The programming device (10) according to Supplementary Note 5, wherein the setting unit (113) switches the priorities assigned to one or more setting items associated with the selected adjustment objective based on the priority information, based on current setting values ​​of the one or more setting items associated with the selected adjustment objective. (Supplementary Note 7) The programming device (10) according to Supplementary Note 6, wherein the setting unit (113) lowers the priority of or cancels the assignment of the priority to a setting item determined to have no room for adjustment, based on current setting values ​​of the one or more setting items associated with the selected adjustment objective. (Supplementary Note 8) The programming device (10) according to Supplementary Note 5, further comprising a processing time measurement unit (123) capable of measuring a processing time for each function corresponding to each setting item when the image processing program is executed, and wherein the setting unit (113) switches the priorities assigned to one or more setting items associated with the selected adjustment objective based on the priority information, based on measured values ​​of the processing time for each function corresponding to each setting item when the image processing is executed. (Supplementary Note 9) The programming device according to Supplementary Note 5, further comprising a state analysis unit (115) that determines, as the execution state of the image processing program, the state of the image captured by the visual sensor (30) or the state of stability of the detection result of the object included in the image processing. (Supplementary Note 10) The programming device (10) according to any one of Supplements 1 to 9, wherein the setting unit (113) displays a setting screen including a first area that displays the plurality of adjustment objectives and a second area that displays one or more setting items related to the selected one adjustment objective, and the display state of the second area is switched according to a priority corresponding to the selected one adjustment objective. (Supplementary Note 11) The programming device (10) according to any one of Supplements 1 to 10, wherein the plurality of adjustment objectives include one or more of increasing processing speed, taking measures against erroneous detection, taking measures against non-detection, improving detection accuracy, or limiting adjustment items.(Supplementary Note 12) The programming device (10) according to any one of Supplements 1 to 11, wherein the setting unit (113) additionally displays guidance information corresponding to the selected one adjustment purpose for at least one of the one or more setting items displayed so that the priority can be recognized. (Supplementary Note 13) A setting adjustment method for adjusting settings related to image processing in the programming device (10), comprising: selecting a plurality of adjustment purposes related to the image processing in a first area on a display screen; displaying one or more setting items associated with one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that the priority corresponding to the selected one adjustment purpose can be recognized; and accepting an adjustment operation for the one or more setting items in the second area. (Appendix 14) A program for causing a computer processor to execute the steps of: displaying a plurality of adjustment purposes related to image processing in a selectable manner in a first area on a display screen; displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that the priority according to the selected one adjustment purpose can be recognized; and accepting adjustment operations for the one or more setting items in the second area.

[0099] REFERENCE SIGNS LIST 10 Programming device 11 Processor 12 Memory unit 13 Display unit 14 Operation unit 20 Image processing device 21 Processor 22 Memory unit 30 Visual sensor 31 Lighting device 90 Object 100 Image processing system 111 Program creation unit 112 Program execution unit 113 Setting unit 114 Processing time acquisition unit 115 Status analysis unit 121 Visual sensor control unit 122 Image processing unit 123 Processing time measurement unit

Claims

1. a program creation unit that provides a function for creating an image processing program that executes image processing on an image captured by a visual sensor; a setting unit that accepts settings related to the image processing, The setting unit, on the display screen, A plurality of adjustment purposes relating to the image processing are displayed in a selectable manner, a programming device that displays one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in such a way that a priority according to the selected one adjustment purpose can be recognized;

2. 2. The programming device according to claim 1, wherein the setting unit displays one or more setting items to which a priority corresponding to the selected adjustment purpose has been assigned in a manner that highlights the one or more setting items to which the priority has not been assigned more highly than the setting items to which the priority has not been assigned.

3. 2. The programming device according to claim 1, wherein the setting unit displays, for one or more setting items to which a priority corresponding to the selected one adjustment purpose has been assigned, setting items having a higher priority corresponding to the selected one adjustment purpose so as to be more strongly emphasized.

4. a storage unit configured to store, for each of the plurality of adjustment purposes, priority information associating at least one setting item related to the adjustment purpose with a priority for the at least one setting item according to the adjustment purpose; 2. The programming device according to claim 1, wherein the setting unit assigns a priority corresponding to the selected adjustment purpose to one or more setting items related to the selected adjustment purpose based on the priority information.

5. 5. The programming device according to claim 4, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on the setting or execution state of the image processing program.

6. 6. The programming device according to claim 5, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on current setting values ​​of the one or more setting items related to the selected adjustment purpose.

7. 7. The programming device according to claim 6, wherein the setting unit lowers the priority of a setting item that is determined to have no room for adjustment based on current setting values ​​of one or more setting items related to the selected adjustment purpose, or cancels the assignment of the priority.

8. a processing time measurement unit capable of measuring processing time for each function corresponding to each setting item when the image processing program is executed; 6. The programming device according to claim 5, wherein the setting unit switches the priority assigned to one or more setting items related to the selected adjustment purpose based on the priority information, based on a measured value of processing time per function corresponding to each setting item when the image processing is executed.

9. 6. The programming device according to claim 5, further comprising a state analysis unit that determines, as the execution state of the image processing program, the state of the image captured by the visual sensor or the stability state of the detection result of the object included in the image processing.

10. the setting unit displays a setting screen including a first area that displays the plurality of adjustment purposes and a second area that displays one or more setting items related to the selected one adjustment purpose; 10. The programming device according to claim 1, wherein the display state of the second area is switched in accordance with a priority corresponding to the selected one adjustment purpose.

11. 10. The programming device according to claim 1, wherein the plurality of adjustment purposes include one or more of: increasing processing speed, taking measures against erroneous detection, taking measures against non-detection, improving detection accuracy, or limiting adjustment items.

12. 10. The programming device according to claim 1, wherein the setting unit additionally displays guidance information corresponding to the selected adjustment purpose for at least one of the one or more setting items displayed so that the priority can be recognized.

13. 1. A method for adjusting image processing settings in a programming device, comprising: a plurality of adjustment purposes relating to the image processing are displayed in a selectable manner in a first area on a display screen; displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that a priority according to the one selected adjustment purpose can be recognized; A setting adjustment method, comprising: receiving an operation for adjusting the one or more setting items in the second area.

14. The computer processor a step of displaying a plurality of adjustment purposes related to image processing in a selectable manner in a first area on a display screen; a step of displaying one or more setting items related to one adjustment purpose selected from the plurality of adjustment purposes in a second area on the display screen so that a priority according to the one selected adjustment purpose can be recognized; a procedure for accepting an adjustment operation for the one or more setting items in the second area; and a program for executing the procedure.