Teaching Device

Abstract

A teaching device (10) comprises: an object detection unit (111) that detects an object from a captured image obtained by imaging the object with a visual sensor; an imaging condition setting unit (112) that sets a plurality of imaging conditions relating to the imaging of the object, and causes the object detection unit (111) to image and detect the object with each of the plurality of imaging conditions; and a detection result determination unit (113) that determines the detection results to formally use, on the basis of an indicator representing statistical properties of a plurality of detection results obtained by imaging and detection under the plurality of imaging conditions.

Description

[Technical Field]

[0001] The present invention relates to a teaching device. [Background technology]

[0002] A visual detection function is known that uses an image processing device to detect a specific object from an image within the field of view of an imaging device and acquires the position of the detected object. By using the position of the object detected by the visual detection function, a robot can operate an object that has not been positioned (see, for example, Patent Document 1).

[0003] In addition, Patent Document 2 states that, regarding the suction / holding ability test program, "the moving device moves the nozzle holder more vigorously than usual, and the imaging device takes images of the electronic circuit component held by the suction nozzle before and after the movement. The image data obtained as a result of the images taken before and after the movement is processed, and if there is no change in the relative position of the electronic circuit component to the suction nozzle before and after the movement, it is determined that the suction nozzle has sufficient ability to hold the electronic circuit component" (abstract). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-113895 [Patent Document 2] Japanese Patent Application Laid-Open No. 2003-304095 Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional robot systems using visual sensors such as that described in Patent Document 1, non-detection or erroneous detection can occur due to the influence of changes in the imaging position or environment when detecting the target workpiece. Non-detection or erroneous detection can cause the system to stop and affect the cycle time, so it is desirable to be able to prevent such occurrences before they occur. [Means for solving the problem]

[0006] One aspect of the present disclosure includes an object detection unit that detects the position of an object from an image obtained by capturing an image of the object using a visual sensor; an image capture condition setting unit that sets a plurality of image capture conditions for capturing the object and causes the object detection unit to capture an image of the object and detect the position of the object under each of the plurality of image capture conditions; and a detection result determination unit that determines the detected position to be officially adopted as the detection result based on an index that represents the statistical properties of a plurality of detection results related to the detected position of the object, obtained by capturing the image and detecting the position under the plurality of image capture conditions. wherein the imaging condition setting unit is configured to allow a user to specify, as the imaging conditions, a detection program for the visual sensor, a register for storing the number of detected objects, the number of imaging positions, and whether or not to automatically generate imaging positions. It is a teaching device. [Effects of the Invention]

[0007] According to the above configuration, it is possible to obtain more accurate detection results, and to prevent non-detection or erroneous detection from occurring.

[0008] These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the invention illustrated in the accompanying drawings. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an overall configuration of a robot system including a teaching pendant according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the hardware configuration of a robot control device and a teaching pendant. [Figure 3] FIG. 2 is a functional block diagram of a teaching pendant, a robot control device, and a visual sensor control device. [Figure 4] FIG. 10 is a diagram showing an example in which checkboxes for automatically setting a plurality of imaging conditions are provided on a parameter setting screen for an execution command or a detection program of a visual detection function. [Figure 5] 10A and 10B are diagrams illustrating an example of a method for automatically generating a plurality of imaging positions around an imaging position instructed or designated by a user. [Figure 6] 10 is a flowchart illustrating an imaging and detection process including a process for determining a detection result from images captured at a plurality of imaging positions. [Figure 7] 10A and 10B are diagrams showing an example of the detection results when workpiece detection is performed at six imaging positions including an imaging position automatically generated by an imaging condition setting unit. [Figure 8] 8 is a flowchart showing a process in which a detection result determination unit determines a formal detection result based on the detection result shown in FIG. 7. [Figure 9] 10A and 10B are diagrams showing an example of a detection result when workpiece detection is performed at three imaging positions including an imaging position automatically generated by an imaging condition setting unit. DETAILED DESCRIPTION OF THE INVENTION

[0010] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings, like components or functional parts are designated by like reference numerals. The scales of these drawings have been changed appropriately to facilitate understanding. Furthermore, the embodiment shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the illustrated embodiment.

[0011] 1 is a diagram showing the overall configuration of a robot system 100 including a teaching pendant 10 according to one embodiment. The robot system 100 includes a robot 30 having a hand (gripping device) 33 mounted on the end of its arm, a robot control device 50 for controlling the robot 30, the teaching pendant 10 as a teaching device, a visual sensor 70 attached to the end of the arm of the robot 30, and a visual sensor control device 20 for controlling the visual sensor 70. The visual sensor control device 20 is connected to the robot control device 50, and the teaching pendant 10 is connected to the robot control device 50. The robot system 100 detects an object (hereinafter referred to as a workpiece) 1 on a workbench 2 using the visual sensor 70, and can handle the workpiece 1 with a hand 33 mounted on the robot 30.

[0012] The robot 30 as an industrial machine is assumed to be a vertical articulated robot here, but other types of robots may also be used. The robot controller 50 controls the operation of the robot 30 in accordance with an operation program loaded into the robot controller 50 or commands input from the teaching pendant 10.

[0013] The visual sensor control device 20 has a function to control the visual sensor 70 and a function to perform image processing on the image captured by the visual sensor 70. The visual sensor control device 20 detects the position of the workpiece 1 from the image captured by the visual sensor 70 and provides the detection result to the robot control device 50. This enables the robot control device 50 to handle the workpiece 1 that has not been positioned. The detection result may include the detected position of the workpiece 1 and an evaluation value related to the detection (detection score, image contrast, etc.).

[0014] The visual sensor 70 may be a camera that captures grayscale or color images, or a stereo camera or 3D sensor that can acquire distance images or 3D point clouds. The visual sensor control device 20 stores a model pattern of the workpiece and can execute image processing to detect the workpiece by pattern matching between the image of the workpiece in the captured image and the model pattern. In this embodiment, the visual sensor 70 is assumed to be calibrated, and the visual sensor control device 20 stores calibration data that defines the relative positional relationship between the visual sensor 70 and the robot 30. This allows positions on the image captured by the visual sensor 70 to be converted to positions on a coordinate system (such as a robot coordinate system) fixed to the workspace.

[0015] In FIG. 1, the visual sensor control device 20 is configured as a device separate from the robot control device 50, but the function of the visual sensor control device 20 may be incorporated into the robot control device 50.

[0016] As will be described below, the robot system 100 is configured to improve detection accuracy and prevent non-detection or erroneous detection from occurring by determining a formal detection result by integrating the results of detection processing performed on each of the images of the target object captured under multiple imaging conditions.

[0017] FIG. 2 is a diagram showing an example of the hardware configuration of the robot control device 50 and the teaching pendant 10. The robot control device 50 may have a configuration similar to that of a general computer, in which a processor 51 is connected to a memory 52 (e.g., ROM, RAM, nonvolatile memory), an input / output interface 53, an operation unit 54 including various operation switches, and the like via a bus. The teaching pendant 10 is used as a device for inputting operations and displaying screens to teach the robot 30 (i.e., to create an operation program). The teaching pendant 10 may have a configuration similar to that of a general computer, in which a processor 11 is connected to a memory 12 (e.g., ROM, RAM, nonvolatile memory), a display unit 13, an operation unit 14 including input devices such as a keyboard and a touch panel (software keys), an input / output interface 15, and the like via a bus. Note that various information processing devices such as a tablet terminal, a smartphone, or a personal computer may be used as the teaching device instead of the teaching pendant 10.

[0018] The visual sensor control device 20 may also have a configuration as a general computer, in which a memory (ROM, RAM, non-volatile memory, etc.), an input / output interface, a display unit, an operation unit, etc. are connected to a processor via a bus.

[0019] FIG. 3 is a functional block diagram of the teaching pendant 10, the robot control device 50, and the visual sensor control device 20.

[0020] The visual sensor control device 20 has a storage unit 121 and an image processing unit 122. The storage unit 121 stores various data (such as model patterns) required for image processing, detection results, calibration data, etc. The calibration data includes the relative positional relationship between the coordinate system set for the robot 30 and the coordinate system set for the visual sensor 70. The calibration data may also include internal parameters related to the imaging optical system (focal length, image size, lens distortion, etc.). The image processing unit 122 is responsible for the function of executing pattern matching and various other image processing.

[0021] The robot control device 50 has a storage unit 151 and an operation control unit 152. The storage unit 151 stores various programs such as operation programs, and various other information used for controlling the robot. The operation programs are provided, for example, from the teaching pendant 10. The operation control unit 152 controls the operation of the robot 30 and the like in accordance with commands or operation programs from the teaching pendant 10. The control by the operation control unit 152 includes control of the hand 33 and control of the visual sensor control device 20.

[0022] 3, the teaching pendant 10 includes a program creation unit 110, an object detection unit 111, an imaging condition setting unit 112, a detection result determination unit 113, an imaging condition adjustment unit 114, and a storage unit 115. The teaching pendant 10 has various functions related to teaching the robot, but FIG. 3 shows functional blocks focusing on the functions for creating programs and performing detection using a visual sensor.

[0023] The program creation unit 110 has various functions related to program creation, such as providing various user interface screens for program creation (teaching), etc. With the support of the program creation unit 110, the user can create various programs such as operation programs and detection programs.

[0024] The object detection unit 111 captures an image of the workpiece 1 using the visual sensor 70 and controls the detection of the workpiece 1 from the captured image. More specifically, the object detection unit 111 sends commands for capturing and detecting the workpiece 1 to the robot control device 50, causing the robot control device 50 to execute the capturing and detection operations. The object detection unit 111 may be realized as a detection program that operates under the control of the processor 11 of the teaching pendant 10. For example, the object detection unit 111 detects an image of the object by matching feature points extracted from the captured image with feature points of a model pattern of the object. Here, the feature points may be edge points. The object detection unit 111 may then determine whether the detection was successful or an error using an evaluation value (e.g., a detection score) that indicates the degree of match between the feature points of the object in the captured image and the feature points of the model pattern.

[0025] The imaging condition setting unit 112 sets a plurality of imaging conditions for imaging the workpiece 1, and causes the object detection unit 111 to perform imaging and detection processing of the workpiece 1 under each of the plurality of imaging conditions.

[0026] For example, the imaging conditions may include at least one of the camera's imaging position (or imaging area), camera exposure time, light intensity of the light source (LED, etc.), gain, binning, detection range, and robot position and orientation. These conditions can affect the detection results. Note that binning refers to a capture method in which multiple pixels on the imaging element are grouped together and treated as a single pixel, and the detection range refers to the range in the captured image that should be available for detection. The robot's position and orientation as an imaging condition is significant as information that defines the camera's position and orientation.

[0027] The detection result determination unit 113 operates to improve the accuracy of the detection result by determining a formal detection result by integrating the detection results under the multiple imaging conditions set by the imaging condition setting unit 112. In this embodiment, the detection result determination unit 113 determines the detection result to be formally adopted based on an index representing the statistical properties of the multiple detection results obtained by performing imaging and detection under the multiple imaging conditions. Here, the index representing the statistical properties may include various statistical quantities such as the mode, mean, median, standard deviation, etc.

[0028] The imaging condition adjustment unit 114 adjusts the imaging conditions previously taught to the object detection unit 111, based on a plurality of detection results obtained by detection under a plurality of imaging conditions set by the imaging condition setting unit 112. Note that the imaging conditions previously taught to the object detection unit 111 are imaging conditions taught by the user via a parameter setting screen or in a format described in the detection program.

[0029] The storage unit 115 is used to store various information including information related to teaching settings, information for programming, and the like.

[0030] Setting of multiple imaging conditions by the imaging condition setting unit 112 will be described. The imaging condition setting unit 112 can automatically generate multiple imaging conditions. The imaging condition setting unit 112 may generate multiple imaging conditions according to a parameter such as the number of imaging conditions specified by the user. The imaging condition setting unit 112 can automatically generate imaging conditions according to the number of imaging conditions specified by the user in the following manner: (a1) A method in which a parameter setting screen for setting detailed parameters of a visual detection function command (command icon, etc.) or a processing program (detection program) includes a check item for automatically generating multiple imaging conditions; (a2) A possible method is to provide a programming syntax for automatically generating a plurality of imaging conditions and use this syntax when performing text-based programming.

[0031] The above method (a1) can be realized, for example, by providing a check box such as "□Automatic setting of imaging conditions" on the parameter setting screen. When the user checks this check box, the imaging condition setting unit 112 automatically generates multiple imaging conditions.

[0032] 4 shows an example in which check boxes for automatically setting multiple imaging conditions are provided on a parameter setting screen for an execution command or detection program for the visual detection function. The parameter setting screen 200 in FIG. 4 includes setting items such as "camera imaging position" 210 and "detection settings" 220. "Detection settings" 220 includes a specification field 221 for specifying a detection program and a specification field 222 for a register for storing the number of detected workpieces. By pressing a teach button 211 in the specification field for "camera imaging position" 210, the user can operate the jog operation button on the teaching console 10 to move the robot 30 and teach the imaging position.

[0033] The parameter setting screen 200 further includes a check box 230 for automatically generating a plurality of imaging positions as imaging conditions, and a specification field 231 for specifying the number of imaging positions in this case. When this check box 230 is checked, the imaging condition setting unit 112 automatically generates the number of imaging positions specified in the specification field for the number of detections, based on the imaging positions instructed by the user in the specification field for "camera imaging position" 210.

[0034] A specific example of the above method (a2) can be realized by providing the following command syntax, for example. Detect position number ("Number of imaging positions", "Detection program name") The above command provides a function to generate the number of imaging positions specified by the argument “number of imaging positions” for the detection program specified by the argument “detection program name.” For example, when the above syntax is described in an operation program, the imaging condition setting unit 112 generates the number of imaging positions specified by the number of imaging positions around the imaging position specified (instructed) by the user in the detection program.

[0035] An example of a method for automatically generating one or more imaging positions around an imaging position instructed by the user will be described with reference to FIG. 5. In this case, the imaging position may also include the posture. The central imaging area 301 shown in FIG. 5 is assumed to be an imaging area on the imaging target surface corresponding to the imaging position instructed in advance by the user in the "Camera imaging position" 210 field on the parameter setting screen 200. The imaging target surface is, for example, the surface on which the workpiece 1 is placed (the upper surface of the workbench 2). In this example, the number of imaging positions designated for automatic generation is assumed to be four.

[0036] 5, the imaging condition setting unit 112 arranges four imaging regions 311-314 at equal angular intervals (90-degree intervals in this example) in the circumferential direction of a circle centered on the center C01 of the imaging region 301. The distances from the center C01 of the imaging region 301 to the centers C11, C12, C13, and C14 of the imaging regions 311, 312, 313, and 314 may be the same. In this case, the distances from the center C01 of the imaging region 301 to the centers C11, C12, C13, and C14 of the imaging regions 311, 312, 313, and 314 may be automatically set by the imaging condition setting unit 112 depending on the extent to which each of the imaging regions 311-314 overlaps with the imaging region 301, or may be configured to be specified by the user. For ease of explanation, the left side of FIG. 5 illustrates the five imaging areas 301, 311-314 as not overlapping one another. However, in reality, the imaging areas 311-314 are arranged to partially overlap the imaging area 301, as shown on the right side of the figure, so that the target object is included in each imaging area. The imaging condition setting unit 112 can determine the imaging positions of the visual sensor 70 corresponding to each imaging area 311-314 based on various conditions, such as the positions of the imaging areas set in this manner, information about the imaging optical system of the visual sensor 70 (focal length, angle of view, etc. of the imaging lens), and the relative positional relationship between the visual sensor 70 and the surface of the target object. The imaging condition setting unit 112 can acquire information necessary for generating the imaging conditions from the robot control device 50. The target detection unit 111 moves the robot 30 (hand 33) so that images of the workpiece 1 are captured at the multiple imaging positions set by the imaging condition setting unit 112 as described above.

[0037] 6 is a flowchart showing an "imaging and detection process" in which a process of determining a detection result from images captured under a plurality of imaging conditions by the functions of the imaging condition setting unit 112 and the detection result determination unit 113 is added to the basic detection process function of the object detection unit 111, which is to capture an image of an object under pre-instructed imaging conditions and perform detection. Note that this process is realized by the teaching operation panel 10, the robot control device 50, and the visual sensor control device 20 working together under the control of the processor 11 of the teaching operation panel 10.

[0038] The processing content of Fig. 6 will be explained. When automatic setting of imaging conditions is specified via the parameter setting screen 200 as described above, or automatic generation of imaging conditions is specified by a command syntax, the imaging condition setting unit 112 generates the number of imaging conditions specified by the user (step S1). Then, the imaging condition setting unit 112 causes the object detection unit 111 to repeatedly execute, for the plurality of imaging conditions, a normal detection process consisting of a process of capturing an image of a workpiece (step S2) and a detection process (image processing) using the captured image (step 3) (loop process L1). Note that the plurality of imaging conditions to be loop-processed here include imaging conditions previously instructed by the user and imaging conditions automatically generated by the imaging condition setting unit 112.

[0039] When the imaging and detection processes under multiple imaging conditions are completed and the process exits the loop, the final detection results are determined by integrating the multiple detection results obtained by imaging under multiple imaging conditions (step S4).

[0040] The detection result determination process in step S4 of Fig. 6 is provided by the function of the detection result determination unit 113. There are various determination examples of the detection result determined by the detection result determination unit 113 based on an index that represents the statistical properties of multiple detection results. As examples, there are the following determination examples (b1) to (b4). (b1) From a plurality of detection results relating to the detection of an object under a plurality of imaging conditions, a detection result to be officially adopted is determined by majority vote based on the number of detection results that match each other. (b2) A formal detection result is determined by averaging multiple detection results regarding the detection of the object under multiple imaging conditions. (b3) The official detection result is determined by evaluating the detection results based on the sum of the scores of multiple detection results obtained under multiple imaging conditions. Under this method, for example, the sum of the scores of a group of detection results that produce matching detection results is used as an evaluation index. For example, when the sum of the scores exceeds a threshold, the result may be adopted as the official detection result. Alternatively, when there are multiple groups with matching detection results, the sum of the scores of each group may be compared to determine which group is adopted as the official detection result. (b4) Remove outliers from multiple detection results and then average them.

[0041] Next, as specific examples of the operation of imaging and detecting an object under the multiple imaging conditions shown in FIG. 6, an operation example (first embodiment) according to the above method (b1) and an operation example (second embodiment) according to the above method (b2) will be described.

[0042] An example of the operation (first embodiment) of the imaging and detection process of an object under multiple imaging conditions according to the above method (b1) will be described with reference to FIGS. 7 and 8. FIG. 7 shows detection results 411-416 obtained when workpiece 1 is detected at six imaging positions, including the imaging position automatically generated by the imaging condition setting unit 112. Each detection result 411-416 includes a captured image and a detection position. Specifically, detection result 411 includes captured image M1 and detection position P1, detection result 412 includes captured image M2 and detection position P2, detection result 413 includes captured image M3 and detection position P3, detection result 414 includes captured image M4 and detection position P4, detection result 415 includes captured image M5 and detection position P5, and detection result 416 includes captured image M6 and detection position P6. The detection position is the three-dimensional position of the workpiece 1 detected from the captured image, which is a two-dimensional image. As described above, the visual sensor control device 20 stores calibration data. This calibration data includes external parameters that define the relative positional relationship of the coordinate system (camera coordinate system) set in the visual sensor 70 with respect to the robot coordinate system, and internal parameters related to the imaging optical system. A transformation matrix set based on this calibration data allows three-dimensional positions in the robot coordinate system to be mapped onto a two-dimensional image. That is, three-dimensional positions in the robot coordinate system are transformed into positions in the camera coordinate system using external parameters, and positions in the camera coordinate system are mapped to positions on the image plane using internal parameters. Based on this mapping, three-dimensional positions in a coordinate system (robot coordinate system) fixed to the workspace can be calculated from positions on the image captured by the visual sensor 70.

[0043] The detection results 411-416 in FIG. 7 are images of the workpiece 1 taken from different imaging positions. Here, the detection position P1 in the detection result 411, the detection position P3 in the detection result 413, and the detection position P5 in the detection result 415 are identical. Furthermore, the detection position P2 in the detection result 412 and the detection position P4 in the detection result 414 are identical, and there is no other identical detection position P6. Note that, in this specification, when the detection positions (detection results) match or are consistent, this also includes cases where the detection positions (detection results) do not exactly match, but the difference is within a predetermined tolerance (e.g., a difference that does not cause problems in the handling of the workpiece 1 by the robot 30). Note that, in the example of FIG. 7, the detection positions P1, P3, and P5 are correct detection positions, while the detection positions P2, P4, and P6 are erroneous detections.

[0044] FIG. 8 is a flowchart showing the process (step S4 in FIG. 6) when the detection result determination unit 113 determines a formal detection result according to the above-mentioned determination method (b1) based on the detection results 411-416 shown in FIG. 7. First, the detection result determination unit 113 compares the six detection positions P1-P6 from the detection results 411-416 (step S11). The detection positions (three-dimensional positions) can be obtained from the position information of the robot 30 at the time of image capture and the detection results on the captured image. In this case, the detection result determination unit 113 recognizes that the detection positions P1, P3, and P5 are the same detection position, and that the detection positions P2 and P4 are the same detection position, and that there is no detection result P6 that is the same (step S12).

[0045] Next, the detection result determination unit 113 adopts the detected positions (in this case, the detected positions P1, P3, and P5 based on the detection results 411, 413, and 415) that are the same in the most captured images as the official detected positions (step S13).

[0046] In this way, by adopting the detection result with the greatest number of matching detection results (i.e., the most frequent value) as the official detection result, it is possible to increase the reliability of the detection results and prevent non-detection or erroneous detection from occurring.

[0047] Next, the adjustment of the imaging conditions instructed in advance to the object detection unit 111 by the imaging condition adjustment unit 114 will be described. As can be understood from the above description, imaging conditions that resulted in a detection result with a smaller number of matching detection results (detection positions P2 and P4, or detection position P6) can be considered to be "imaging conditions with a lower probability of successful detection" than imaging conditions that resulted in a detection result with the largest number of matching detection results (detection positions P1, P3, P5). Conversely, imaging conditions that resulted in a detection result with the largest number of matching detection results (detection positions P1, P3, P5) can be considered to be "imaging conditions with a higher probability of successful detection" than imaging conditions that resulted in a detection result with an even smaller number of matching detection results (detection positions P2 and P4, or detection result P6). Furthermore, with regard to the evaluation value (detection score, etc.) included in the detection result, the imaging condition with a lower evaluation value can be considered to be an imaging condition with a lower probability of successful detection. Furthermore, the higher the evaluation value of an imaging condition, the higher the probability of successful detection.

[0048] Therefore, the imaging condition adjustment unit 114 (c1) The imaging condition that produces a detection result with a larger number of matches is the imaging condition that produces a higher probability of successful detection. (c2) The higher the predetermined evaluation value related to the detection result, the higher the probability of successful detection is. Using at least one of the above criteria, the imaging condition setting unit 112 can extract "imaging conditions that are not suitable for use in imaging and detection" and / or "imaging conditions that are suitable for use in imaging and detection" from the multiple imaging conditions set by the imaging condition setting unit 112. Then, the imaging condition adjustment unit 114 can adjust the imaging conditions that have been taught to the object detection unit 111 in advance, based on the extracted "imaging conditions that are not suitable for use in imaging and detection" and / or "imaging conditions that are suitable for use in imaging and detection." Note that "(c1) The greater the number of matching images in a detection result, the higher the probability of successful detection is the imaging condition" is equivalent to "The greater the number of matching images in a detection result, the lower the probability of successful detection is the imaging condition." Furthermore, "(c2) The higher the predetermined evaluation value related to the detection result is for an imaging condition, the higher the probability of successful detection is for that imaging condition" is equivalent to "The lower the predetermined evaluation value related to the detection result is for that imaging condition, the lower the probability of successful detection is for that imaging condition."

[0049] The imaging condition adjustment unit 114 may extract, according to the above-mentioned criterion (c1), imaging conditions that result in detection results P1, P3, and P5 as "imaging conditions that are considered suitable for use in imaging and detection." Furthermore, according to the above-mentioned criterion (c1), the imaging condition adjustment unit 114 may extract, according to the above-mentioned criterion (c1), imaging conditions that result in detection positions P2 and P4 or detection result P6 as "imaging conditions that are considered unsuitable for use in imaging and detection."

[0050] The detection positions P1, P3, and P5 are correct detection positions, while the detection positions P2, P4, and P6 are erroneous detections. In this case, the detection scores of the detection positions P1, P3, and P5 are higher than the detection scores of the detection results P2 and P4 or the detection result P6. Therefore, the imaging condition adjustment unit 114 may extract the imaging conditions that resulted in the detection results P1, P3, and P5 as "imaging conditions that are considered suitable for use in imaging and detection" according to the above-mentioned judgment criterion (c2). Furthermore, the imaging condition adjustment unit 114 may extract the imaging conditions that resulted in the detection positions P2 and P4 or the detection result P6 as "imaging conditions that are considered unsuitable for use in imaging and detection" according to the above-mentioned judgment criterion (c2).

[0051] When criterion (c2) is used together with criterion (c1), for example, (d1) Even if the “imaging conditions positioned as suitable for use in imaging and detection” extracted by the judgment method (c1) result in a detection result in which the detection score is lower than a predetermined value, the “imaging conditions positioned as suitable for use in imaging and detection” are not used. (d2) When there are two groups with the same number of matching detection results, the imaging conditions that produced the detection results of the group with the higher detection score (average value, total value, etc.) are determined to be "imaging conditions that are considered suitable for use in imaging and detection." The following operation may be performed.

[0052] The imaging condition adjustment unit 114 stores the extracted "imaging conditions deemed unsuitable for use in imaging and detection" and / or "imaging conditions deemed suitable for use in imaging and detection" in the storage unit 115 so that they can be used to adjust the imaging conditions. Here, the storage unit 115 can include any storage device that can be configured within the memory 52, such as a storage area for variables that can be referenced from a program, or a file in a non-volatile memory. Alternatively, the storage unit 115 may be configured outside the teaching pendant 10.

[0053] For example, the imaging condition adjustment unit 114 may operate to output a message urging the user to change the imaging conditions when the imaging conditions instructed by the user or designated in advance for the detection program correspond to "imaging conditions deemed unsuitable for use in imaging and detection," or may operate to update the imaging conditions instructed by the detection program or set in advance to "imaging conditions deemed suitable for use in imaging and detection" when "imaging conditions deemed suitable for use in imaging and detection" are stored in the storage unit 115. By such an operation, it is possible to prevent the object detection unit 111 from using "imaging conditions deemed unsuitable for use in imaging and detection" as imaging conditions.

[0054] According to the above configuration, in cases where detection errors occur frequently during actual operation when a detection program is executed using pre-instructed imaging conditions, the process shown in FIG. 6 can be executed once to obtain "imaging conditions that are deemed suitable for use in imaging and detection" or "imaging conditions that are deemed unsuitable for use in imaging and detection," thereby making it possible to appropriately adjust the imaging conditions.

[0055] Next, an example (second embodiment) of the operation of imaging and detecting an object under multiple imaging conditions according to the above-mentioned method (b2) "a method of determining a formal detection result by averaging multiple detection results relating to the detection of an object under multiple imaging conditions" will be described with reference to FIG. 9. As shown in FIG. 9, detection results 511-513 are obtained when workpiece 1 is detected at three imaging positions, including the imaging position automatically generated by the imaging condition setting unit 112. Detection result 511 includes captured image M11 and detection position P11, detection result 512 includes captured image M12 and detection position P12, and detection result 513 includes captured image M13 and detection position P13. Note that detection positions P11, P12, and P13 in the example of FIG. 9 are all correct detection positions (detection results that are not erroneous detections).

[0056] The detection result determination unit 113 adopts the average value of the detection positions P11-P13 as the official detection position as the three detection results. Since the three-dimensional position of the workpiece 1 as the detection result generally varies, the detection accuracy can be improved by averaging these detection positions (for example, by taking the average value of the detection positions P11-P13 as the official detection position). Note that if the detection positions P11, P12, and P13 include a false detection (one whose evaluation value is lower than a predetermined value), the false detection detection position may be removed from the averaging. Note that in this example, an example has been described in which the average value is determined as the official detection result, but there may also be an example in which the median is determined as the official detection result.

[0057] Even when the detection result determination unit 113 performs the above-described operation using the above-described method (b2), the imaging condition adjustment unit 114 can extract "imaging conditions that are deemed unsuitable for use in imaging and detection" and / or "imaging conditions that are deemed suitable for use in imaging and detection" from the multiple imaging conditions set by the imaging condition setting unit 112, using the above-described judgment criterion (c2) "the higher the predetermined evaluation value related to the detection result, the higher the probability of successful detection is for the imaging conditions." Then, the imaging condition adjustment unit 114 can adjust the imaging conditions as described above, that is, adjust the imaging conditions so that "imaging conditions that are deemed unsuitable for use in imaging and detection" are not used as imaging conditions by the object detection unit 111, or update the imaging conditions previously taught to the object detection unit 111 with "imaging conditions that are deemed suitable for use in imaging and detection."

[0058] In addition, since there may be variations in the detection positions P1, P3, and P5 in the first embodiment shown in Figure 7, the detection result determination unit 113 may average the detection positions P1, P3, and P5 adopted in the first embodiment to determine the official detection positions in the first embodiment.

[0059] As described above, according to this embodiment, it is possible to obtain detection results with higher accuracy, and to prevent undetected hairs and erroneous detections from occurring.

[0060] Although the present invention has been described using exemplary embodiments, those skilled in the art will appreciate that modifications and various other changes, omissions, and additions can be made to the above-described embodiments without departing from the scope of the present invention.

[0061] The robot system described in the above embodiment is configured to mount a visual sensor on the robot and capture images of a workpiece placed on a workbench. However, the various functions described in the above embodiment may also be applied to a system in which the visual sensor is fixed within a workspace in which the robot is installed, and the robot grips a workpiece and moves to present the workpiece to the visual sensor. The multiple imaging positions set by the imaging condition setting unit 112 described with reference to FIG. 5 are imaging positions of the visual sensor 70 relative to the workpiece. Therefore, in such a system configuration, the object detection unit 111 may control (i.e., issue a command to the robot control device) the robot to move the workpiece gripped by the robot so that the visual sensor 70 captures images of the workpiece at the multiple imaging positions set by the imaging condition setting unit 112. In this case, the imaging target surface may be defined as a surface including the surface of the workpiece.

[0062] In the above embodiment, the operation of the imaging condition setting unit 112 to automatically generate multiple imaging conditions has been described. However, the imaging condition setting unit 112 may also operate to accept user input specifying multiple imaging conditions and cause the object detection unit 111 to use the multiple imaging conditions input by the user.

[0063] The arrangement of the functions in the functional block diagram shown in Fig. 3 is an example, and various modifications regarding the arrangement of the functions are possible. For example, a configuration example is possible in which the object detection unit 111, the imaging condition setting unit 112, the detection result determination unit 113, the imaging condition adjustment unit 114, and the storage unit 115, which are arranged in the teaching pendant 10, are arranged on the robot control device 50 side. In this case, the functions of the visual sensor control device 20 may also be included in the robot control device 50. In this case, the entire functions provided by the teaching pendant 10 and the robot control device 50 can also be defined as a teaching device.

[0064] The functional blocks of the teaching operation panel 10, the robot control device 50, and the visual sensor control device 20 shown in Figure 3 may be realized by the processors of these devices executing various software stored in memory devices, or may be realized by a hardware-based configuration such as an ASIC (Application Specific Integrated Circuit).

[0065] The programs that execute various processes such as the imaging and detection processes in the above-described embodiments can be recorded on various computer-readable recording media (e.g., semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, and optical discs such as CD-ROM and DVD-ROM). [Explanation of symbols]

[0066] 1 Work 2 workbenches 10 Teaching control panel 11 processors 12 Memory 13 Display section 14 Control section 15 Input / Output Interface 20 Visual sensor control device 30 Robot 33 hands 50 Robot control device 51 processors 52 memory 53 Input / Output Interface 54 Operation section 70 Visual Sensor 100 Robot Systems 110 Program Creation Department 111 Object detection unit 112 Imaging condition setting unit 113 Detection result determination unit 114 Imaging condition adjustment unit 115 Storage section 151 Storage section 152 Motion control section 121 Storage section 122 Image processing section 200 Parameter setting screen 411-416 Detection results 511-513 Detection results

Claims

1. an object detection unit that detects the position of an object from an image obtained by capturing an image of the object using a visual sensor; an imaging condition setting unit that sets a plurality of imaging conditions related to imaging of the object and causes the object detection unit to capture an image of the object and detect a position of the object under each of the plurality of imaging conditions; a detection result determination unit that determines a detection position to be officially adopted as a detection result based on an index that represents a statistical property of a plurality of detection results related to the detection position of the object, the detection results being obtained by performing the imaging and position detection under the plurality of imaging conditions; the imaging condition setting unit is configured to allow a user to specify, as the imaging conditions, a detection program for the visual sensor, a register for storing the number of detected objects, the number of imaging positions, and whether or not to automatically generate imaging positions; Teaching device.

2. The teaching device according to claim 1 , wherein the detection result determination unit determines the detection result to be officially adopted based on a mode value of the plurality of detection results as the index.

3. The teaching device according to claim 2 , wherein the detection result determination unit further averages one or more of the plurality of detection results determined as the officially adopted detection result, and uses the averaged result as the officially adopted detection result.

4. an object detection unit that detects an object from a captured image obtained by capturing an image of the object using a visual sensor; an imaging condition setting unit that sets a plurality of imaging conditions related to imaging of the object and causes the object detection unit to capture and detect the object under each of the plurality of imaging conditions; a detection result determination unit that determines a detection result to be officially adopted based on an index that represents a statistical property of a plurality of detection results obtained by performing the imaging and detection under the plurality of imaging conditions; an imaging condition adjustment unit that adjusts imaging conditions that have been taught to the object detection unit in advance based on the plurality of detection results, the detection result determination unit determines the detection result to be officially adopted based on a mode value of the plurality of detection results as the index; The imaging condition adjustment unit (1) The imaging conditions that result in a detection result with a larger number of matches are imaging conditions with a higher probability of successful detection. (2) The higher the predetermined evaluation value related to the detection result, the higher the probability of successful detection is. extracting, from the plurality of imaging conditions, imaging conditions that are not suitable for use in the imaging and detection, or imaging conditions that are suitable for use in the imaging and detection, using at least one of the criteria above; adjusting the imaging conditions previously taught to the object detection unit based on the extracted imaging conditions that are not suitable for use in the imaging and detection, or the extracted imaging conditions that are suitable for use in the imaging and detection; Teaching device.

5. The teaching device according to claim 4, wherein the imaging condition adjustment unit extracts imaging conditions that are deemed unsuitable for use in the imaging and detection, and adjusts the imaging conditions that are deemed unsuitable for use in the imaging and detection so that they are not used as imaging conditions by the object detection unit.

6. The teaching device according to claim 4, wherein the imaging condition adjustment unit extracts imaging conditions that are deemed suitable for use in the imaging and detection, and updates the imaging conditions that have been taught to the object detection unit in advance with the imaging conditions that are deemed suitable for use in the imaging and detection.

7. The teaching device according to claim 1 , wherein the detection result determination unit determines the formal detection result by using an average value of the plurality of detection results as the index.

8. an object detection unit that detects an object from a captured image obtained by capturing an image of the object using a visual sensor; an imaging condition setting unit that sets a plurality of imaging conditions related to imaging of the object and causes the object detection unit to capture and detect the object under each of the plurality of imaging conditions; a detection result determination unit that determines a detection result to be officially adopted based on an index that represents a statistical property of a plurality of detection results obtained by performing the imaging and detection under the plurality of imaging conditions; an imaging condition adjustment unit that adjusts imaging conditions that have been taught to the object detection unit in advance based on the plurality of detection results, the detection result determination unit determines the formal detection result using an average value of the plurality of detection results as the index; the imaging condition adjustment unit extracts, from the plurality of imaging conditions, imaging conditions that are deemed unsuitable for use in the imaging and detection, or imaging conditions that are deemed suitable for use in the imaging and detection, using a judgment criterion that an imaging condition that brings about a result with a higher predetermined evaluation value related to the detection result is an imaging condition with a higher probability of successful detection; adjusting the imaging conditions previously taught to the object detection unit based on the extracted imaging conditions that are not suitable for use in the imaging and detection, or the extracted imaging conditions that are suitable for use in the imaging and detection; Teaching device.

9. The teaching device according to claim 8, wherein the imaging condition adjustment unit extracts imaging conditions that are deemed unsuitable for use in the imaging and detection, and adjusts the imaging conditions that are deemed unsuitable for use in the imaging and detection so that they are not used as imaging conditions by the object detection unit.

10. The teaching device according to claim 8, wherein the imaging condition adjustment unit extracts imaging conditions that are deemed suitable for use in the imaging and detection, and updates the imaging conditions that have been taught to the object detection unit in advance with the imaging conditions that are deemed suitable for use in the imaging and detection.

11. The teaching device according to claim 1 , wherein the imaging condition setting unit sets the plurality of imaging conditions by generating one or more imaging conditions based on one imaging condition that serves as a reference.

12. The teaching device according to claim 11 , wherein the one reference imaging condition is an imaging condition that is taught to the object detection unit in advance.

13. the imaging condition is an imaging position of the visual sensor, The teaching device according to claim 11 or 12, wherein the imaging condition setting unit determines one or more imaging positions around the single reference imaging position so that one or more imaging areas on the imaging target surface at the one or more imaging positions partially overlap with the imaging area on the imaging target surface at the single reference imaging position.

14. The visual sensor is mounted on a robot, The teaching device according to claim 13 , wherein the object detection unit moves the robot so that images of the object are taken at a plurality of image capturing positions set by the image capturing condition setting unit.

15. the visual sensor is fixed to a workspace in which a robot equipped with a hand is installed, the robot grasps the object with the hand, The teaching device according to claim 13 , wherein the object detection unit moves the robot so that images of the object are taken at a plurality of image capturing positions set by the image capturing condition setting unit.

Images