Teaching device

Abstract

Provided is a teaching device (30) comprising a determination unit (154) that determines whether or not a storage condition relating to the result of processing a designated object by a visual sensor is satisfied, and a history storage unit (155) that stores history information indicating the result of processing into a storage device if the storage condition is determined to be satisfied.

Description

Technical Field

[0001] The present invention relates to a teaching device.

Background Art

[0002] There is known a vision detection function that uses an imaging device to detect a specific object from images within a field of view and acquire the position of the detected object. In such a vision detection function, it is common to also have a function of saving the detection results as an execution history.

[0003] Regarding this, Patent Document 1 describes an information management system that "notifies an image processing system 20 from a facility control system 10 of the timing (hereinafter referred to as the 'imaging timing') at which an image of a workpiece 82 to be processed or the like should be taken in each process, and transmits identification information, which is information for identifying (specifying) the workpiece 82 corresponding to the notification, from the facility control system 10 to the image processing system 20" (paragraph 0032).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the history information saving function in the vision detection function, it is desired to be able to save under flexible conditions and to suppress the compression of memory capacity and the increase in cycle time associated with the saving of history information.

Means for Solving the Problems

[0006] One aspect of the present disclosure includes a determination unit that determines whether storage conditions related to the result of processing on an object by a visual sensor are satisfied, and a history storage unit that stores history information as the result of the processing in a storage device when it is determined that the storage conditions are satisfied. , The storage conditions include conditions regarding parameters used for image processing to detect the object from an image obtained by imaging the object with the visual sensor. It is an instruction device.

Effect of the Invention

[0007] According to the above configuration, it becomes possible to store history information under flexible conditions, and it is possible to suppress the pressure on the memory capacity and the increase in cycle time associated with the storage of history information.

[0008] These objects, features, and advantages of the present invention, as well as other objects, features, and advantages, will become more apparent from the detailed description of typical embodiments of the present invention shown in the accompanying drawings.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11

Figure 12

Figure 13A

Figure 13B

Embodiments for Carrying Out the Invention

[0010] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings to be referred to, the same reference numerals are assigned to the same constituent parts or functional parts. For ease of understanding, the scales of these drawings are appropriately changed. Also, the forms shown in the drawings are one example for carrying out the present invention, and the present invention is not limited to the illustrated forms.

[0011] FIG. 1 is a diagram showing the overall configuration of a robot system including a teaching device 30 according to an embodiment. The robot system 100 includes a robot 10, a vision sensor control device 20, a robot control device 50 that controls the robot 10, a teaching operation panel 40, and a storage device 60. A hand 11 as an end effector is mounted at the tip of the arm of the robot 10. Further, a vision sensor 71 is attached to the tip of the arm of the robot 10. The vision sensor control device 20 controls the vision sensor 71. The robot system 100 can detect an object (workpiece W) placed on the workbench 81 by the vision sensor 71, correct the position of the robot 10, and perform handling of the workpiece W. In this specification, the function of detecting an object using the vision sensor 71 may also be referred to as a vision detection function.

[0012] In the robot system 100, the teaching operation panel 40 is used as an operation terminal for performing various teachings (i.e., programming) on the robot 10. When a robot program generated using the teaching operation panel 40 is registered in the robot control device 50, hereinafter, the robot control device 50 can execute control of the robot 10 according to the robot program. In the present embodiment, it is assumed that the teaching device 30 is configured by the functions of the teaching operation panel 40 and the robot control device 50. The functions of the teaching device 30 include a function of teaching the robot 10 (function as a programming device) and a function of controlling the robot 10 according to the teaching content.

[0013] In this embodiment, the teaching device 30 is configured to determine whether to save the history information obtained as a result of executing the process on the object by the vision sensor 71 according to the saving conditions related to the result of the process on the object by the vision sensor 71. Here, the process on the object by the vision sensor 71 may include detection of the object, determination, and various processes using other functions of the vision sensor 71. In this embodiment, as an example, the vision detection function will be taken up for explanation. The teaching device 30 provides a function for programming to realize such a function. Such a function of the teaching device 30 enables the history information to be saved under flexible saving conditions, and also suppresses the compression of the memory capacity and the increase in the cycle time associated with the saving of the history information. Note that the history information as the execution result of the vision detection function includes the captured image (history image), various information related to the quality of the history image, information related to the result of image processing such as pattern matching, and various data generated along with the execution of other vision detection functions.

[0014] The storage device 60 is connected to the robot control device 50 and stores the history information as the execution result of the vision detection function by the vision sensor 71. The storage device 60 may be further configured to store the setting information of the vision sensor 71, the program for vision detection, the setting information, and other various information. The storage device 60 may be an external storage device (USB memory) of the robot control device 50, or may be a computer, a file server, or other data storage devices network-connected to the robot control device 50. In FIG. 1, as an example, the storage device 60 is configured as a device separate from the robot control device 50, but the storage device 60 may be configured as an internal storage device of the robot control device 50 or an internal storage device of the teaching operation panel 40. The function as the teaching device 30 may include the storage device 60.

[0015] The visual sensor control device 20 has a function of controlling the visual sensor 71 and a function of performing image processing on the image captured by the visual sensor 71. The visual sensor control device 20 detects the workpiece W from the image captured by the visual sensor 71 and provides the position of the detected workpiece W to the robot control device 50. Thereby, the robot control device 50 can correct the taught position and execute the taking-out of the workpiece W and the like. The visual sensor 71 may be a camera (2D camera) that captures a grayscale image or a color image, or a stereo camera or a 3D sensor that can acquire a distance image or a 3D point cloud. The visual sensor control device 20 holds the model pattern of the workpiece W and executes image processing for detecting the object by pattern matching between the image of the object in the captured image and the model pattern. The visual sensor control device 20 may hold calibration data obtained by calibrating the visual sensor 71. The calibration data includes information on the relative position of the visual sensor 71 (sensor coordinate system) with respect to the robot 10 (for example, robot coordinate system). In FIG. 1, the visual sensor control device 20 is configured as a separate device from the robot control device 50, but the function as the visual sensor control device 20 may be incorporated in the robot control device 50.

[0016] In addition, as a configuration for detecting the workpiece W using the visual sensor 71 in the robot system 100, in addition to the configuration shown in FIG. 1, there may also be a configuration in which the visual sensor 71 is installed at a fixed position in the work space. Also, in this case, it may be configured to show the workpiece W to the fixedly installed visual sensor 71 by gripping it with the hand of the robot 10.

[0017] FIG. 2 is a diagram showing an example of the hardware configuration of the robot control device 50 and the teaching operation panel 40. The robot control device 50 may have a configuration as a general computer in which a memory 52 (ROM, RAM, non-volatile memory, etc.), an input / output interface 53, an operation unit 54 including various operation switches, etc. are connected to a processor 51 via a bus. The teaching operation panel 40 may have a configuration as a general computer in which a memory 42 (ROM, RAM, non-volatile memory, etc.), a display unit 43, an operation unit 44 constituted by an input device such as a keyboard (or software keys), an input / output interface 45, etc. are connected to a processor 41 via a bus. Note that, as the teaching operation panel 40, a tablet terminal, a smartphone, a personal computer, or other various information processing devices can be used.

[0018] FIG. 3 is a block diagram showing a functional configuration (that is, a functional configuration as the teaching device 30) constituted by the teaching operation panel 40 and the robot control device 50. As shown in FIG. 3, the robot control device 50 includes an operation control unit 151 that controls the operation of the robot 10 according to a robot program or the like, a storage unit 152, a storage condition setting unit 153, a determination unit 154, a history storage unit 155, an outlier detection unit 156, and a learning unit 157.

[0019] The storage unit 152 stores a robot program and other various information. Further, the storage unit 152 may be configured to store storage conditions (denoted by reference numeral 152a in FIG. 3) set by the storage condition setting unit 153.

[0020] The storage condition setting unit 153 provides a function for setting storage conditions for storing history information. The function for setting the storage conditions by the storage condition setting unit 153 is realized by the cooperation of a function for accepting the setting of storage conditions in programming via the function of the program creation unit 141 and a function for setting the storage conditions realized in the robot control device 50 by registering the program created by the function in the robot control device 50. Note that the programming mentioned here includes programming by text-based instructions and programming by instruction icons. These programs will be described later.

[0021] The determination unit 154 determines whether or not the storage conditions are satisfied. The history storage unit 155 stores the history information in the storage device 60 when the determination unit 154 determines that the storage conditions are satisfied.

[0022] The outlier detection unit 156 is responsible for the function of detecting whether or not the value of the data (parameters) included in the history information as the execution result of the vision detection function is an outlier. The learning unit 157 is responsible for the function of learning the storage conditions based on the history information.

[0023] Each function of the robot control device 50 shown in FIG. 3 may be realized, for example, by registering a program (such as a robot program, a program of the vision detection function, etc.) created by the teaching operation panel 40 in the robot control device 50 and the processor 51 of the robot control device 50 executing these programs. Note that at least a part of the functions as the storage unit 152, the storage condition setting unit 153, the determination unit 154, the history storage unit 155, the outlier detection unit 156, and the learning unit 157 in the robot control device 50 may be configured to be mounted on the vision sensor control device 20. In this case, the vision sensor control device 20 may be included in the function as the teaching device 30.

[0024] The teaching operation panel 40 has a program creation unit 141 for creating various programs such as the robot program of the robot 10 and the program for realizing the vision detection function (hereinafter also referred to as the vision detection program). The program creation unit 141 creates and displays a user interface for performing various inputs related to programming, including the input of instructions and detailed settings related to the instructions, that is, a user interface creation unit 142 (hereinafter referred to as the UI creation unit 142), an operation input reception unit 143 that receives various user operations via the user interface, and a program generation unit 144 that generates a program based on the input instructions and settings.

[0025] Through the program creation function of the teaching operation panel 40, the user can create a robot program for controlling the robot 10 and a vision detection program. When the vision detection program is created and registered in the robot control device 50, thereafter, the robot control device 50 can execute an operation of executing the robot program including the vision detection program and handling the workpiece W while detecting the workpiece W using the vision sensor 71.

[0026] In this embodiment, the user can create a program for saving the history information as an execution result when the vision detection function is executed through the function of the program creation unit 141 when the storage condition is satisfied. When such a program is registered in the robot control device 50, thereafter, the robot control device 50 can operate to save the history information only when the storage condition is satisfied. Thereby, it is possible to suppress the compression of the memory capacity and the increase in the cycle time associated with the storage of the history information.

[0027] FIG. 4 is a flowchart showing a process (vision detection and history storage process) of storing history information by a vision detection function in a robot control device 50 based on storage conditions. The vision detection and history storage process is executed, for example, under the control of a processor 51 of the robot control device 50. Note that the process in FIG. 4 targets one workpiece W. When there are a plurality of workpieces to be processed, the process in FIG. 4 may be executed for each workpiece.

[0028] When the vision detection and history storage process is started, first, the workpiece W is imaged by a vision sensor 71 (camera) (step S1). Next, the workpiece model is detected (that is, the workpiece W is detected) for the captured image using pattern matching or the like with the taught workpiece model (step S2). Next, based on the detection result of the workpiece W, the position of the workpiece model (that is, the position of the workpiece W) is calculated (step S3). The position of the workpiece model (the position of the workpiece W) is calculated, for example, as a position in the robot coordinate system.

[0029] When the position of the model (workpiece W) is calculated, next, correction data for correcting the position of the robot 10 is calculated (step S4). The correction data is, for example, data for correcting the taught point.

[0030] Next, the robot control device 50 determines whether or not storage conditions for storing history information are satisfied (step S5). The process in step S5 corresponds to the function of the determination unit 154. When the storage conditions are satisfied (S5: YES), the robot control device 50 writes the history information to the storage device 60 (step S6) and exits this process. The process in step S6 corresponds to the function of the history storage unit 155. Note that after exiting this process, this process may be continued for the next workpiece W. On the other hand, when the storage conditions are not satisfied (S5: NO), this process ends without storing the history information.

[0031] A program for executing vision detection and history storage processing as shown in Fig. 4 can be created as a text-based program or as a program of instruction icons through the function of the program creation unit 141 of the teaching operation panel 40. The UI creation unit 142 mainly provides various user interfaces for programming with instruction icons on the screen of the display unit 43. The user interfaces provided by the UI creation unit 142 include a detailed setting screen for performing detailed settings related to the instruction icons. Examples of such interface screens will be described later.

[0032] The operation input reception unit 143 receives various operation inputs for the program creation screen. For example, the operation input reception unit 143 supports operations such as inputting text-based instructions on the program creation screen, selecting a desired instruction icon from a list of instruction icons and arranging it on the program creation screen, selecting an instruction icon and displaying a detailed setting screen for detailed settings for the icon, and inputting detailed settings through the user interface screen.

[0033] Fig. 5 shows a program 201 as an example when the vision detection and history storage processing of Fig. 4 is realized as a text-based program. In the program 201 of Fig. 5, the numbers on the left of each line represent line numbers. When creating a text-based program 201 as shown in Fig. 5, the user inputs instructions on the program creation screen 210 provided by the program creation unit 141.

[0034] The instruction "Vision Detection '...'" on the first line corresponds to the processing of steps S1 - S3 in Fig. 4, which is for imaging the work W using the vision sensor 71, detecting the work W from the captured image according to the taught work model, and detecting the position of the model (the position of the work W). The " '...'" after the instruction "Vision Detection" specifies the program name (macro name) for executing this processing.

[0035] The instruction on the second line, "Vision Correction Data Stock '...'", corresponds to the process of step S4 in FIG. 4, and is a process of calculating data for correcting the teaching point based on the detection result of the position of the workpiece. The " '...' " after the instruction "Vision Correction Data Stock" specifies the program name (macro name) for executing this process. In the next instruction, "Vision Register [...]", the vision register number for storing the correction data is specified. The three-dimensional position of the corrected teaching point is stored in the vision register specified here.

[0036] The instruction on the third line, "Mos [...]=[...] ", corresponds to the process of step S5 in FIG. 4 and is an instruction for specifying the storage condition. When the specified storage condition is satisfied, the instruction for history storage on the fourth line, "Vision History Save '...'", is executed. If the storage condition is not satisfied, the instruction for history storage on the fourth line is not executed. Thus, by using the vision register specified here, it becomes possible to perform position correction of the robot in the robot program. Note that after the instruction for specifying the vision register, an instruction "Jump Label [...]" for jumping to the specified label may be described in order to execute other processes.

[0037] The instruction on the fourth line, "Vision History Save '...'", corresponds to the process of step S6 in FIG. 4 and is an instruction for saving the history information as the execution result of the above vision detection function. Note that it may be possible to specify the save destination of the history information in the " '...' " part after this instruction.

[0038] FIG. 6 shows a vision detection program 301 as an example of realizing the vision detection and history storage process of FIG. 4 by instruction icons. When creating a vision detection program 301 as shown in FIG. 6, the user arranges icons on the program creation screen 310 provided by the UI creation unit 142 for programming. Here, an example of arranging the icons in the order of execution from top to bottom is shown.

[0039] The vision detection program 301 is composed of the following icons. Vision detection icon 321 Snapshot icon 322 Pattern match icon 323 Condition judgment icon 324

[0040] The vision detection icon 321 is an icon that undertakes a general function of instructing an operation to perform correction based on the vision detection result using one camera. As its internal functions, it includes a snapshot icon 322 and a pattern match icon 323. The snapshot icon 322 corresponds to a command to image an object using one camera. The pattern match icon 323 is an icon that instructs an operation to detect a workpiece by pattern matching on the captured image data. The pattern match icon 323 includes a condition judgment icon 324 as its internal function. The condition judgment icon 324 provides a function of specifying conditions for performing various operations according to the result of pattern matching.

[0041] The vision detection icon 321 is in charge of an operation to obtain correction data for correcting teaching points according to the detection results of workpieces obtained by the snapshot icon 322 and the pattern match icon 323. By the functions of these icons, the vision detection and history saving process shown as a flow in FIG. 4 can be realized.

[0042] In this embodiment, as a saving condition for determining whether to save history information, the saving condition can be set in the following ways. (1) Use the saving condition specified by the user. (2) Detect outliers to perform anomaly detection. (3) Construct the saving condition by learning. (4) Use the preset saving condition.

[0043] The method of using the saving condition specified by the user will be described. The methods of using the storage conditions specified by the user include a method of setting storage conditions in the text-based program shown in FIG. 5 and a method of setting storage conditions via the user interface in the program of the instruction icon shown in FIG. 6. Here, the latter will be described in detail.

[0044] FIG. 7 is an example of a user interface screen 330 for performing detailed settings of the condition determination icon 324. The user interface screen 330 includes a value setting column 341 for specifying the type of value to be used for condition determination and a setting column 342 for specifying conditions based on the set value. In the illustrated example, as the value setting, the score obtained as a result of pattern matching is specified. Also, as the condition setting, "when the value is greater than a constant (here 0.0)" is specified. The user interface screen 330 further includes a pop-up 343 for specifying an operation when the condition is satisfied. Among the menus of this pop-up 343, an item 344 of "save the history image" is included. In this way, by including the value setting and condition setting for saving the history image in the user interface screen 330 for detailed settings of the condition determination icon 324, it is possible to save the history image (history information) under any conditions. Note that FIG. 7 describes an example in which an item of "save the history image" is provided as the operation when the condition is satisfied, but a configuration in which an item of "save only the history information other than the history image" is further provided is also possible. Thereby, the user can select whether to include an image as the history information to be saved. In this case, it is possible to reduce or minimize the amount of data to be stored. Note that a configuration in which a menu for selecting information to be saved (object to be saved) can be presented as the storage condition is also possible. In this configuration, when the condition is satisfied, only the information selected as the storage object can be stored in the storage device 60.

[0045] As a user interface for setting storage conditions, it is also possible to adopt a configuration that uses the user interface screen 350 for detailed settings of the vision detection icon 321 shown in FIG. 8. The user interface screen 350 is configured to include an item for specifying the conditions for storing history information. The user interface screen 350 in FIG. 8 can be activated by performing a predetermined operation while the vision detection icon 321 is selected on the program creation screen 310. The user interface screen 350 in FIG. 8 includes an item 362 for "detailed settings" in the setting menu of an item 361 for specifying image storage. Here, by selecting the item 362 for "detailed settings", it is possible to display a condition setting screen 380 which is a user interface for specifying the storage conditions shown in FIG. 9.

[0046] The condition setting screen 380 in FIG. 9 includes an item 381 for "value setting" for setting the type of value to be used as a condition, and an item 382 for "condition setting" for setting the condition for the set value. In the example of FIG. 9, as the storage condition, "when the score is greater than 0.0 as a result of pattern matching" is specified. The condition setting screen 380 may further include an item 383 for specifying the storage destination for storing the history image when the condition is satisfied.

[0047] An example of setting the storage conditions via the condition setting screen 380 in FIG. 9 will be described with reference to FIGS. 10A and 10B. FIG. 10A shows an example in which the storage conditions are set for the condition setting screen 380. The value setting in FIG. 10A includes setting the following five types of values as the values used for condition setting. Here, the values as parameters obtained as the execution results when a certain pattern matching operation is executed are specified. Value 1: Score of the pattern matching result (sign 301a) Value 2: Vertical position of the image as the range of the detection position (sign 381b) Value 3: Horizontal position of the image as the range of the detection position (sign 381c) Value 4: Contrast of the image (sign 381d) Value 5: Angle of the detected object (reference sign 381e)

[0048] On the condition setting screen of FIG. 10A, the item of "Condition setting" includes the following five conditions for condition setting using the above Value 1 to Value 5. Condition 1: The score (Value 1) is greater than the constant 50 (reference sign 382a) Condition 2: The detection position (Value 2) is in a range greater than the vertical position 100 of the image (reference sign 382b) Condition 3: The detection position (Value 3) is in a range greater than the horizontal position 150 of the image (reference sign 382c) Condition 4: The contrast of the image (Value 4) is 11 or less (reference sign 382d) Condition 5: The rotation angle of the workpiece as the detection result (Value 5) is greater than 62 degrees (reference sign 382e) Condition 1 is a condition to save the history information when the score of the detection result (a value representing the proximity to the taught model) exceeds 50. When Conditions 2 and 3 are set simultaneously, it is a condition to save the history information when the detection position of the workpiece W is in a range where the vertical range within the image 400 is 100 or more and the horizontal range is 150 or more. This range is illustrated as the range 410 specified by hatching in FIG. 10B. For example, such a setting is effective when it is desired to limit the detection range within the image 400. Condition 4 is a condition to save the history information when the contrast of the detected image is 11 or less. Condition 5 is a condition to save the history information when the angle of the object as the detection result (how much it is rotated with respect to the taught model data) is greater than 62 degrees.

[0049] In addition to the above, as an example of the saving conditions, depending on the specific detection results output by individual detection methods, such as the "diameter" which is a feature specific to circle detection, the setting conditions can be specified.

[0050] (2) When detecting outliers for anomaly detection Next, the operation when saving history information according to the result of outlier detection by the outlier detection unit 156 will be described. The image 501 shown on the left side in FIG. 11 is an example of an image when normal detection is performed. On the other hand, when an abnormality such as lens breakage occurs in the visual sensor 71, for example, it is considered that an image without contrast such as the image 551 is captured. Such an abnormality can be detected as an outlier in the contrast of the history image. The outlier detection unit 156 detects a situation where an accident such as breakage of the visual sensor 71 has occurred as an outlier in the imaging data. Then, when such an outlier is detected, the history storage unit 155 stores the captured image as an abnormal state. In this case, a dedicated storage destination 561 for outlier occurrence may be set as the storage destination. The storage destination 561 may be set in advance or may be settable by the user.

[0051] As determination materials (parameters) for detecting an abnormal occurrence (outlier), for example, a score, contrast, position, angle, and size can be used. Here, the contrast is the contrast of the detected image, and the position, angle, and size respectively refer to the position, angle, and size as the difference from the teaching data of the detected object. As determination conditions for the abnormal state, for example, the score is lower than a predetermined value, the contrast is lower than a predetermined value, the difference in the position of the detected object with respect to the position of the taught model data is larger than a predetermined threshold value, the rotation angle of the detected object with respect to the rotation position of the taught model data is larger than a predetermined threshold value, the difference in the size of the detected object with respect to the size of the taught model data is larger than a predetermined threshold value, and the like.

[0052] As specific values of the threshold for detecting outliers, for example, the average value can be used. Based on the average value of the normal-time values, when the value deviates significantly from this (for example, when it is less than 10% of the average value), it may be determined as an outlier. The standard deviation may also be used as an indicator for detecting outliers. For example, there may be cases where a detected value that falls outside the range of three standard deviations is regarded as an outlier. Alternatively, the value of the latest detection result may be considered correct, and outliers may be determined using only the latest detection result as a reference. Other methods known in the art may also be used for detecting outliers.

[0053] Note that such anomaly detection by detecting outliers can also be positioned as "unsupervised learning" because it can be said that the storage conditions are set when outliers occur even if no storage conditions are set in advance.

[0054] (3) When constructing storage conditions by learning The learning unit 157 is configured to learn the relationship between one or more pieces of data (parameters) included in the history information as the detection result by the vision sensor 71 and the storage conditions. The learning of the storage conditions by the learning unit 157 will be described below. Here, there are various learning methods, and here, supervised learning, which is one of machine learning, will be exemplified. Supervised learning is a learning method that uses labeled data as teacher data to learn and construct a learning model.

[0055] The learning unit 157 constructs a learning model using, as input data, data related to the history information as the execution result of the vision detection function, and using teacher data with information related to the storage of the history information as labels. Once the learning model is constructed, it can be used as the storage conditions. As an example, a learning model may be constructed using a three-layer neural network having an input layer, an intermediate layer, and an output layer. It is also possible to perform learning using a so-called deep learning method using a neural network having three or more layers.

[0056] When using a history image as history information as input, a CNN (Convolutional Neural Network) may be used. In this case, as shown in FIG. 12, the input data 601 for the CNN 602 is set as the history image, and teacher data with the label (output) 603 being information related to the storage of history information is used to learn the weighting parameters in the CNN 602 by the error backpropagation method.

[0057] An example of learning using a detected image will be described. In the first example, the detected image is used as input data, and labels of "saved '1'" and "not saved '0'" are assigned as output labels and used as teacher data for machine learning (supervised learning). As illustrated in FIG. 13A, for the detected image, when the user saves it, "saved '1'" is assigned as the label 702, and when the user does not save it, "not saved '0'" is assigned as the label 712, and learning is performed using these as teacher data. When learning is performed with a sufficient number of teacher data (training data) and a learning model is constructed, when an input image 610 as shown in FIG. 13A is given as test data, an output 620 indicating whether it should be saved or not can be obtained.

[0058] The second example of learning using a detected image is to use the detected image as input data, assign the save destination as the output label, and perform machine learning (supervised learning) using these as teacher data. For example, as shown in FIG. 13B, when the detected image is saved in the save destination folder that stores the detection result, "detection folder '1'" is assigned as the label 722. On the other hand, when the detected image is saved in the "undetected folder" where the history image is saved in the case of non-detection, "undetected folder '0'" is assigned as the label 732. Then, machine learning is performed using these as teacher data (training data). When a learning model is constructed by machine learning, when the input image 630 shown in FIG. 13B is given as test data, an output 640 indicating the save destination can be obtained.

[0059] Note that by combining the storage learning function (second learning function) shown in the second example with the learning function (first learning function) regarding whether to store the history information shown in the first example, it is also possible to configure the history information to be stored automatically in a desired storage destination.

[0060] As another example when constructing the storage conditions by learning, there may be an example of using data related to detection results other than images. For example, learning can be performed from teacher data with any one of parameters such as score, contrast, position of the detected object, angle of the detected object, and size of the detected object as input data and whether the history image has been stored as a label. As a method of learning (supervised learning) in this case, regression or classification may be used. As an example, by using the score and data indicating whether the history image has been stored as teacher data, the relationship between whether to store the score and the image (for example, storing the history image when the score is 50 or more) can be obtained.

[0061] In this way, the learning unit learns the relationship (that is, the storage conditions) between the input data included in the history information and the output related to the storage of the history information, and constructs a learning model. Therefore, when the learning model is constructed, thereafter, by inputting the input data into the learning model, it becomes possible to obtain whether the history information should be stored as its output, or the storage destination of the history information.

[0062] (4) When using preset storage conditions In the above, the cases of setting the storage conditions as text-based instructions, as setting information of the instruction icon, as an outlier detection operation, and setting by learning have been described. However, the storage conditions may be preset in the memory (such as memory 42) in the teaching device 30.

[0063] As described above, according to the present embodiment, the history information can be stored under flexible conditions. Further, thereby, it becomes possible to suppress the compression of the memory capacity and the increase in the cycle time associated with the storage of the history information.

[0064] The history information is useful for knowing in what situations an object is detected or cannot be detected, etc., and is useful when improving the object detection method or reviewing the detection environment. By making the storage conditions of the history information flexible as in this embodiment and enabling setting of conditions according to the user's intention, it becomes possible to efficiently collect only the history information useful for improving the detection method.

[0065] As described above, the present invention has been described using typical embodiments. However, those skilled in the art will understand that various changes, omissions, and additions can be made to the above-described embodiments without departing from the scope of the present invention.

[0066] The functional blocks configured in the robot control device shown in FIG. 3 may be realized by the processor of the robot control device executing various software stored in the storage device, or may be realized by a configuration mainly composed of hardware such as an ASIC (Application Specific Integrated Circuit).

[0067] Programs for executing various processes such as vision detection and history storage processing in the above-described embodiments can be recorded on various computer-readable recording media (for example, semiconductor memories such as ROM, EEPROM, and flash memory, magnetic recording media, optical disks such as CD-ROM and DVD-ROM).

Explanation of Reference Numerals

[0068] 10 Robot 11 Hand 20 Vision Sensor Control Device 30 Teaching Device 40 Teaching Operation Panel 41 Processor 42 Memory 43 Display Unit 44 Operation Unit 45 Input / Output Interface 50 Robot Control Device 51 Processor 52 Memory 53 Input / Output Interface 54 Operation Unit 60 Storage Device 71 Vision Sensor 81 Workbench 100 Robot System 141 Program Creation Unit 142 User Interface Creation Unit 143 Operation Input Reception Unit 144 Program Generation Unit 151 Operation Control Unit 152 Storage Unit 152a Saving Condition 153 Saving Condition Setting Unit 154 Judgment Unit 155 History Saving Unit 156 Outlier Detection Unit 157 Learning Unit 201 Program 210, 310 Program Creation Screen 301 Vision Detection Program 330, 350 User Interface Screen 380 Condition Setting Screen 601 Input Data 602 Convolutional Neural Network 603, 702, 712, 722, 732 Label

Claims

1. A determination unit that determines whether a storage condition related to a result of processing on an object by a visual sensor is satisfied; A history storage unit that stores history information as the result of the processing in a storage device when it is determined that the storage condition is satisfied, and includes: The storage condition includes a condition related to a parameter used for image processing for detecting the object from an image obtained by imaging the object with the visual sensor, a teaching device.

2. A determination unit that determines whether a storage condition related to a result of processing on an object by a visual sensor is satisfied; A history storage unit that stores history information as the result of the processing in a storage device when it is determined that the storage condition is satisfied, and includes: The storage condition includes a condition for designating information to be stored among the history information, The history storage unit stores the information to be stored among the history information, a teaching device.

3. The teaching device according to claim 1 or 2, further comprising a storage condition setting unit for setting the storage condition.

4. The teaching device according to claim 3, wherein the storage condition setting unit accepts setting of the storage condition by a text-based instruction.

5. The teaching device according to claim 3, wherein the storage condition setting unit presents a user interface for setting the storage condition on a display screen and accepts setting of the storage condition via the user interface.

6. Further comprising a learning unit that learns the storage condition based on the history information, The determination unit uses the storage condition obtained by learning by the learning unit, the teaching device according to claim 1.

7. A determination unit that determines whether a storage condition related to a result of processing on an object by a visual sensor is satisfied; A history storage unit that stores history information as the result of the processing in a storage device when it is determined that the storage condition is satisfied; A learning unit that learns the storage condition based on the history information, and includes: The learning unit performs first learning using the history information as an input and teacher data having whether the history information is stored as an output label, The determination unit uses the learning model obtained by the first learning as the storage condition, a teaching device.

8. The learning unit further performs second learning using the history information as an input and teacher data having a storage destination of the history information as an output label, The teaching device according to claim 7, wherein the history storage unit determines a storage destination when storing the history information by using the learning model obtained by the second learning.

9. The teaching device further includes an outlier detection unit that detects whether there is an outlier in predetermined data included in the history information. The determination unit uses, as the storage condition, whether the outlier is detected by the outlier detection unit, in the teaching device according to claim 1.

10. The teaching device according to claim 9, wherein the history storage unit stores the history information in a predetermined storage destination when the outlier is detected.

Images