Work status determination device, method, and program

Abstract

To determine an operation state including a normal system and an abnormal system.SOLUTION: A determination device 36 for acquiring data detected by a force sensor which detects a force applied on the hand tip of a robot hand by an acquisition unit 32 and determining an operation state of the robot hand by a determination unit 38 is configured such that a first determination unit for extracting characteristics of an operation state of a normal state and a second determination unit for extracting characteristics of an operation state of an abnormal state are coupled together, and the data acquired by the acquisition unit 32 is input to the determination device 36 that outputs three states of operation completion, operation continuance, and inhibited operation continuance as operation states, and an operation state is determined.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a work state determination device, a work state determination method, and a work state determination program. [Background technology]

[0002] In assembly work by robots at manufacturing sites, technology is being researched to simplify adjustment work without using jigs by increasing the intelligence of the robots. For example, a judgment method using an RNN (Recurrent Neural Network) has been proposed for the assembly work of workpieces with snap fits (see Non-Patent Document 1). This method focuses on the transient changes in the reaction force of the robot's hand when the workpiece is fitted, and configures and learns an RNN with a small time width that captures the transient waveform.

[0003] Also, for example, a robot system has been proposed that judges whether a fitting or insertion operation is successful or not, which depends on various causes (see Patent Document 1). In this system, the range of presence of feature values ​​when the operation is successful and the range of presence of feature values ​​when the operation fails are calculated for each cause of failure, the area of ​​presence of each feature value is recorded, and the success or failure of the operation and the cause of failure are judged based on which feature area the feature value falls within when the operation is performed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-230245 A [Non-patent literature]

[0005] [Non-Patent Document 1] M. Karlsson, A. Robertsson, and R. Johansson, "Detection and control of contact force transients in robotic manipulation without a force sensor", In 2018 IEEE International Conference on Robotics and Automation (ICRA), 2018. Summary of the Invention [Problem to be solved by the invention]

[0006] In the conventional technology, it is possible to determine whether an assembly operation is complete or not. However, in an actual manufacturing site, for example, there may be an abnormal state in which the assembly operation cannot be continued unless the workpiece is changed over. In the conventional technology, there is a problem that such an abnormal state cannot be determined, or that the determiner needs to be adjusted for each product by feature engineering.

[0007] The present invention has been made in consideration of the above points, and has an object to determine an operation state including a normal state and an abnormal state. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the work state determination device of the present invention includes an acquisition unit that acquires data detected by a force sensor that detects the force acting on the tip of a robot hand, and a determiner that determines the work state of the robot hand, which is a determiner that combines a first determiner that extracts characteristics of a normal work state and a second determiner that extracts characteristics of an abnormal work state, and which inputs the data acquired by the acquisition unit into a determiner that outputs three states, namely, work completed, work continuing, and work unable to continue, as the work state, and determines the work state.

[0009] In addition, the time series data acquired by the acquisition unit may be input to the determiner, and the time width of the data input to the second determiner may be longer than the time width of the data input to the first determiner.

[0010] In addition, the time width for the first determiner may be a time width corresponding to a transient response of data occurring when a task is completed, and the time width for the second determiner may be a time width corresponding to the time from when an abnormality occurs to when the operation of the robot hand is stopped.

[0011] The first classifier may be trained using learning data in which, when the work state transitions from continuing work to completing work, each piece of data in the time series is labeled as continuing work and each piece of data after the transition is labeled as completing work, and the second classifier may be trained using learning data in which, when the work state transitions from continuing work to not being able to continue work, each piece of data before the transition is labeled as continuing work and each piece of data after the transition is labeled as not being able to continue work.

[0012] The work state determination device according to the present invention may further include a learning unit that uses the learning data to execute learning of the determiner.

[0013] The learning unit may generate pseudo learning data using the learning data and a random number, and add the pseudo learning data to the learning data for use in learning the classifier.

[0014] In addition, the acquisition unit may further acquire the position of the end effector of the robot hand, and the judgment unit may input the data detected by the force sensor and the position of the end effector to the judger to judge the working state.

[0015] In addition, the decision unit may include a first neural network constituting the first decision unit and a second neural network constituting the second decision unit, and output a result of combining an output of the first neural network and an output of the second neural network.

[0016] Moreover, the work state determination device according to the present invention may be configured to further include a control unit that controls an operation of the robot hand based on the work state determined by the determination unit.

[0017] In addition, the work state determination method of the present invention is a method in which an acquisition unit acquires data detected by a force sensor that detects the force acting on the tip of a robot hand, and a determination unit is a determiner that determines the work state of the robot hand, and a first determiner that extracts characteristics of a normal work state and a second determiner that extracts characteristics of an abnormal work state are combined, and the data acquired by the acquisition unit is input into a determiner that outputs three work states, namely, work completed, work continuing, and work cannot be continued, to determine the work state.

[0018] In addition, the work state determination program of the present invention is a program for causing a computer to function as a determination unit that determines the work state by inputting the data acquired by the acquisition unit into a determiner that outputs three work states, namely, work completed, work continuing, and work unable to continue, and that includes an acquisition unit that acquires data detected by a force sensor that detects the force acting on the tip of a robot hand, and a determiner that determines the work state of the robot hand, the determiner being combined with a first determiner that extracts characteristics of a normal work state and a second determiner that extracts characteristics of an abnormal work state, and that outputs three work states, namely, work completed, work continuing, and work unable to continue, Effect of the Invention

[0019] According to the work state determination device, method, and program of the present invention, work states including normal and abnormal states can be determined. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 is a configuration diagram showing an outline of a robot system. [Diagram 2] FIG. 13 is a diagram for explaining a working state. [Diagram 3] 2 is a block diagram showing a hardware configuration of the work state determination device. FIG. [Figure 4] 2 is a block diagram showing an example of a functional configuration of the work state determination device; [Diagram 5] FIG. 13 is a diagram for explaining generation of learning data. [Figure 6] FIG. 13 illustrates an example of a determiner. [Figure 7] 13 is a flowchart showing the flow of a learning process. [Figure 8] 13 is a flowchart showing a flow of a determination control process. [Figure 9] FIG. 13 is a diagram for explaining generation of pseudo learning data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, an example of an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensions and ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0022] 1, a robot system 100 according to this embodiment includes a work state determination device 10 and a robot unit 50. The robot unit 50 includes a manipulator 52 including a gripper 56 as an end effector, and a force sensor 54. The manipulator 52 is an example of a "robot hand" of the disclosed technology, and the gripper 56 is an example of a "end effector" of the disclosed technology.

[0023] The manipulator 52 is, for example, a vertical articulated robot having six degrees of freedom required for operation in a three-dimensional space. The degrees of freedom of the manipulator 52 may be seven degrees of freedom, which is six degrees of freedom plus a redundant degree of freedom. The manipulator 52 drives each joint so that the rotation angle of each joint is an angle according to control information (described in detail later) output from the work state determination device 10. As a result, the gripper 56 is controlled to be in a specified position and posture. The manipulator 52 also outputs the current rotation angle of each joint to the work state determination device 10.

[0024] It is preferable that the gripper 56 has the versatility to grip a variety of workpieces in a variety of gripping states and is configured to prevent significant gripping misalignment. For example, the gripper 56 may be a parallel two-finger gripper with anti-slip rubber attached to the surface that grips the workpiece.

[0025] The force sensor 54 is installed between the wrist of the manipulator 52 and the gripper 56, detects external force values ​​in six directions, namely, three parallel axes and three rotational axes, acting on the gripper 56, and outputs the values ​​as force sensor values. The force sensor values ​​are an example of "data detected by a force sensor" of the disclosed technology.

[0026] The work state determination device 10 controls the operation of the manipulator 52 while determining the work state of the manipulator 52. In this embodiment, as shown in Fig. 1, it is assumed that a main work 60 held by a gripper 56 is inserted into a sub-work 62 on a workbench. The main work 60 is, for example, a connector, and the sub-work 62 is, for example, a socket. In this embodiment, it is assumed that the main work 60 and the sub-work 62 are joined by a snap fit when the main work 60 is appropriately inserted into the sub-work 62.

[0027] In this embodiment, the work states to be determined are set as work complete, work cannot be continued, and work continue, as shown in Fig. 2. Work complete is a state in which insertion of the main workpiece 60 into the sub-workpiece 62 is completed. Work cannot be continued is a state in which the tip of the gripper 56 interferes with the sub-workpiece 62 before insertion of the main workpiece 60 into the sub-workpiece 62 is completed, and work cannot be continued unless the gripper 56 changes the grip of the main workpiece 60. Work continue is a state in which insertion of the main workpiece 60 into the sub-workpiece 62 can be continued.

[0028] Fig. 3 is a block diagram showing the hardware configuration of the work state determination device 10 according to this embodiment. As shown in Fig. 3, the work state determination device 10 has a CPU (Central Processing Unit) 12, a memory 14, a storage device 16, an input device 18, an output device 20, a storage medium reading device 22, and a communication I / F (Interface) 24. Each component is connected to each other via a bus 26 so as to be able to communicate with each other.

[0029] The storage device 16 stores a working state determination program for executing a learning process and a determination control process, which will be described later. The CPU 12 is a central processing unit, and executes various programs and controls each component. That is, the CPU 12 reads the program from the storage device 16 and executes the program using the memory 14 as a working area. The CPU 12 controls each of the components and performs various calculation processes according to the program stored in the storage device 16.

[0030] The memory 14 is made up of a RAM (Random Access Memory) and temporarily stores programs and data as a working area. The storage device 16 is made up of a ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including an operating system, and various data.

[0031] The input device 18 is a device for performing various inputs, such as a keyboard or a mouse. The output device 20 is a device for outputting various information, such as a display or a printer. A touch panel display may be used as the output device 20 to function as the input device 18. The storage medium reader 22 reads data stored in various storage media, such as a CD (Compact Disc)-ROM, a DVD (Digital Versatile Disc)-ROM, a Blu-ray disc, and a USB (Universal Serial Bus) memory, and writes data to the storage media. The communication I / F 24 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0032] Next, the functional configuration of the work state determination device 10 according to the first embodiment will be described.

[0033] Fig. 4 is a block diagram showing an example of the functional configuration of the work state determination device 10. As shown in Fig. 4, the work state determination device 10 includes, as its functional configuration, an acquisition unit 32, a learning unit 34, a determination unit 38, and a control unit 40. Each functional configuration is realized by the CPU 12 reading out a work state determination program stored in the storage device 16, expanding it in the memory 14, and executing it. In addition, a determiner 36 is stored in a predetermined storage area of ​​the work state determination device 10.

[0034] The acquisition unit 32 acquires learning data input to the working state determination device 10 during the learning process. The learning data is data in which the force sensor value detected by the force sensor 54 at each time, information on the hand position of the gripper 56 at each time, and a label indicating the correct answer of the working state at each time are associated with each other. The information on the hand position of the gripper 56 (hereinafter simply referred to as the "hand position") may be the absolute position of a predetermined position such as the tip of the gripper 56, or may be the relative movement amount or absolute movement amount of a predetermined position during one time step. The hand position is obtained by a matrix calculation based on the rotation angle of each joint (hereinafter referred to as the "joint angle value") output from the manipulator 52 and the kinematics information of the manipulator 52.

[0035] Here, the generation of learning data will be described. As described above, in this embodiment, three work states, work completion, work not continuing, and work continuing, are determined. Therefore, when the work state transitions from work continuing to work completion, as shown by the dashed line in the left diagram of FIG. 5, the time corresponding to the peak of the transient response in the force sensor value is specified as the time when the work state is switched. This is to specify the timing when the work is completed based on the waveform indicating the transient response that occurs when the main work 60 is inserted into the sub-work 62 in the work to which the snap fit is applied, which is assumed in this embodiment. In addition, in FIG. 5, the force sensor value represents the force sensor value related to the vertical force acting on the gripper 56. Then, using the specified time as the criterion for the transition of the work state, the data at each time before the transition of the time series data of the force sensor value and the hand position is labeled as work continuation, and the data at each time after the transition is labeled as work completion.

[0036] Furthermore, when the work state transitions from continue work to unable to continue work, as shown by the dashed line in the right diagram of FIG. 5, the timing at which the manipulator 52 stops moving is identified as the time at which the work state switched. The timing at which the manipulator 52 stops moving may be the timing at which the force sensor value becomes equal to or less than a predetermined value. This is based on the phenomenon in which the force sensor value gradually decreases due to the gripper 56 interfering with the sub-workpiece 62, and captures the timing at which the manipulator 52 stops. Then, using the identified time as the criterion for the transition of the work state, a label of "continue work" is attached to the data at each time before the transition of the time series data of the force sensor value and the hand position, and a label of "unable to continue work" is attached to the data at each time after the transition.

[0037] When the learning data is generated as described above, the number of learning data assigned with the labels of "task completed" and "task not continued" is smaller than the number of learning data assigned with the label of "task continued," resulting in a bias in the number of learning data. Therefore, the learning data with the fewer labels may be copied and increased so that the number of learning data for each label is approximately the same. This can prevent overlearning and prevent the generalization performance from being impaired.

[0038] When the acquisition unit 32 acquires the learning data, the acquisition unit 32 passes the acquired learning data to the learning unit 34. Hereinafter, the learning data when the work state transitions from continuing the work to completing the work is referred to as "normal learning data", and the learning data when the work state transitions from continuing the work to not being able to continue the work is referred to as "abnormal learning data".

[0039] During the judgment control process, the acquisition unit 32 also acquires force sensor values ​​output from the force sensor 54 and joint angle values ​​output from the manipulator 52. The acquisition unit 32 calculates the hand position by matrix calculation based on the acquired joint angle values ​​and kinematics information of the manipulator 52. The acquisition unit 32 associates the acquired force sensor values ​​with the calculated hand position to prepare target data and transfers them to the judgment unit 38.

[0040] The learning unit 34 uses the learning data transferred from the acquisition unit 32 to execute learning of parameters of a determiner 36 that determines the working state of the manipulator 52. The determiner 36 may be configured, for example, by a DNN (Deep Neural Network). In this embodiment, a case will be described in which the determiner 36 is configured by an RNN (Recurrent Neural Network) targeted at time-series data.

[0041] An example of the classifier 36 is shown in Fig. 6. In the example of Fig. 6, the classifier 36 is configured by combining a first classifier 36A that extracts features of a normal working state with a second classifier 36B that extracts features of an abnormal working state. The classifier 36 includes an input layer, an intermediate layer, and an output layer. A vector value x i (t) is input. i is a number for distinguishing between the first decision unit 36A and the second decision unit 36B, where i=1 represents the first decision unit 36A and i=2 represents the second decision unit 36B. i (t) is the vector value F which is the force sensor value at time t (t) and the vector value of the hand position P (t) t is a vector value that combines 1, 2, . . . , T i It is.

[0042] Each neuron in the hidden layer receives a weighted value output from the corresponding neuron in the input layer. Also, each neuron in the hidden layer receives a weighted value output from the neuron in the hidden layer one time period earlier. Each neuron in the hidden layer applies an activation function to the input value, and outputs the result to the neuron in the hidden layer one time period later. For example, each neuron in the hidden layer receives a vector value h i (t) may be output. h i (1) =tanh(b i +U i x i (1) ) h i (t)=tanh(b i +W i h i (t-1) +U i x i (t) ) i∈[1;2],t∈[2;T i ]

[0043] U i is a parameter that represents the weight applied to the value output from each neuron in the input layer, and W i is a parameter that represents the weight applied to the value output from each neuron in the hidden layer one time before. Also, tanh is an example of an activation function, and b i is a parameter representing the bias. The output h1 of the intermediate layer of the first decider 36A (T_1) is a vector value obtained by converting the force sensor value and the hand tip position from time t=1 to t=T1 into features. Also, the output h2 of the intermediate layer of the second decision unit 36B (T_2) is a vector value obtained by converting the force sensor value and the hand position from time t=1 to t=T2 into feature quantities.

[0044] The output layer combines the output from the first decider 36A and the output from the second decider 36B, for example, as shown below, and outputs a decision result. o=c+V1h1 (T_1) +V2h2 (T_2) y^=softmax(o) V i t=T i The output from the hidden layer neuron h i (T_i) is a weight related to the task, c is a parameter, and softmax is an example of an activation function. y^ (in FIG. 6, a "^ (hat)" is placed above "y") is the task state determination result, and is a three-dimensional vector representing the probability that the current task state is the task complete, the probability that the task cannot be continued, and the probability that the task continues. The output layer may be configured to output the task state with the highest probability as the determination result.

[0045] Here, the time width T of the data input to each of the first decision unit 36A and the second decision unit 36B is i (i∈[1;2]) will be described. As described above, when the work state transitions from continuing work to completing work, as shown in the left diagram of FIG. 5, a waveform showing a transient response occurs in the force sensor value when the work state is switched. Therefore, it is preferable to set the time width of a characteristic transient response waveform that gives meaning to the state to be determined as T1. On the other hand, when the work state transitions from continuing work to not being able to continue work, as shown in the right diagram of FIG. 5, interference occurs between the gripper 56 and the sub-workpiece 62, and a waveform showing a gradual decrease in the force sensor value occurs. Therefore, it is preferable to set the time width of T2 to be longer than T1, which can capture this gradual change in the waveform. In this way, by setting an appropriate time width according to the characteristics of each of the normal system and the abnormal system in the determiner 36, the number of learning data can be relatively reduced.

[0046] The learning unit 34 determines the parameters of the determinator 36 by executing learning of the determinator 36 as described above using the learning data passed from the acquisition unit 32. Specifically, the learning unit 34 first determines T1 and T2. For example, the learning unit 34 may determine T1 and T2 by accepting T1 and T2 specified by a user based on the learning data. The learning unit 34 may also determine T1 and T2 that provide the most accurate output by executing learning while changing the time width of each of T1 and T2 by a predetermined value.

[0047] The learning unit 34 sets the determined T1 and T2 in the first and second decision units 36A and 36B, respectively, and sets the output h i (T-i) Add a temporary output layer as shown below. o i =c i +V i h i (T_i) y i ^=softmax(o i ) ,i∈[1;2]

[0048] The learning unit 34 inputs normal learning data to the first decision device 36A to which a temporary output layer has been added, and inputs abnormal learning data to the second decision device 36B to which a temporary output layer has been added. Then, the learning unit 34 inputs the output value y i The parameter U included in the input layer and intermediate layer of the classifier 36 is set so as to minimize the difference (e.g., cross entropy) between ^ and the label assigned to the training data. i , W i , and b i Determine.

[0049] Furthermore, the learning unit 34 determines T i , U i , W i , and b i After fixing , the learning data is input to the decision unit 36. Then, the learning unit 34 adjusts the parameter V included in the output layer so as to minimize the difference (e.g., cross entropy) between the output value y and the label assigned to the learning data. i and c are determined. The learning data input to the decision unit 36 ​​here may be different from the learning data used when the parameters of the input layer and the intermediate layer are determined. Furthermore, the normal learning data and the abnormal learning data used when the parameters of the input layer and the intermediate layer are determined may be randomly mixed and used.

[0050] The learning unit 34 learns the parameter T i , U i , W i , b i , V i , and the decision unit 36 ​​in which c has been determined is stored in a predetermined storage area.

[0051] The determination unit 38 inputs the target data passed from the acquisition unit 32 to the determiner 36 to determine the current work state, and passes the determination result to the control unit 40.

[0052] The control unit 40 determines the behavior of the manipulator 52 based on the working state indicated by the determination result transferred from the determination unit 38. For example, the control unit 40 may determine the behavior based on a predetermined behavior rule of the manipulator 52 according to the working state. The behavior rule may be, for example, returning to the initial posture when the working state is the work completion, or changing the main work 60 when the working state is the work cannot be continued. The control unit 40 generates control information for making the manipulator 52 execute an operation according to the determined behavior. Specifically, the control information is the rotation angle of each joint for realizing the posture of the manipulator 52 in the determined behavior. The control unit 40 outputs the generated control information to the manipulator 52.

[0053] Next, the operation of the robot system 100 according to this embodiment will be described.

[0054] When learning data is input to the work state determination device 10 and learning of the determinator 36 is instructed, a learning process is executed in the work state determination device 10. Then, when work is started by the manipulator 52 with the determinator 36 for which learning has been executed stored in a predetermined storage area, a determination control process is executed in the work state determination device 10. The learning process and the determination control process are examples of a "work state determination method" of the disclosed technology.

[0055] Fig. 7 is a flowchart showing the flow of the learning process executed by the CPU 12 of the work state determination device 10. The CPU 12 reads out the work state determination program from the storage device 16, expands it in the memory 14, and executes a process related to the learning process, whereby the CPU 12 functions as each functional component of the work state determination device 10, and the learning process shown in Fig. 7 is executed.

[0056] In step S10, the acquisition unit 32 acquires learning data including normal and abnormal systems input to the work state determination device 10, and passes the data to the learning unit 34. Next, in step S12, the learning unit 34 determines the time span T1 of the first determinator 36A and the time span T2 of the second determinator 36B, for example, by accepting T1 and T2 from a user.

[0057] Next, in step S14, the learning unit 34 sets the determined T1 and T2 in the first and second decision units 36A and 36B, respectively, and sets the output h i (T-i) A temporary output layer is added to the first classifier 36A. The learning unit 34 inputs normal learning data to the first classifier 36A to which the temporary output layer has been added, and inputs abnormal learning data to the second classifier 36B to which the temporary output layer has been added. The learning unit 34 then adjusts parameters U i , W i , and b i Determine (i∈[1;2]).

[0058] Next, in step S16, the learning unit 34 calculates the T i , U i , W i , and b i After fixing, the learning data is input to the decision unit 36. Then, the learning unit 34 determines the parameter V included in the output layer so as to minimize the difference between the output value from the output layer and the label assigned to the learning data. i and determine c.

[0059] Next, in step S18, the learning unit 34 calculates the parameter T i , U i , W i , b i , V i , and the determinator 36 for which c has been determined is stored in a predetermined storage area, and the learning process ends.

[0060] Fig. 8 is a flowchart showing the flow of the judgment control process executed by the CPU 12 of the work state judgment device 10. The CPU 12 reads out the work state judgment program from the storage device 16, expands it in the memory 14, and executes a process related to the judgment control process, whereby the CPU 12 functions as each functional component of the work state judgment device 10, and the judgment control process shown in Fig. 8 is executed.

[0061] In step S20, the acquisition unit 32 acquires the force sensor value output from the force sensor 54 and the joint angle value output from the manipulator 52. Next, in step S22, the acquisition unit 32 calculates the hand position by a matrix operation based on the acquired joint angle values ​​and kinematics information of the manipulator 52.

[0062] Next, in step S24, the acquisition unit 32 associates the force sensor value acquired in step S20 with the hand position calculated in step S22, and passes the result as target data to the determination unit 38. The determination unit 38 determines the vector value x i (t) to the neuron one time before, and the target data transferred from the acquisition unit 32 is i (t) Then, the decision unit 38 decides the current working state based on the output value from the output layer of the decision unit 36, and passes the decision result to the control unit 40.

[0063] Next, in step S26, the control unit 40 determines the action of the manipulator 52 based on the working state indicated by the determination result passed from the determination unit 38. Next, in step S28, the control unit 40 generates control information for causing the manipulator 52 to execute an operation corresponding to the determined action, and outputs the control information to the manipulator 52. Then, the determination control process ends.

[0064] As described above, in the robot system according to this embodiment, the work state determination device acquires the force sensor value and the hand position of the gripper of the manipulator. Then, the force sensor value and the hand position are input to a determiner that determines the work state of the manipulator, which is combined with a first determiner that extracts the characteristics of a normal work state and a second determiner that extracts the characteristics of an abnormal work state. Based on the output value of the determiner, the work state determination device determines whether the work state is work complete, work continues, or work cannot continue, and controls the manipulator based on the determination result. This makes it possible to determine work states including normal and abnormal states.

[0065] Furthermore, according to this embodiment, since it is possible to determine a work state in which work cannot be continued, even in the case of assembly work involving various workpieces, it is possible to promote automation of work by a robot system, such as automatically executing an action such as changing over a workpiece. Furthermore, according to this embodiment, the user is freed from feature engineering such as setting a threshold value for the force sensor value for each target workpiece, thereby reducing the burden of starting up an automatic assembly production line.

[0066] Furthermore, according to this embodiment, when the waveforms of the force sensor values ​​in the normal and abnormal systems are similar to the learning data used for learning the determiner, the work state can be determined with high accuracy even for work that targets a workpiece different from the workpiece used during learning. For example, the work state of an insertion and assembly work of an unlearned 9-pin type vehicle connector can be determined by using a determiner that has been trained using the force sensor values ​​during the insertion and assembly work of a 4-pin type vehicle connector as learning data.

[0067] Furthermore, by making the time width of the data input to the second classifier longer than the time width of the data input to the first classifier, it is possible to capture the characteristics of normal and abnormal data and accurately determine the work status.

[0068] In the above embodiment, the force sensor value and the hand position are used as data to be input to the determiner. However, the hand position is not essential, and it is sufficient to use at least the force sensor value as an input.

[0069] In addition, when both the force sensor value and the hand position are used, the determination may be made by logical product (AND) or logical sum (OR) of the force sensor value and the hand position. In the first determiner that determines whether the work should be continued or completed, it is desirable to make an AND determination in order to prevent the occurrence of defective products when an incomplete product is erroneously determined to be completed. In addition, in the second determiner that determines whether the work should be continued or cannot be continued, whether to make an AND determination or an OR determination may be set according to the content of the work and the method of dealing with the case where the work cannot be continued. In order to make such an AND determination or an OR determination, it is sufficient to perform learning of the determiner using learning data that is labeled according to the AND condition or the OR condition. When learning data that is labeled for the AND condition of the force sensor value and the hand position is used, the determiner becomes an AND determination. When learning data that is labeled for the OR condition is used, the determiner becomes an OR determination.

[0070] As the learning data, in addition to the data (original learning data) actually acquired from the force sensor or the manipulator, learning data generated pseudo using random numbers may be used. Specifically, random numbers are determined from data variation factors that may occur on-site, such as sensor noise and the scale of the force sensor value, and the pseudo learning data is generated based on the determined random numbers. For example, as shown in FIG. 9, the force sensor value, which is the original learning data, may be scaled by multiplying it by a generated random number to generate the pseudo learning data. By adding the pseudo learning data generated in this way to the original learning data and executing learning, a highly accurate classifier can be trained using a smaller amount of on-site learning data.

[0071] In the above embodiment, the functional configurations are realized by one computer, but the present invention is not limited to this. For example, the work state determination device including the acquisition unit, the learning unit, and the determination unit and the control device including the control unit may be realized by different computers. Furthermore, the work state determination device may also be realized by a learning device including the acquisition unit and the learning unit and a determination device including the acquisition unit and the determination unit, each of which may be realized by different computers.

[0072] In addition, the learning process and the judgment control process that the CPU reads and executes the software (program) in the above embodiment may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. In addition, the learning process and the judgment control process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA). In addition, the hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0073] In the above embodiment, the working state determination program is stored (installed) in advance in the storage device, but the present invention is not limited to this. The program may be provided in a form stored in a storage medium such as a CD-ROM, a DVD-ROM, a Blu-ray disc, or a USB memory. The program may be downloaded from an external device via a network.

[0074] This embodiment is applicable to industrial robots, service robots, and the like that have a function of performing automatic insertion and fitting operations, such as inserting a connector and fitting a cover part. [Explanation of symbols]

[0075] 10. Working condition determination device 12 CPU 14. Memory 16 Storage device 18 Input Devices 20 Output Devices 22 Storage media reader 24 Communication I / F 26 Bus 32 Acquisition Department 34 Learning Department 36 Judgment device 36A 1st determiner 36B 2nd judge 38 Judgment section 40 Control section 50 Robot Club 52 Manipulator 54 Force sensor 56 Gripper 60 Main Work 62 Secondary Work 100 Robot System

Claims

1. an acquisition unit that acquires data detected by a force sensor that detects a force acting on a tip portion of the robot hand; a determiner for determining a working state of the robot hand, the determiner comprising a first determiner for extracting characteristics of a normal working state and a second determiner for extracting characteristics of an abnormal working state, the determiner inputting data acquired by the acquisition unit into the determiner for outputting three working states, i.e., work completed, work continued, and work not possible to continue, to determine the working state; The time series data acquired by the acquisition unit is input to the decision unit, and a time width of the data input to the second decision unit is longer than a time width of the data input to the first decision unit. Work status determination device.

2. 2. The work status determination device according to claim 1, wherein the time width for the first determiner is a time width corresponding to a transient response of data generated when a task is completed, and the time width for the second determiner is a time width corresponding to a time from when an abnormality occurs until the operation of the robot hand is stopped.

3. The work state determination device according to claim 1 or claim 2, wherein the first classifier is trained using learning data in which, when the work state transitions from continuing work to completing work, each piece of time-series data before the transition is labeled as continuing work and each piece of data after the transition is labeled as completing work, and the second classifier is trained using learning data in which, when the work state transitions from continuing work to not being able to continue work, each piece of time-series data before the transition is labeled as continuing work and each piece of data after the transition is labeled as not being able to continue work.

4. The work state determination device according to claim 3 , further comprising a learning unit that uses the learning data to perform learning of the determiner.

5. The work state determination device according to claim 4 , wherein the learning unit generates pseudo learning data by using the learning data and a random number, and adds the pseudo learning data to the learning data for use in learning the determiner.

6. The acquisition unit further acquires a position of an end effector of the robot hand, The determination unit inputs data detected by the force sensor and the position of the hand to the determiner to determine the working state. The work state determination device according to any one of claims 1 to 5.

7. The work status determination device according to any one of claims 1 to 6, wherein the determiner includes a first neural network constituting the first determiner and a second neural network constituting the second determiner, and outputs a result of combining an output of the first neural network and an output of the second neural network.

8. The work state determination device according to any one of claims 1 to 7, further comprising a control unit that controls an operation of the robot hand based on the work state determined by the determination unit.

9. The acquisition unit acquires data detected by a force sensor that detects a force acting on a tip portion of the robot hand, a determination unit which is a determiner for determining a working state of the robot hand, the determiner being a combination of a first determiner for extracting characteristics of a normal working state and a second determiner for extracting characteristics of an abnormal working state, and which outputs three states, i.e., work completed, work continued, and work cannot be continued, as the working state, and inputs the data acquired by the acquisition unit into the determiner to determine the working state; The time series data acquired by the acquisition unit is input to the decision unit, and a time width of the data input to the second decision unit is longer than a time width of the data input to the first decision unit. Method for determining working condition.

10. Computer, An acquisition unit that acquires data detected by a force sensor that detects a force acting on a tip portion of the robot hand; and A determiner for determining a work state of the robot hand, the determiner being a combination of a first determiner for extracting characteristics of a normal work state and a second determiner for extracting characteristics of an abnormal work state, and the determiner outputs three states, work completed, work continued, and work not possible, as the work state. The determiner determines the work state by inputting the data acquired by the acquisition unit. A work state determination program for causing the work state determination program to function as The time series data acquired by the acquisition unit is input to the decision unit, and a time width of the data input to the second decision unit is longer than a time width of the data input to the first decision unit. Work status determination program.

Images